v3.5.1 changelog
quickstart →
Pricing Benchmarks Docs Changelog Blog Troubleshooting GitHub Install in 10 min

GitHub Actions runner benchmarks

At 4 vCPU, RunsOn is the best value on all three builds.

Rust spotify-player

RunsOn m8a.xlarge · EBS gp3

1m 59s $0.0034 per build

GitHub 4-core: 13× the cost, 1.9× the time, 9th of 9

TypeScript opencode

RunsOn r8a.xlarge · tmpfs

1m 07s $0.0026 per build

GitHub 4-core: 12× the cost, 2.3× the time, 8th of 8

Docker PostHog image

RunsOn r8a.xlarge · tmpfs

5m 20s $0.0115 per build

GitHub 4-core: 16× the cost, 2.8× the time, 8th of 8

Best value per provider

Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by time × cost per build.; shades are build phases in order.; build time on the right.

Rustspotify-player
  1. RunsOn4 vCPU · 16 GBm8a.xlarge · EBS gp31m 59s, cost per build spot $0.0034, on-demand $0.0096, best-value of 9 machines; phases: setup 14s, cargo test 1m 00s, clippy 34s, runner overhead 11s, ×1.00
  2. Namespace4 vCPU · 8 GB4x81m 40s, cost per build $0.0100, best-value of 2 machines; phases: setup 10s, cargo test 53s, clippy 29s, runner overhead 8s, ×2.45
  3. Ubicloud4 vCPU · 16 GBstandard2m 38s, cost per build $0.0066, best-value of 2 machines; phases: setup 22s, cargo test 1m 18s, clippy 42s, runner overhead 16s, ×2.54
  4. StarSling4 vCPU · 16 GB1m 38s, cost per build $0.0131; phases: setup 10s, cargo test 54s, clippy 29s, runner overhead 5s, ×3.13
  5. Avrea4 vCPU · 16 GB1m 40s, cost per build $0.0133; phases: setup 13s, cargo test 53s, clippy 25s, runner overhead 9s, ×3.26
  6. Blacksmith4 vCPU · 16 GB1m 51s, cost per build $0.0148; phases: setup 15s, cargo test 56s, clippy 30s, runner overhead 10s, ×4.02
  7. Warpbuild4 vCPU · 16 GB2m 23s, cost per build $0.0191; phases: setup 13s, cargo test 1m 22s, clippy 42s, runner overhead 6s, ×6.67
  8. AWS CodeBuild4 vCPU · 8 GBmedium3m 54s, cost per build $0.0390; phases: setup 32s, cargo test 1m 58s, clippy 1m 06s, runner overhead 18s, ×22
  9. GitHub4 vCPU · 16 GB3m 49s, cost per build $0.0458; phases: setup 21s, cargo fmt 1s, cargo test 2m 11s, clippy 1m 12s, runner overhead 4s, ×26

Bar shades, light to dark, in execution order: setup = checkout + packages + toolchaincargo fmtcargo testclippyrunner overhead

cost per build× = time × cost against the best

TypeScriptopencode
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs1m 07s, cost per build spot $0.0026, on-demand $0.0076, best-value of 9 machines; phases: toolchain 1s, clone 1s, bun install 10s, typecheck 38s, runner overhead 17s, ×1.00
  2. Avrea4 vCPU · 16 GB55s, cost per build $0.0073; phases: toolchain 1s, clone 3s, bun install 5s, typecheck 30s, runner overhead 16s, ×2.33
  3. StarSling4 vCPU · 16 GB57s, cost per build $0.0076; phases: toolchain 1s, clone 2s, bun install 9s, typecheck 33s, runner overhead 12s, ×2.50
  4. Ubicloud4 vCPU · 16 GBstandard1m 59s, cost per build $0.0050, best-value of 2 machines; phases: toolchain 2s, clone 5s, bun install 25s, typecheck 1m 00s, runner overhead 27s, ×3.41
  5. Warpbuild4 vCPU · 16 GB1m 23s, cost per build $0.0111; phases: toolchain 1s, clone 2s, bun install 14s, typecheck 57s, runner overhead 9s, ×5.30
  6. Namespace4 vCPU · 16 GB4x161m 09s, cost per build $0.0138; phases: toolchain 1s, clone 1s, bun install 14s, typecheck 38s, runner overhead 15s, ×5.49
  7. Blacksmith4 vCPU · 16 GB2m 06s, cost per build $0.0168; phases: toolchain 1s, clone 2s, bun install 15s, typecheck 1m 10s, runner overhead 38s, ×12
  8. GitHub4 vCPU · 16 GB2m 31s, cost per build $0.0302; phases: toolchain 1s, clone 3s, bun install 21s, typecheck 1m 35s, runner overhead 31s, ×26
  9. AWS CodeBuildfailed · out of memory

Bar shades, light to dark, in execution order: toolchainclonebun installtypecheckrunner overhead

cost per build× = time × cost against the best

DockerPostHog image
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs5m 20s, cost per build spot $0.0115, on-demand $0.0311, best-value of 8 machines; phases: fetch source 5s, buildx setup 3s, docker build 4m 45s, runner overhead 27s, ×1.00
  2. Avrea4 vCPU · 16 GB5m 18s, cost per build $0.0424; phases: fetch source 21s, buildx setup 3s, docker build 4m 28s, runner overhead 26s, ×3.67
  3. Blacksmith4 vCPU · 16 GB5m 27s, cost per build $0.0436; phases: fetch source 20s, buildx setup 2s, docker build 4m 39s, runner overhead 26s, ×3.88
  4. StarSling4 vCPU · 16 GB5m 44s, cost per build $0.0459; phases: fetch source 20s, buildx setup 4s, docker build 4m 52s, runner overhead 28s, ×4.29
  5. Namespace4 vCPU · 16 GB4x164m 54s, cost per build $0.0588; phases: fetch source 20s, buildx setup 2s, docker build 4m 18s, runner overhead 14s, ×4.70
  6. Ubicloud4 vCPU · 16 GBstandard11m 30s, cost per build $0.0288, best-value of 2 machines; phases: fetch source 14s, buildx setup 12s, docker build 10m 19s, runner overhead 45s, ×5.40
  7. Warpbuild4 vCPU · 16 GB7m 37s, cost per build $0.0609; phases: fetch source 20s, buildx setup 4s, docker build 6m 57s, runner overhead 17s, ×7.58
  8. GitHub4 vCPU · 16 GB15m 02s, cost per build $0.1804; phases: fetch source 24s, buildx setup 11s, docker build 12m 22s, runner overhead 2m 05s, ×44
  9. AWS CodeBuildnot run: needs more than 8 GB of memory

Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead

cost per build× = time × cost against the best

Each provider's best-value machine (lowest time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.

Queue and consistency

How long jobs wait in a burst, and how far build times move from run to run.

Queue under a burst15 jobs at once · each provider's quickest-starting runner at the chosen size · ranked from 3 runs
  1. GitHub 2 vCPU · 8 GB p50 3s, p10–p90 2s to 4s, min 2s, max 5s, 75 samples
  2. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 10s, min 7s, max 13s, 75 samples
  3. Namespace 2 vCPU · 8 GB 2x8 p50 9s, p10–p90 7s to 12s, min 6s, max 20s, 75 samples
  4. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  5. Avrea 8 vCPU · 32 GB p50 10s, p10–p90 8s to 13m 21s, min 7s, max 14m 35s, 75 samples
  6. Warpbuild 8 vCPU · 32 GB p50 12s, p10–p90 10s to 14m 26s, min 8s, max 15m 35s, 75 samples
  7. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 p50 15s, p10–p90 14s to 18s, min 13s, max 20s, 75 samples
  8. AWS CodeBuild 2 vCPU · 3 GB small p50 20s, p10–p90 18s to 10m 46s, min 17s, max 21m 34s, 75 samples
  9. Ubicloud 2 vCPU · 8 GB premium p50 21s, p10–p90 17s to 36s, min 13s, max 42s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 3s, p10–p90 2s to 4s, min 2s, max 5s, 75 samples
  2. Blacksmith 2 vCPU · 8 GB p50 9s, p10–p90 7s to 11s, min 7s, max 14s, 75 samples
  3. Namespace 2 vCPU · 8 GB 2x8 p50 9s, p10–p90 7s to 12s, min 6s, max 20s, 75 samples
  4. Avrea 2 vCPU · 8 GB p50 11s, p10–p90 8s to 9m 08s, min 8s, max 11m 13s, 75 samples
  5. Warpbuild 2 vCPU · 8 GB p50 13s, p10–p90 11s to 12m 30s, min 10s, max 14m 42s, 75 samples
  6. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs p50 16s, p10–p90 13s to 18s, min 12s, max 21s, 75 samples
  7. AWS CodeBuild 2 vCPU · 3 GB small p50 20s, p10–p90 18s to 10m 46s, min 17s, max 21m 34s, 75 samples
  8. Ubicloud 2 vCPU · 8 GB premium p50 21s, p10–p90 17s to 36s, min 13s, max 42s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 3s, p10–p90 2s to 4s, min 2s, max 13s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB p50 10s, p10–p90 7s to 12s, min 7s, max 21s, 75 samples
  3. Namespace 4 vCPU · 8 GB 4x8 p50 10s, p10–p90 8s to 2m 04s, min 6s, max 2m 06s, 75 samples
  4. StarSling 4 vCPU · 16 GB p50 10s, p10–p90 7s to 13m 50s, min 6s, max 17m 22s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 11s, p10–p90 9s to 10m 10s, min 9s, max 13m 06s, 75 samples
  6. Warpbuild 4 vCPU · 16 GB p50 14s, p10–p90 11s to 13m 29s, min 9s, max 16m 36s, 75 samples
  7. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 p50 15s, p10–p90 14s to 18s, min 13s, max 20s, 75 samples
  8. AWS CodeBuild 4 vCPU · 8 GB medium p50 20s, p10–p90 19s to 16m 12s, min 18s, max 21m 09s, 75 samples
  9. Ubicloud 4 vCPU · 16 GB premium p50 29s, p10–p90 16s to 5m 51s, min 15s, max 10m 54s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 10s, min 7s, max 13s, 75 samples
  2. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  3. Avrea 8 vCPU · 32 GB p50 10s, p10–p90 8s to 13m 21s, min 7s, max 14m 35s, 75 samples
  4. Namespace 8 vCPU · 32 GB 8x32 p50 10s, p10–p90 7s to 14s, min 6s, max 11m 43s, 75 samples
  5. Warpbuild 8 vCPU · 32 GB p50 12s, p10–p90 10s to 14m 26s, min 8s, max 15m 35s, 75 samples
  6. RunsOn 8 vCPU · 16 GB c8a.2xlarge · EBS gp3 2 spot-interrupted p50 16s, p10–p90 13s to 18s, min 13s, max 20s, 73 samples
  7. AWS CodeBuild 8 vCPU · 16 GB large p50 20s, p10–p90 18s to 14m 42s, min 17s, max 18m 27s, 75 samples
  8. Ubicloud 8 vCPU · 32 GB premium p50 23s, p10–p90 16s to 33s, min 15s, max 50s, 75 samples
  9. GitHub 8 vCPU · 32 GB p50 1m 16s, p10–p90 1m 06s to 1m 34s, min 3s, max 2m 12s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 7s, min 4s, max 8s, 75 samples
  2. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 11s, p10–p90 10s to 18s, min 9s, max 24s, 75 samples
  3. Blacksmith 8 vCPU · 32 GB p50 12s, p10–p90 11s to 15s, min 10s, max 15s, 75 samples
  4. Ubicloud 2 vCPU · 8 GB standard p50 12s, p10–p90 8s to 41s, min 8s, max 53s, 75 samples
  5. Warpbuild 12 vCPU · 28 GB xfast (Apple M4 Pro) p50 15s, p10–p90 14s to 18s, min 13s, max 3m 42s, 75 samples
  6. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 18s, min 14s, max 22s, 75 samples
  7. Avrea 2 vCPU · 8 GB p50 20s, p10–p90 16s to 24s, min 14s, max 28s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 7s, p10–p90 6s to 7s, min 5s, max 22s, 75 samples
  2. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 11s, p10–p90 10s to 18s, min 9s, max 24s, 75 samples
  3. Ubicloud 2 vCPU · 8 GB standard p50 12s, p10–p90 8s to 41s, min 8s, max 53s, 75 samples
  4. Blacksmith 2 vCPU · 6 GB p50 13s, p10–p90 11s to 14s, min 11s, max 15s, 75 samples
  5. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 18s, min 14s, max 22s, 75 samples
  6. Avrea 2 vCPU · 8 GB p50 20s, p10–p90 16s to 24s, min 14s, max 28s, 75 samples
  7. Warpbuild 2 vCPU · 8 GB p50 41s, p10–p90 36s to 48s, min 33s, max 1m 22s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 7s, min 4s, max 8s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB p50 13s, p10–p90 10s to 16s, min 10s, max 5m 13s, 75 samples
  3. Namespace 4 vCPU · 16 GB Apple silicon 4x16 p50 13s, p10–p90 10s to 32s, min 9s, max 36s, 75 samples
  4. RunsOn 4 vCPU · 8 GB c8g.xlarge · EBS gp3 p50 17s, p10–p90 15s to 20s, min 14s, max 39s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 21s, p10–p90 18s to 24s, min 16s, max 1m 15s, 75 samples
  6. Ubicloud 4 vCPU · 16 GB standard p50 26s, p10–p90 10s to 46s, min 8s, max 1m 12s, 75 samples
  7. Warpbuild 4 vCPU · 16 GB p50 39s, p10–p90 13s to 43s, min 6s, max 46s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 12s, p10–p90 11s to 15s, min 10s, max 15s, 75 samples
  2. RunsOn 8 vCPU · 16 GB c8g.2xlarge · EBS gp3 p50 16s, p10–p90 13s to 21s, min 13s, max 1m 02s, 75 samples
  3. Namespace 8 vCPU · 16 GB 8x16 p50 17s, p10–p90 15s to 21s, min 14s, max 23s, 75 samples
  4. Avrea 8 vCPU · 32 GB p50 21s, p10–p90 16s to 1m 50s, min 15s, max 1m 54s, 75 samples
  5. Ubicloud 8 vCPU · 32 GB standard p50 27s, p10–p90 20s to 2m 08s, min 8s, max 5m 51s, 75 samples
  6. Warpbuild 8 vCPU · 32 GB p50 38s, p10–p90 17s to 43s, min 8s, max 47s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
Build time, run to runeach provider's machine from the cards above, at the chosen size, with 3+ workflow runs · build steps time, one dot per run · % off its own p50
  1. RunsOn 4 vCPU · 16 GB m8a.xlarge · EBS gp3 spread 3.9% (p10–p90) around a p50 of 1m 47s, min 1m 46s, max 1m 51s, 5 samples
  2. AWS CodeBuild 8 vCPU · 16 GB large spread 9.5% (p10–p90) around a p50 of 2m 12s, min 2m 05s, max 2m 18s, 5 samples
  3. StarSling 4 vCPU · 16 GB spread 10.2% (p10–p90) around a p50 of 1m 32s, min 1m 29s, max 1m 42s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 12.7% (p10–p90) around a p50 of 1m 33s, min 1m 28s, max 1m 45s, 5 samples
  5. Namespace 4 vCPU · 8 GB 4x8 spread 14.9% (p10–p90) around a p50 of 1m 35s, min 1m 28s, max 1m 45s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 18.9% (p10–p90) around a p50 of 2m 24s, min 1m 55s, max 2m 35s, 5 samples
  7. Avrea 4 vCPU · 16 GB spread 24% (p10–p90) around a p50 of 1m 31s, min 1m 25s, max 1m 56s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 40.7% (p10–p90) around a p50 of 1m 59s, min 1m 49s, max 2m 53s, 5 samples
  9. Blacksmith 4 vCPU · 16 GB spread 66.6% (p10–p90) around a p50 of 1m 40s, min 1m 39s, max 3m 16s, 5 samples
one run p50 p10–p90 min–max
  1. Namespace 2 vCPU · 8 GB 2x8 spread 6.7% (p10–p90) around a p50 of 2m 36s, min 2m 25s, max 2m 39s, 5 samples
  2. RunsOn 2 vCPU · 8 GB m8a.large · EBS gp3 spread 10.5% (p10–p90) around a p50 of 2m 53s, min 2m 51s, max 3m 18s, 5 samples
  3. AWS CodeBuild 2 vCPU · 3 GB small spread 11% (p10–p90) around a p50 of 6m 35s, min 6m 06s, max 7m 03s, 5 samples
  4. Avrea 2 vCPU · 8 GB spread 23.7% (p10–p90) around a p50 of 2m 22s, min 2m 13s, max 3m 05s, 5 samples
  5. GitHub 2 vCPU · 8 GB spread 25.2% (p10–p90) around a p50 of 6m 41s, min 5m 39s, max 7m 37s, 5 samples
  6. Warpbuild 2 vCPU · 8 GB spread 31.5% (p10–p90) around a p50 of 3m 41s, min 3m 21s, max 4m 53s, 5 samples
  7. Blacksmith 2 vCPU · 8 GB spread 45.8% (p10–p90) around a p50 of 3m 55s, min 2m 40s, max 5m 04s, 5 samples
  8. Ubicloud 2 vCPU · 8 GB standard spread 66.6% (p10–p90) around a p50 of 3m 56s, min 3m 39s, max 7m 25s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m8a.xlarge · EBS gp3 spread 3.9% (p10–p90) around a p50 of 1m 47s, min 1m 46s, max 1m 51s, 5 samples
  2. AWS CodeBuild 4 vCPU · 8 GB medium spread 5.6% (p10–p90) around a p50 of 3m 35s, min 3m 28s, max 3m 44s, 5 samples
  3. StarSling 4 vCPU · 16 GB spread 10.2% (p10–p90) around a p50 of 1m 32s, min 1m 29s, max 1m 42s, 5 samples
  4. Namespace 4 vCPU · 8 GB 4x8 spread 14.9% (p10–p90) around a p50 of 1m 35s, min 1m 28s, max 1m 45s, 5 samples
  5. Warpbuild 4 vCPU · 16 GB spread 17.6% (p10–p90) around a p50 of 2m 16s, min 2m 01s, max 2m 31s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 18.9% (p10–p90) around a p50 of 2m 24s, min 1m 55s, max 2m 35s, 5 samples
  7. Avrea 4 vCPU · 16 GB spread 24% (p10–p90) around a p50 of 1m 31s, min 1m 25s, max 1m 56s, 5 samples
  8. GitHub 4 vCPU · 16 GB spread 31.6% (p10–p90) around a p50 of 3m 41s, min 3m 06s, max 4m 27s, 5 samples
  9. Blacksmith 4 vCPU · 16 GB spread 66.6% (p10–p90) around a p50 of 1m 40s, min 1m 39s, max 3m 16s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 2.1% (p10–p90) around a p50 of 1m 15s, min 1m 13s, max 1m 15s, 5 samples
  2. AWS CodeBuild 8 vCPU · 16 GB large spread 9.5% (p10–p90) around a p50 of 2m 12s, min 2m 05s, max 2m 18s, 5 samples
  3. Warpbuild 8 vCPU · 32 GB spread 12.7% (p10–p90) around a p50 of 1m 33s, min 1m 28s, max 1m 45s, 5 samples
  4. Avrea 8 vCPU · 32 GB spread 13.6% (p10–p90) around a p50 of 1m 06s, min 1m 03s, max 1m 16s, 5 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 18% (p10–p90) around a p50 of 1m 40s, min 1m 32s, max 2m 00s, 5 samples
  6. StarSling 8 vCPU · 32 GB spread 34.6% (p10–p90) around a p50 of 1m 10s, min 1m 01s, max 1m 30s, 5 samples
  7. GitHub 8 vCPU · 32 GB spread 40.7% (p10–p90) around a p50 of 1m 59s, min 1m 49s, max 2m 53s, 5 samples
  8. Namespace 8 vCPU · 16 GB 8x16 spread 42.9% (p10–p90) around a p50 of 1m 09s, min 1m 03s, max 1m 47s, 5 samples
  9. Blacksmith 8 vCPU · 32 GB spread 47.8% (p10–p90) around a p50 of 1m 30s, min 1m 08s, max 2m 07s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 4 vCPU · 16 GB spread 4.2% (p10–p90) around a p50 of 1m 16s, min 1m 15s, max 1m 19s, 5 samples
  2. RunsOn 8 vCPU · 32 GB m9g.2xlarge · EBS gp3 spread 5.9% (p10–p90) around a p50 of 1m 15s, min 1m 12s, max 1m 18s, 5 samples
  3. GitHub 4 vCPU · 16 GB spread 6% (p10–p90) around a p50 of 3m 16s, min 3m 03s, max 3m 18s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 11.4% (p10–p90) around a p50 of 1m 59s, min 1m 55s, max 2m 09s, 5 samples
  5. Blacksmith 8 vCPU · 32 GB spread 22.9% (p10–p90) around a p50 of 2m 59s, min 2m 28s, max 3m 19s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 28.6% (p10–p90) around a p50 of 5m 31s, min 4m 17s, max 6m 06s, 5 samples
  7. Namespace 8 vCPU · 16 GB 8x16 spread 136% (p10–p90) around a p50 of 3m 44s, min 3m 22s, max 11m 29s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 8 GB m9gd.large · local NVMe spread 1.9% (p10–p90) around a p50 of 2m 51s, min 2m 50s, max 2m 54s, 5 samples
  2. GitHub 2 vCPU · 8 GB spread 13.9% (p10–p90) around a p50 of 5m 16s, min 5m 11s, max 6m 09s, 5 samples
  3. Warpbuild 2 vCPU · 8 GB spread 16% (p10–p90) around a p50 of 4m 39s, min 4m 20s, max 5m 17s, 5 samples
  4. Namespace 2 vCPU · 8 GB Apple silicon 2x8 spread 16.7% (p10–p90) around a p50 of 2m 06s, min 1m 55s, max 2m 24s, 5 samples
  5. Avrea 2 vCPU · 8 GB spread 17.6% (p10–p90) around a p50 of 1m 56s, min 1m 48s, max 2m 20s, 5 samples
  6. Blacksmith 2 vCPU · 6 GB spread 25.7% (p10–p90) around a p50 of 6m 48s, min 6m 07s, max 8m 08s, 5 samples
  7. Ubicloud 2 vCPU · 8 GB standard spread 27.9% (p10–p90) around a p50 of 8m 48s, min 7m 44s, max 10m 25s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe spread 1.9% (p10–p90) around a p50 of 1m 46s, min 1m 45s, max 1m 47s, 5 samples
  2. Avrea 4 vCPU · 16 GB spread 4.2% (p10–p90) around a p50 of 1m 16s, min 1m 15s, max 1m 19s, 5 samples
  3. GitHub 4 vCPU · 16 GB spread 6% (p10–p90) around a p50 of 3m 16s, min 3m 03s, max 3m 18s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 7% (p10–p90) around a p50 of 3m 02s, min 2m 53s, max 3m 13s, 5 samples
  5. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 19.1% (p10–p90) around a p50 of 1m 26s, min 1m 15s, max 1m 37s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 27.7% (p10–p90) around a p50 of 4m 24s, min 3m 44s, max 5m 06s, 5 samples
  7. Ubicloud 4 vCPU · 16 GB standard spread 28.6% (p10–p90) around a p50 of 5m 31s, min 4m 17s, max 6m 06s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m9g.2xlarge · EBS gp3 spread 5.9% (p10–p90) around a p50 of 1m 15s, min 1m 12s, max 1m 18s, 5 samples
  2. Avrea 8 vCPU · 32 GB spread 9% (p10–p90) around a p50 of 58s, min 55s, max 1m 01s, 5 samples
  3. Warpbuild 8 vCPU · 32 GB spread 11.4% (p10–p90) around a p50 of 1m 59s, min 1m 55s, max 2m 09s, 5 samples
  4. Ubicloud 8 vCPU · 32 GB standard spread 18.2% (p10–p90) around a p50 of 3m 48s, min 3m 22s, max 4m 14s, 5 samples
  5. Blacksmith 8 vCPU · 32 GB spread 22.9% (p10–p90) around a p50 of 2m 59s, min 2m 28s, max 3m 19s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 136% (p10–p90) around a p50 of 3m 44s, min 3m 22s, max 11m 29s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 8 vCPU · 32 GB p50 26s, min 25s, max 27s, 3 samples, not ranked
  2. AWS CodeBuild 8 vCPU · 16 GB large p50 1m 01s, min 1m 01s, max 1m 06s, 3 samples, not ranked
  3. StarSling 4 vCPU · 16 GB p50 45s, min 44s, max 48s, 3 samples, not ranked
  4. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 36s, min 31s, max 36s, 3 samples, not ranked
  5. GitHub 8 vCPU · 32 GB p50 1m 10s, min 1m 05s, max 1m 16s, 3 samples, not ranked
  6. Warpbuild 4 vCPU · 16 GB p50 1m 13s, min 56s, max 1m 14s, 3 samples, not ranked
  7. Ubicloud 4 vCPU · 16 GB standard p50 1m 44s, min 1m 26s, max 1m 52s, 3 samples, not ranked
  8. Blacksmith 8 vCPU · 32 GB p50 46s, min 38s, max 54s, 3 samples, not ranked
  9. Namespace 8 vCPU · 16 GB 8x16 p50 28s, min 28s, max 1m 30s, 3 samples, not ranked
one run p50 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs p50 1m 29s, min 1m 25s, max 1m 38s, 3 samples, not ranked
  2. Blacksmith 2 vCPU · 8 GB p50 2m 09s, min 1m 46s, max 2m 36s, 3 samples, not ranked
  3. Ubicloud 2 vCPU · 8 GB standard p50 3m 36s, min 2m 55s, max 4m 44s, 3 samples, not ranked
  4. Warpbuild 2 vCPU · 8 GB p50 2m 37s, min 2m 05s, max 3m 31s, 3 samples, not ranked
one run p50 min–max
2 providers have under 3 runs so far

Avrea and GitHub.

  1. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs p50 50s, min 49s, max 50s, 3 samples, not ranked
  2. Avrea 4 vCPU · 16 GB p50 40s, min 39s, max 40s, 3 samples, not ranked
  3. StarSling 4 vCPU · 16 GB p50 45s, min 44s, max 48s, 3 samples, not ranked
  4. GitHub 4 vCPU · 16 GB p50 2m 00s, min 1m 44s, max 2m 01s, 3 samples, not ranked
  5. Warpbuild 4 vCPU · 16 GB p50 1m 13s, min 56s, max 1m 14s, 3 samples, not ranked
  6. Ubicloud 4 vCPU · 16 GB standard p50 1m 44s, min 1m 26s, max 1m 52s, 3 samples, not ranked
  7. Blacksmith 4 vCPU · 16 GB p50 1m 28s, min 1m 21s, max 1m 44s, 3 samples, not ranked
  8. Namespace 4 vCPU · 16 GB 4x16 p50 54s, min 42s, max 1m 00s, 3 samples, not ranked
one run p50 min–max
  1. StarSling 8 vCPU · 32 GB p50 31s, min 31s, max 31s, 3 samples, not ranked
  2. Avrea 8 vCPU · 32 GB p50 26s, min 25s, max 27s, 3 samples, not ranked
  3. AWS CodeBuild 8 vCPU · 16 GB large p50 1m 01s, min 1m 01s, max 1m 06s, 3 samples, not ranked
  4. Warpbuild 8 vCPU · 32 GB p50 41s, min 38s, max 42s, 3 samples, not ranked
  5. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 36s, min 31s, max 36s, 3 samples, not ranked
  6. GitHub 8 vCPU · 32 GB p50 1m 10s, min 1m 05s, max 1m 16s, 3 samples, not ranked
  7. Ubicloud 8 vCPU · 32 GB premium p50 58s, min 50s, max 1m 01s, 3 samples, not ranked
  8. Blacksmith 8 vCPU · 32 GB p50 46s, min 38s, max 54s, 3 samples, not ranked
  9. Namespace 8 vCPU · 16 GB 8x16 p50 28s, min 28s, max 1m 30s, 3 samples, not ranked
one run p50 min–max
  1. Warpbuild 4 vCPU · 16 GB p50 1m 14s, min 1m 13s, max 1m 16s, 3 samples, not ranked
  2. GitHub 4 vCPU · 16 GB p50 1m 25s, min 1m 25s, max 1m 30s, 3 samples, not ranked
  3. Blacksmith 8 vCPU · 32 GB p50 1m 10s, min 1m 09s, max 1m 15s, 3 samples, not ranked
  4. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe p50 32s, min 30s, max 34s, 3 samples, not ranked
  5. Avrea 4 vCPU · 16 GB p50 41s, min 39s, max 45s, 3 samples, not ranked
  6. Namespace 8 vCPU · 16 GB 8x16 p50 1m 46s, min 1m 35s, max 2m 02s, 3 samples, not ranked
  7. Ubicloud 4 vCPU · 16 GB standard p50 2m 21s, min 2m 08s, max 24m 16s, 3 samples, not ranked
one run p50 min–max
  1. RunsOn 2 vCPU · 16 GB r9g.large · tmpfs p50 1m 27s, min 1m 25s, max 1m 30s, 3 samples, not ranked
  2. Blacksmith 2 vCPU · 6 GB p50 4m 52s, min 4m 33s, max 5m 13s, 3 samples, not ranked
  3. Warpbuild 2 vCPU · 8 GB p50 2m 57s, min 2m 28s, max 3m 15s, 3 samples, not ranked
one run p50 min–max
2 providers have under 3 runs so far

Avrea and GitHub.

  1. Warpbuild 4 vCPU · 16 GB p50 1m 14s, min 1m 13s, max 1m 16s, 3 samples, not ranked
  2. GitHub 4 vCPU · 16 GB p50 1m 25s, min 1m 25s, max 1m 30s, 3 samples, not ranked
  3. Blacksmith 4 vCPU · 16 GB p50 1m 57s, min 1m 54s, max 2m 01s, 3 samples, not ranked
  4. RunsOn 4 vCPU · 16 GB m9g.xlarge · tmpfs p50 48s, min 48s, max 53s, 3 samples, not ranked
  5. Avrea 4 vCPU · 16 GB p50 41s, min 39s, max 45s, 3 samples, not ranked
  6. Namespace 4 vCPU · 16 GB Apple silicon 4x16 p50 43s, min 42s, max 1m 31s, 3 samples, not ranked
  7. Ubicloud 4 vCPU · 16 GB standard p50 2m 21s, min 2m 08s, max 24m 16s, 3 samples, not ranked
one run p50 min–max
  1. Avrea 8 vCPU · 32 GB p50 30s, min 30s, max 31s, 3 samples, not ranked
  2. Blacksmith 8 vCPU · 32 GB p50 1m 10s, min 1m 09s, max 1m 15s, 3 samples, not ranked
  3. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe p50 32s, min 30s, max 34s, 3 samples, not ranked
  4. Warpbuild 8 vCPU · 32 GB p50 50s, min 43s, max 50s, 3 samples, not ranked
  5. Namespace 8 vCPU · 16 GB 8x16 p50 1m 46s, min 1m 35s, max 2m 02s, 3 samples, not ranked
  6. Ubicloud 8 vCPU · 32 GB standard p50 1m 28s, min 1m 23s, max 1m 46s, 3 samples, not ranked
one run p50 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 2.9% (p10–p90) around a p50 of 3m 37s, min 3m 32s, max 3m 41s, 5 samples
  2. Namespace 8 vCPU · 16 GB 8x16 spread 5.3% (p10–p90) around a p50 of 3m 37s, min 3m 27s, max 3m 41s, 5 samples
  3. Avrea 4 vCPU · 16 GB p50 4m 52s, min 4m 41s, max 5m 03s, 4 samples, not ranked
  4. StarSling 4 vCPU · 16 GB spread 10% (p10–p90) around a p50 of 5m 16s, min 5m 15s, max 5m 57s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 15.5% (p10–p90) around a p50 of 11m 59s, min 10m 57s, max 13m 27s, 5 samples
  6. Warpbuild 8 vCPU · 32 GB spread 24.6% (p10–p90) around a p50 of 4m 58s, min 4m 39s, max 6m 36s, 5 samples
  7. Blacksmith 4 vCPU · 16 GB spread 29.2% (p10–p90) around a p50 of 5m 01s, min 4m 38s, max 6m 49s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 31.3% (p10–p90) around a p50 of 7m 23s, min 6m 22s, max 8m 55s, 5 samples
  9. Ubicloud 4 vCPU · 16 GB standard spread 54.5% (p10–p90) around a p50 of 10m 45s, min 6m 51s, max 15m 54s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 13% (p10–p90) around a p50 of 11m 16s, min 10m 57s, max 13m 05s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 4 vCPU · 16 GB p50 4m 52s, min 4m 41s, max 5m 03s, 4 samples, not ranked
  2. StarSling 4 vCPU · 16 GB spread 10% (p10–p90) around a p50 of 5m 16s, min 5m 15s, max 5m 57s, 5 samples
  3. Warpbuild 4 vCPU · 16 GB p50 7m 19s, min 6m 35s, max 7m 46s, 4 samples, not ranked
  4. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 13.2% (p10–p90) around a p50 of 4m 57s, min 4m 46s, max 5m 48s, 5 samples
  5. GitHub 4 vCPU · 16 GB spread 22.4% (p10–p90) around a p50 of 12m 53s, min 11m 17s, max 15m 13s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 29.2% (p10–p90) around a p50 of 5m 01s, min 4m 38s, max 6m 49s, 5 samples
  7. Namespace 4 vCPU · 16 GB 4x16 spread 45.9% (p10–p90) around a p50 of 4m 40s, min 4m 30s, max 7m 56s, 5 samples
  8. Ubicloud 4 vCPU · 16 GB standard spread 54.5% (p10–p90) around a p50 of 10m 45s, min 6m 51s, max 15m 54s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 2.9% (p10–p90) around a p50 of 3m 37s, min 3m 32s, max 3m 41s, 5 samples
  2. Namespace 8 vCPU · 16 GB 8x16 spread 5.3% (p10–p90) around a p50 of 3m 37s, min 3m 27s, max 3m 41s, 5 samples
  3. Avrea 8 vCPU · 32 GB spread 15.4% (p10–p90) around a p50 of 3m 45s, min 3m 26s, max 4m 13s, 5 samples
  4. AWS CodeBuild 8 vCPU · 16 GB large spread 15.5% (p10–p90) around a p50 of 11m 59s, min 10m 57s, max 13m 27s, 5 samples
  5. StarSling 8 vCPU · 32 GB spread 20.8% (p10–p90) around a p50 of 4m 07s, min 3m 55s, max 5m 10s, 5 samples
  6. Ubicloud 8 vCPU · 32 GB standard spread 23.4% (p10–p90) around a p50 of 7m 38s, min 7m 01s, max 9m 33s, 5 samples
  7. Warpbuild 8 vCPU · 32 GB spread 24.6% (p10–p90) around a p50 of 4m 58s, min 4m 39s, max 6m 36s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 31.3% (p10–p90) around a p50 of 7m 23s, min 6m 22s, max 8m 55s, 5 samples
  9. Blacksmith 8 vCPU · 32 GB spread 47.2% (p10–p90) around a p50 of 3m 52s, min 3m 36s, max 6m 14s, 5 samples
one run p50 p10–p90 min–max
  1. Namespace 8 vCPU · 16 GB 8x16 spread 4.1% (p10–p90) around a p50 of 9m 58s, min 9m 39s, max 10m 14s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 6.2% (p10–p90) around a p50 of 9m 45s, min 9m 25s, max 10m 07s, 5 samples
  3. Ubicloud 4 vCPU · 16 GB standard spread 13% (p10–p90) around a p50 of 13m 43s, min 12m 38s, max 15m 05s, 5 samples
  4. Avrea 4 vCPU · 16 GB spread 14.9% (p10–p90) around a p50 of 4m 46s, min 4m 39s, max 5m 30s, 5 samples
  5. Blacksmith 8 vCPU · 32 GB spread 24.6% (p10–p90) around a p50 of 7m 34s, min 6m 53s, max 9m 01s, 5 samples
  6. Warpbuild 8 vCPU · 32 GB spread 28.2% (p10–p90) around a p50 of 6m 30s, min 6m 02s, max 8m 51s, 5 samples
  7. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 54.9% (p10–p90) around a p50 of 4m 07s, min 3m 45s, max 7m 21s, 5 samples
one run p50 p10–p90 min–max

No arm64 Docker build finished at 2 vCPU.

  1. GitHub 4 vCPU · 16 GB spread 6.2% (p10–p90) around a p50 of 9m 45s, min 9m 25s, max 10m 07s, 5 samples
  2. Ubicloud 4 vCPU · 16 GB standard spread 13% (p10–p90) around a p50 of 13m 43s, min 12m 38s, max 15m 05s, 5 samples
  3. Avrea 4 vCPU · 16 GB spread 14.9% (p10–p90) around a p50 of 4m 46s, min 4m 39s, max 5m 30s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 15.9% (p10–p90) around a p50 of 10m 02s, min 9m 34s, max 12m 00s, 5 samples
  5. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 19.8% (p10–p90) around a p50 of 5m 05s, min 4m 40s, max 6m 04s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 25.9% (p10–p90) around a p50 of 11m 10s, min 9m 46s, max 13m 03s, 5 samples
  7. RunsOn 4 vCPU · 32 GB r9g.xlarge · tmpfs spread 40.3% (p10–p90) around a p50 of 5m 20s, min 5m 10s, max 8m 42s, 5 samples
one run p50 p10–p90 min–max
  1. Namespace 8 vCPU · 16 GB 8x16 spread 4.1% (p10–p90) around a p50 of 9m 58s, min 9m 39s, max 10m 14s, 5 samples
  2. Ubicloud 8 vCPU · 32 GB standard spread 13.4% (p10–p90) around a p50 of 10m 23s, min 9m 56s, max 11m 36s, 5 samples
  3. Avrea 8 vCPU · 32 GB spread 15.2% (p10–p90) around a p50 of 3m 41s, min 3m 30s, max 4m 16s, 5 samples
  4. Blacksmith 8 vCPU · 32 GB spread 24.6% (p10–p90) around a p50 of 7m 34s, min 6m 53s, max 9m 01s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 28.2% (p10–p90) around a p50 of 6m 30s, min 6m 02s, max 8m 51s, 5 samples
  6. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 54.9% (p10–p90) around a p50 of 4m 07s, min 3m 45s, max 7m 21s, 5 samples
one run p50 p10–p90 min–max

Is a bigger runner always worth it?

What doubling the vCPU did to each build, on each provider's own runner line: each step usually saves less time and costs more per build.

Median across providers: from 4 to 8 vCPU, each build ran 25–28% faster for 49–51% more per build; from 2 to 4 vCPU, 39–51% faster, cost per build −3% to +16%.

Median across providers: from 4 to 8 vCPU, each build ran 25–28% faster for 43–51% more per build; from 2 to 4 vCPU, 39–51% faster, cost per build −3% to +22%.

Median across providers: from 4 to 8 vCPU, each build ran 26–29% faster for 39–49% more per build; from 2 to 4 vCPU, 36–56% faster, cost per build −13% to +27%.

Median across providers: from 4 to 8 vCPU, each build ran 26–29% faster for 42–49% more per build; from 2 to 4 vCPU, 36–56% faster, cost per build −13% to +27%.

  • Each line is one provider's runner at 2, 4 and 8 vCPU, its vCPU count in each dot (hollow: under 3 runs so far); RunsOn's is the bold one (m8a / m9g, not its fastest or cheapest).
  • Right costs more per build, lower is faster: the shaded corner, bottom left, is where you want to be.
  • Dashed lines join equal time × cost. A step that drops more steeply than them bought time cheaply; one that runs flatter mostly bought cost.
Rustspotify-player
faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingAvreaBlacksmithRunsOnWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 28% faster, +50% cost (9 providers)
2 → 4 vCPU: 39% faster, +16% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingAvreaBlacksmithRunsOnWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 28% faster, +43% cost (9 providers)
2 → 4 vCPU: 39% faster, +22% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +39% cost (6 providers)
2 → 4 vCPU: 36% faster, +27% cost (7 providers)

faster & cheaper$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +42% cost (6 providers)
2 → 4 vCPU: 36% faster, +27% cost (7 providers)

TypeScriptopencode
faster & cheaper$0.0020$0.0050$0.0100$0.02001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingNamespaceRunsOnWarpbuildUbicloudBlacksmithGitHub

Median across providers: 4 → 8 vCPU: 25% faster, +51% cost (8 providers)
2 → 4 vCPU: 51% faster, −3% cost (3 providers)

faster & cheaper$0.0050$0.0100$0.02001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingNamespaceRunsOnWarpbuildUbicloudBlacksmithGitHub

Median across providers: 4 → 8 vCPU: 25% faster, +51% cost (8 providers)
2 → 4 vCPU: 51% faster, −3% cost (3 providers)

faster & cheaper$0.0020$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithUbicloudNamespace

Median across providers: 4 → 8 vCPU: 26% faster, +49% cost (6 providers)
2 → 4 vCPU: 56% faster, −13% cost (3 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithUbicloudNamespace

Median across providers: 4 → 8 vCPU: 26% faster, +49% cost (6 providers)
2 → 4 vCPU: 56% faster, −13% cost (3 providers)

DockerPostHog image
faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceAvreaBlacksmithStarSlingWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 26% faster, +49% cost (8 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceAvreaBlacksmithStarSlingWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 26% faster, +49% cost (8 providers)

faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 27% faster, +47% cost (6 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 27% faster, +47% cost (6 providers)

Beyond builds

The same runners, one question each.

  • CPU Fastest single-thread on x64: Namespace 4x8, 4,668 PassMark.
  • Cache 4 GB restore on x64, any size: RunsOn 8s (12 vCPU), then Blacksmith 11s (2 vCPU); GitHub's own service 31s (8 vCPU).4 GB restore on x64 at 2 vCPU: Blacksmith 11s, then Avrea 12s; GitHub's own service 1m 03s.4 GB restore on x64 at 4 vCPU: Avrea 11s, then Blacksmith 13s; GitHub's own service 46s.4 GB restore on x64 at 8 vCPU: Avrea 11s, then RunsOn 12s; GitHub's own service 31s.
  • Disk Small random reads (QD1) on x64: Namespace 2x8 leads with 104,659 per second, 59× the slowest disk (AWS CodeBuild, 1,761). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
  • CPU Fastest single-thread on arm64: Avrea 2 vCPU, 5,000 PassMark.
  • Cache 4 GB restore on arm64, any size: Avrea 11s (4 vCPU), then RunsOn 13s (8 vCPU); GitHub's own service 30s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then RunsOn 24s; GitHub's own service 53s.4 GB restore on arm64 at 4 vCPU: Avrea 11s, then RunsOn 18s; GitHub's own service 30s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 13s.
  • Disk Small random reads (QD1) on arm64: Warpbuild xfast (Apple M4 Pro) 12 vCPU leads with 29,545 per second, 3.8× the slowest disk (GitHub, 7,775). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.

Providers

Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.

Head-to-head

Two runners of the same size, build for build. Cost per build decides: a faster runner that costs twice as much per build is not the better runner.

Cost per build is variable cost only; licenses and plan fees are flat, on each provider's page. Estimate your bill