v3.6 changelog
quickstart →
Pricing Benchmarks Docs Changelog Blog Troubleshooting GitHub Install in 10 min

GitHub Actions runner benchmarks

At 4 vCPU, RunsOn is the best value on all three builds.

Rust spotify-player

RunsOn m8a.xlarge · EBS gp3

2m 00s $0.0034 per build

GitHub 4-core: 13× the cost, 1.9× the time, 9th of 9

TypeScript opencode

RunsOn r8a.xlarge · tmpfs

1m 04s $0.0025 per build

GitHub 4-core: 11× the cost, 2.3× the time, 8th of 8

Docker PostHog image

RunsOn r8a.xlarge · tmpfs

5m 20s $0.0115 per build

GitHub 4-core: 16× the cost, 2.9× the time, 8th of 8

Best value per provider

Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by time × cost per build.; shades are build phases in order.; build time on the right.

Rustspotify-player
  1. RunsOn4 vCPU · 16 GBm8a.xlarge · EBS gp32m 00s, cost per build spot $0.0034, on-demand $0.0096, best-value of 9 machines; phases: setup 14s, cargo test 1m 00s, clippy 34s, runner overhead 12s, ×1.00
  2. Ubicloud4 vCPU · 16 GBstandard2m 33s, cost per build $0.0064, best-value of 2 machines; phases: setup 20s, cargo test 1m 16s, clippy 41s, runner overhead 16s, ×2.37
  3. Namespace4 vCPU · 8 GB4x81m 46s, cost per build $0.0106, best-value of 2 machines; phases: setup 9s, cargo test 56s, clippy 29s, runner overhead 12s, ×2.73
  4. Avrea4 vCPU · 16 GB1m 35s, cost per build $0.0127; phases: setup 13s, cargo test 49s, clippy 25s, runner overhead 8s, ×2.92
  5. StarSling4 vCPU · 16 GB1m 38s, cost per build $0.0131; phases: setup 10s, cargo test 54s, clippy 29s, runner overhead 5s, ×3.11
  6. Blacksmith4 vCPU · 16 GB1m 51s, cost per build $0.0148; phases: setup 17s, cargo test 56s, clippy 30s, runner overhead 8s, ×3.99
  7. Warpbuild4 vCPU · 16 GB2m 15s, cost per build $0.0180; phases: setup 12s, cargo test 1m 15s, clippy 42s, runner overhead 6s, ×5.90
  8. AWS CodeBuild4 vCPU · 8 GBmedium3m 48s, cost per build $0.0380; phases: setup 33s, cargo test 1m 55s, clippy 1m 04s, runner overhead 16s, ×21
  9. GitHub4 vCPU · 16 GB3m 43s, cost per build $0.0446; phases: setup 20s, cargo test 2m 04s, clippy 1m 07s, runner overhead 12s, ×24

Bar shades, light to dark, in execution order: setup = checkout + packages + toolchaincargo fmtcargo testclippyrunner overhead

cost per build× = time × cost against the best

TypeScriptopencode
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs1m 04s, cost per build spot $0.0025, on-demand $0.0076, best-value of 9 machines; phases: toolchain 1s, clone 1s, bun install 9s, typecheck 38s, runner overhead 15s, ×1.00
  2. Avrea4 vCPU · 16 GB53s, cost per build $0.0070; phases: toolchain 1s, clone 3s, bun install 5s, typecheck 30s, runner overhead 14s, ×2.27
  3. StarSling4 vCPU · 16 GB59s, cost per build $0.0078; phases: toolchain 1s, clone 2s, bun install 9s, typecheck 34s, runner overhead 13s, ×2.82
  4. Ubicloud4 vCPU · 16 GBstandard1m 52s, cost per build $0.0047, best-value of 2 machines; phases: toolchain 2s, clone 5s, bun install 25s, typecheck 59s, runner overhead 22s, ×3.23
  5. Namespace4 vCPU · 16 GB4x161m 01s, cost per build $0.0122; phases: toolchain 1s, clone 1s, bun install 13s, typecheck 36s, runner overhead 12s, ×4.59
  6. Warpbuild4 vCPU · 16 GB1m 21s, cost per build $0.0108; phases: toolchain 1s, clone 2s, bun install 14s, typecheck 52s, runner overhead 14s, ×5.40
  7. Blacksmith4 vCPU · 16 GB2m 02s, cost per build $0.0163; phases: toolchain 1s, clone 2s, bun install 13s, typecheck 1m 08s, runner overhead 38s, ×12
  8. GitHub4 vCPU · 16 GB2m 25s, cost per build $0.0290; phases: toolchain 1s, clone 3s, bun install 23s, typecheck 1m 23s, runner overhead 36s, ×26
  9. AWS CodeBuildfailed · out of memory

Bar shades, light to dark, in execution order: toolchainclonebun installtypecheckrunner overhead

cost per build× = time × cost against the best

DockerPostHog image
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs5m 20s, cost per build spot $0.0115, on-demand $0.0311, best-value of 8 machines; phases: fetch source 5s, buildx setup 3s, docker build 4m 45s, runner overhead 27s, ×1.00
  2. Avrea4 vCPU · 16 GB5m 21s, cost per build $0.0428; phases: fetch source 21s, buildx setup 3s, docker build 4m 25s, runner overhead 32s, ×3.74
  3. Blacksmith4 vCPU · 16 GB5m 27s, cost per build $0.0436; phases: fetch source 20s, buildx setup 2s, docker build 4m 39s, runner overhead 26s, ×3.88
  4. StarSling4 vCPU · 16 GB5m 56s, cost per build $0.0475; phases: fetch source 20s, buildx setup 4s, docker build 5m 04s, runner overhead 28s, ×4.60
  5. Namespace4 vCPU · 16 GB4x164m 54s, cost per build $0.0588; phases: fetch source 20s, buildx setup 2s, docker build 4m 18s, runner overhead 14s, ×4.71
  6. Ubicloud4 vCPU · 16 GBstandard11m 30s, cost per build $0.0288, best-value of 2 machines; phases: fetch source 18s, buildx setup 11s, docker build 10m 08s, runner overhead 53s, ×5.40
  7. Warpbuild4 vCPU · 16 GB7m 45s, cost per build $0.0620; phases: fetch source 20s, buildx setup 4s, docker build 7m 05s, runner overhead 16s, ×7.85
  8. GitHub4 vCPU · 16 GB15m 38s, cost per build $0.1876; phases: fetch source 25s, buildx setup 11s, docker build 12m 22s, runner overhead 2m 40s, ×48
  9. AWS CodeBuildnot run: needs more than 8 GB of memory

Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead

cost per build× = time × cost against the best

Each provider's best-value machine (lowest time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.

Queue and consistency

How long jobs wait in a burst, and how far build times move from run to run.

Queue under a burst15 jobs at once · each provider's quickest-starting runner at the chosen size · ranked from 3 runs
  1. GitHub 2 vCPU · 8 GB p50 3s, p10–p90 3s to 5s, min 2s, max 6s, 75 samples
  2. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 10s, min 6s, max 13s, 75 samples
  3. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  4. Namespace 2 vCPU · 8 GB 2x8 p50 10s, p10–p90 7s to 14s, min 6s, max 20s, 75 samples
  5. Avrea 2 vCPU · 8 GB p50 11s, p10–p90 9s to 9m 08s, min 8s, max 11m 13s, 75 samples
  6. Warpbuild 8 vCPU · 32 GB p50 12s, p10–p90 9s to 14m 26s, min 8s, max 15m 35s, 75 samples
  7. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs p50 15s, p10–p90 13s to 18s, min 12s, max 21s, 75 samples
  8. AWS CodeBuild 2 vCPU · 3 GB small p50 21s, p10–p90 19s to 10m 46s, min 18s, max 21m 34s, 75 samples
  9. Ubicloud 8 vCPU · 32 GB premium p50 23s, p10–p90 16s to 33s, min 15s, max 50s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 3s, p10–p90 3s to 5s, min 2s, max 6s, 75 samples
  2. Blacksmith 2 vCPU · 8 GB p50 10s, p10–p90 8s to 12s, min 8s, max 14s, 75 samples
  3. Namespace 2 vCPU · 8 GB 2x8 p50 10s, p10–p90 7s to 14s, min 6s, max 20s, 75 samples
  4. Avrea 2 vCPU · 8 GB p50 11s, p10–p90 9s to 9m 08s, min 8s, max 11m 13s, 75 samples
  5. Warpbuild 2 vCPU · 8 GB p50 13s, p10–p90 10s to 12m 30s, min 9s, max 14m 42s, 75 samples
  6. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs p50 15s, p10–p90 13s to 18s, min 12s, max 21s, 75 samples
  7. AWS CodeBuild 2 vCPU · 3 GB small p50 21s, p10–p90 19s to 10m 46s, min 18s, max 21m 34s, 75 samples
  8. Ubicloud 2 vCPU · 8 GB premium p50 27s, p10–p90 17s to 37s, min 13s, max 51s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 3s, p10–p90 3s to 5s, min 2s, max 13s, 75 samples
  2. StarSling 4 vCPU · 16 GB p50 9s, p10–p90 7s to 13m 50s, min 6s, max 17m 22s, 75 samples
  3. Blacksmith 4 vCPU · 16 GB p50 10s, p10–p90 9s to 13s, min 7s, max 21s, 75 samples
  4. Namespace 4 vCPU · 8 GB 4x8 p50 10s, p10–p90 7s to 2m 04s, min 6s, max 2m 06s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 11s, p10–p90 9s to 10m 10s, min 9s, max 13m 06s, 75 samples
  6. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 p50 16s, p10–p90 14s to 19s, min 13s, max 24s, 75 samples
  7. AWS CodeBuild 4 vCPU · 8 GB medium p50 21s, p10–p90 19s to 16m 12s, min 18s, max 21m 09s, 75 samples
  8. Warpbuild 4 vCPU · 16 GB p50 21s, p10–p90 11s to 13m 29s, min 9s, max 16m 36s, 75 samples
  9. Ubicloud 4 vCPU · 16 GB premium p50 34s, p10–p90 16s to 5m 51s, min 14s, max 10m 54s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 10s, min 6s, max 13s, 75 samples
  2. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  3. Namespace 8 vCPU · 32 GB 8x32 p50 10s, p10–p90 8s to 15s, min 6s, max 11m 43s, 75 samples
  4. Avrea 8 vCPU · 32 GB p50 12s, p10–p90 8s to 13m 21s, min 7s, max 14m 35s, 75 samples
  5. Warpbuild 8 vCPU · 32 GB p50 12s, p10–p90 9s to 14m 26s, min 8s, max 15m 35s, 75 samples
  6. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 16s, p10–p90 14s to 18s, min 13s, max 20s, 75 samples
  7. AWS CodeBuild 8 vCPU · 16 GB large p50 21s, p10–p90 18s to 14m 42s, min 17s, max 18m 27s, 75 samples
  8. Ubicloud 8 vCPU · 32 GB premium p50 23s, p10–p90 16s to 33s, min 15s, max 50s, 75 samples
  9. GitHub 8 vCPU · 32 GB p50 1m 13s, p10–p90 4s to 1m 26s, min 3s, max 2m 04s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 8s, min 4s, max 16s, 75 samples
  2. Blacksmith 8 vCPU · 32 GB p50 12s, p10–p90 11s to 15s, min 10s, max 19s, 75 samples
  3. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 13s, p10–p90 10s to 18s, min 9s, max 24s, 75 samples
  4. Warpbuild 6 vCPU · 14 GB xfast (Apple M4 Pro) p50 15s, p10–p90 14s to 18s, min 13s, max 25s, 75 samples
  5. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 19s, min 14s, max 20s, 75 samples
  6. Avrea 2 vCPU · 8 GB p50 19s, p10–p90 16s to 23s, min 14s, max 28s, 75 samples
  7. Ubicloud 2 vCPU · 8 GB standard p50 28s, p10–p90 9s to 59s, min 8s, max 1m 09s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 7s, p10–p90 6s to 7s, min 5s, max 22s, 75 samples
  2. Blacksmith 2 vCPU · 6 GB p50 13s, p10–p90 11s to 14s, min 10s, max 15s, 75 samples
  3. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 13s, p10–p90 10s to 18s, min 9s, max 24s, 75 samples
  4. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 19s, min 14s, max 20s, 75 samples
  5. Avrea 2 vCPU · 8 GB p50 19s, p10–p90 16s to 23s, min 14s, max 28s, 75 samples
  6. Ubicloud 2 vCPU · 8 GB standard p50 28s, p10–p90 9s to 59s, min 8s, max 1m 09s, 75 samples
  7. Warpbuild 2 vCPU · 8 GB p50 42s, p10–p90 38s to 48s, min 36s, max 1m 22s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 8s, min 4s, max 16s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB p50 13s, p10–p90 10s to 16s, min 9s, max 5m 13s, 75 samples
  3. Namespace 4 vCPU · 16 GB Apple silicon 4x16 p50 13s, p10–p90 10s to 32s, min 9s, max 36s, 75 samples
  4. RunsOn 4 vCPU · 8 GB c8g.xlarge · EBS gp3 p50 17s, p10–p90 15s to 20s, min 14s, max 24s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 20s, p10–p90 16s to 23s, min 16s, max 1m 15s, 75 samples
  6. Ubicloud 4 vCPU · 16 GB standard p50 33s, p10–p90 10s to 1m 24s, min 8s, max 1m 50s, 75 samples
  7. Warpbuild 4 vCPU · 16 GB p50 40s, p10–p90 9s to 44s, min 6s, max 48s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 12s, p10–p90 11s to 15s, min 10s, max 19s, 75 samples
  2. RunsOn 8 vCPU · 16 GB c8g.2xlarge · EBS gp3 p50 16s, p10–p90 13s to 22s, min 13s, max 1m 02s, 75 samples
  3. Namespace 8 vCPU · 16 GB 8x16 p50 18s, p10–p90 15s to 22s, min 14s, max 28s, 75 samples
  4. Avrea 8 vCPU · 32 GB p50 21s, p10–p90 16s to 1m 41s, min 15s, max 1m 54s, 75 samples
  5. Ubicloud 8 vCPU · 32 GB standard p50 36s, p10–p90 20s to 3m 32s, min 8s, max 5m 51s, 75 samples
  6. Warpbuild 8 vCPU · 32 GB p50 41s, p10–p90 21s to 44s, min 7s, max 47s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
Build time, run to runeach provider's machine from the cards above, at the chosen size, with 3+ workflow runs · build steps time, one dot per run · % off its own p50
  1. RunsOn 4 vCPU · 16 GB m8a.xlarge · EBS gp3 spread 3.6% (p10–p90) around a p50 of 1m 47s, min 1m 46s, max 1m 51s, 5 samples
  2. StarSling 4 vCPU · 16 GB spread 7% (p10–p90) around a p50 of 1m 32s, min 1m 29s, max 1m 37s, 5 samples
  3. Avrea 4 vCPU · 16 GB spread 8.6% (p10–p90) around a p50 of 1m 26s, min 1m 25s, max 1m 34s, 5 samples
  4. AWS CodeBuild 8 vCPU · 16 GB large spread 9.5% (p10–p90) around a p50 of 2m 12s, min 2m 05s, max 2m 18s, 5 samples
  5. Warpbuild 4 vCPU · 16 GB spread 17.4% (p10–p90) around a p50 of 2m 09s, min 2m 01s, max 2m 31s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 31.2% (p10–p90) around a p50 of 2m 19s, min 1m 42s, max 2m 35s, 5 samples
  7. Namespace 8 vCPU · 16 GB 8x16 spread 44.3% (p10–p90) around a p50 of 1m 05s, min 1m 03s, max 1m 47s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 45% (p10–p90) around a p50 of 1m 59s, min 1m 49s, max 2m 53s, 5 samples
  9. Blacksmith 4 vCPU · 16 GB spread 62.2% (p10–p90) around a p50 of 1m 40s, min 1m 39s, max 3m 16s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 8 GB m8a.large · EBS gp3 spread 6.4% (p10–p90) around a p50 of 2m 53s, min 2m 51s, max 3m 06s, 5 samples
  2. Avrea 2 vCPU · 8 GB spread 7% (p10–p90) around a p50 of 2m 17s, min 2m 13s, max 2m 23s, 5 samples
  3. Namespace 2 vCPU · 8 GB 2x8 spread 8.8% (p10–p90) around a p50 of 2m 32s, min 2m 24s, max 2m 39s, 5 samples
  4. AWS CodeBuild 2 vCPU · 3 GB small spread 11.6% (p10–p90) around a p50 of 6m 35s, min 6m 06s, max 7m 03s, 5 samples
  5. Ubicloud 2 vCPU · 8 GB standard spread 20.6% (p10–p90) around a p50 of 3m 56s, min 3m 39s, max 4m 44s, 5 samples
  6. Warpbuild 2 vCPU · 8 GB spread 25.5% (p10–p90) around a p50 of 3m 49s, min 3m 37s, max 4m 53s, 5 samples
  7. GitHub 2 vCPU · 8 GB spread 45.1% (p10–p90) around a p50 of 5m 46s, min 4m 12s, max 7m 37s, 5 samples
  8. Blacksmith 2 vCPU · 8 GB spread 51.1% (p10–p90) around a p50 of 3m 55s, min 2m 40s, max 5m 04s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m8a.xlarge · EBS gp3 spread 3.6% (p10–p90) around a p50 of 1m 47s, min 1m 46s, max 1m 51s, 5 samples
  2. StarSling 4 vCPU · 16 GB spread 7% (p10–p90) around a p50 of 1m 32s, min 1m 29s, max 1m 37s, 5 samples
  3. Avrea 4 vCPU · 16 GB spread 8.6% (p10–p90) around a p50 of 1m 26s, min 1m 25s, max 1m 34s, 5 samples
  4. AWS CodeBuild 4 vCPU · 8 GB medium spread 8.7% (p10–p90) around a p50 of 3m 32s, min 3m 20s, max 3m 44s, 5 samples
  5. Namespace 4 vCPU · 8 GB 4x8 spread 15.6% (p10–p90) around a p50 of 1m 39s, min 1m 28s, max 1m 45s, 5 samples
  6. Warpbuild 4 vCPU · 16 GB spread 17.4% (p10–p90) around a p50 of 2m 09s, min 2m 01s, max 2m 31s, 5 samples
  7. GitHub 4 vCPU · 16 GB spread 29.7% (p10–p90) around a p50 of 3m 33s, min 3m 05s, max 4m 27s, 5 samples
  8. Ubicloud 4 vCPU · 16 GB standard spread 31.2% (p10–p90) around a p50 of 2m 19s, min 1m 42s, max 2m 35s, 5 samples
  9. Blacksmith 4 vCPU · 16 GB spread 62.2% (p10–p90) around a p50 of 1m 40s, min 1m 39s, max 3m 16s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 2.9% (p10–p90) around a p50 of 1m 15s, min 1m 13s, max 1m 16s, 5 samples
  2. Avrea 8 vCPU · 32 GB spread 6.5% (p10–p90) around a p50 of 1m 05s, min 1m 03s, max 1m 08s, 5 samples
  3. AWS CodeBuild 8 vCPU · 16 GB large spread 9.5% (p10–p90) around a p50 of 2m 12s, min 2m 05s, max 2m 18s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 14.2% (p10–p90) around a p50 of 1m 33s, min 1m 27s, max 1m 45s, 5 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 23.4% (p10–p90) around a p50 of 1m 41s, min 1m 32s, max 2m 00s, 5 samples
  6. Blacksmith 8 vCPU · 32 GB spread 30.4% (p10–p90) around a p50 of 1m 11s, min 1m 08s, max 1m 30s, 5 samples
  7. StarSling 8 vCPU · 32 GB spread 30.6% (p10–p90) around a p50 of 1m 10s, min 1m 01s, max 1m 30s, 5 samples
  8. Namespace 8 vCPU · 16 GB 8x16 spread 44.3% (p10–p90) around a p50 of 1m 05s, min 1m 03s, max 1m 47s, 5 samples
  9. GitHub 8 vCPU · 32 GB spread 45% (p10–p90) around a p50 of 1m 59s, min 1m 49s, max 2m 53s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe spread 1.9% (p10–p90) around a p50 of 1m 46s, min 1m 45s, max 1m 47s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 8.3% (p10–p90) around a p50 of 3m 10s, min 3m 01s, max 3m 18s, 5 samples
  3. Avrea 4 vCPU · 16 GB spread 12.2% (p10–p90) around a p50 of 1m 17s, min 1m 15s, max 1m 28s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 12.9% (p10–p90) around a p50 of 1m 59s, min 1m 55s, max 2m 12s, 5 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 20.4% (p10–p90) around a p50 of 3m 48s, min 3m 16s, max 4m 14s, 5 samples
  6. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 20.7% (p10–p90) around a p50 of 1m 21s, min 1m 15s, max 1m 37s, 5 samples
  7. Blacksmith 8 vCPU · 32 GB spread 25.8% (p10–p90) around a p50 of 2m 39s, min 2m 28s, max 3m 19s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 8 GB m9gd.large · local NVMe spread 1.6% (p10–p90) around a p50 of 2m 50s, min 2m 50s, max 2m 54s, 5 samples
  2. GitHub 2 vCPU · 8 GB spread 13.1% (p10–p90) around a p50 of 5m 31s, min 5m 11s, max 6m 09s, 5 samples
  3. Warpbuild 2 vCPU · 8 GB spread 15.8% (p10–p90) around a p50 of 4m 43s, min 4m 20s, max 5m 17s, 5 samples
  4. Namespace 2 vCPU · 8 GB Apple silicon 2x8 spread 17% (p10–p90) around a p50 of 2m 06s, min 1m 55s, max 2m 24s, 5 samples
  5. Avrea 2 vCPU · 8 GB spread 18.6% (p10–p90) around a p50 of 1m 56s, min 1m 48s, max 2m 20s, 5 samples
  6. Blacksmith 2 vCPU · 6 GB spread 21.3% (p10–p90) around a p50 of 6m 34s, min 6m 07s, max 8m 08s, 5 samples
  7. Ubicloud 2 vCPU · 8 GB standard spread 30.5% (p10–p90) around a p50 of 8m 04s, min 7m 44s, max 10m 25s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe spread 1.9% (p10–p90) around a p50 of 1m 46s, min 1m 45s, max 1m 47s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 8.3% (p10–p90) around a p50 of 3m 10s, min 3m 01s, max 3m 18s, 5 samples
  3. Warpbuild 4 vCPU · 16 GB spread 10.2% (p10–p90) around a p50 of 3m 01s, min 2m 49s, max 3m 13s, 5 samples
  4. Avrea 4 vCPU · 16 GB spread 12.2% (p10–p90) around a p50 of 1m 17s, min 1m 15s, max 1m 28s, 5 samples
  5. Ubicloud 4 vCPU · 16 GB standard spread 17.8% (p10–p90) around a p50 of 5m 41s, min 4m 39s, max 6m 06s, 5 samples
  6. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 20.7% (p10–p90) around a p50 of 1m 21s, min 1m 15s, max 1m 37s, 5 samples
  7. Blacksmith 4 vCPU · 16 GB spread 26% (p10–p90) around a p50 of 4m 03s, min 3m 44s, max 5m 06s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe spread 3.6% (p10–p90) around a p50 of 1m 12s, min 1m 11s, max 1m 14s, 5 samples
  2. Avrea 8 vCPU · 32 GB spread 10% (p10–p90) around a p50 of 58s, min 55s, max 1m 02s, 5 samples
  3. Warpbuild 8 vCPU · 32 GB spread 12.9% (p10–p90) around a p50 of 1m 59s, min 1m 55s, max 2m 12s, 5 samples
  4. Namespace 8 vCPU · 16 GB 8x16 spread 17% (p10–p90) around a p50 of 3m 44s, min 3m 22s, max 4m 08s, 5 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 20.4% (p10–p90) around a p50 of 3m 48s, min 3m 16s, max 4m 14s, 5 samples
  6. Blacksmith 8 vCPU · 32 GB spread 25.8% (p10–p90) around a p50 of 2m 39s, min 2m 28s, max 3m 19s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 8 vCPU · 32 GB p50 26s, min 25s, max 27s, 4 samples, not ranked
  2. StarSling 4 vCPU · 16 GB p50 47s, min 44s, max 48s, 4 samples, not ranked
  3. AWS CodeBuild 8 vCPU · 16 GB large p50 1m 04s, min 1m 01s, max 1m 07s, 4 samples, not ranked
  4. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 34s, min 29s, max 36s, 4 samples, not ranked
  5. GitHub 8 vCPU · 32 GB p50 1m 08s, min 55s, max 1m 16s, 4 samples, not ranked
  6. Warpbuild 4 vCPU · 16 GB p50 1m 07s, min 56s, max 1m 14s, 4 samples, not ranked
  7. Ubicloud 4 vCPU · 16 GB standard p50 1m 35s, min 1m 20s, max 1m 52s, 4 samples, not ranked
  8. Blacksmith 8 vCPU · 32 GB p50 42s, min 31s, max 54s, 4 samples, not ranked
  9. Namespace 8 vCPU · 16 GB 8x16 p50 32s, min 28s, max 1m 30s, 4 samples, not ranked
one run p50 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs p50 1m 28s, min 1m 25s, max 1m 38s, 4 samples, not ranked
  2. GitHub 2 vCPU · 8 GB p50 3m 56s, min 3m 42s, max 4m 30s, 3 samples, not ranked
  3. Ubicloud 2 vCPU · 8 GB standard p50 3m 35s, min 2m 55s, max 4m 44s, 4 samples, not ranked
  4. Blacksmith 2 vCPU · 8 GB p50 1m 58s, min 1m 29s, max 2m 36s, 4 samples, not ranked
  5. Warpbuild 2 vCPU · 8 GB p50 2m 25s, min 2m 05s, max 3m 31s, 4 samples, not ranked
one run p50 min–max

Under 3 runs so far: Avrea.

  1. Avrea 4 vCPU · 16 GB p50 40s, min 39s, max 40s, 4 samples, not ranked
  2. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs p50 50s, min 48s, max 50s, 4 samples, not ranked
  3. StarSling 4 vCPU · 16 GB p50 47s, min 44s, max 48s, 4 samples, not ranked
  4. Warpbuild 4 vCPU · 16 GB p50 1m 07s, min 56s, max 1m 14s, 4 samples, not ranked
  5. GitHub 4 vCPU · 16 GB p50 1m 52s, min 1m 25s, max 2m 01s, 4 samples, not ranked
  6. Ubicloud 4 vCPU · 16 GB standard p50 1m 35s, min 1m 20s, max 1m 52s, 4 samples, not ranked
  7. Namespace 4 vCPU · 16 GB 4x16 p50 50s, min 42s, max 1m 00s, 4 samples, not ranked
  8. Blacksmith 4 vCPU · 16 GB p50 1m 25s, min 44s, max 1m 44s, 4 samples, not ranked
one run p50 min–max
  1. StarSling 8 vCPU · 32 GB p50 31s, min 31s, max 32s, 4 samples, not ranked
  2. Avrea 8 vCPU · 32 GB p50 26s, min 25s, max 27s, 4 samples, not ranked
  3. AWS CodeBuild 8 vCPU · 16 GB large p50 1m 04s, min 1m 01s, max 1m 07s, 4 samples, not ranked
  4. Warpbuild 8 vCPU · 32 GB p50 42s, min 38s, max 47s, 4 samples, not ranked
  5. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 34s, min 29s, max 36s, 4 samples, not ranked
  6. GitHub 8 vCPU · 32 GB p50 1m 08s, min 55s, max 1m 16s, 4 samples, not ranked
  7. Ubicloud 8 vCPU · 32 GB premium p50 54s, min 39s, max 1m 01s, 4 samples, not ranked
  8. Blacksmith 8 vCPU · 32 GB p50 42s, min 31s, max 54s, 4 samples, not ranked
  9. Namespace 8 vCPU · 16 GB 8x16 p50 32s, min 28s, max 1m 30s, 4 samples, not ranked
one run p50 min–max
  1. GitHub 4 vCPU · 16 GB p50 1m 26s, min 1m 25s, max 1m 30s, 4 samples, not ranked
  2. Blacksmith 8 vCPU · 32 GB p50 1m 10s, min 1m 09s, max 1m 15s, 4 samples, not ranked
  3. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe p50 31s, min 30s, max 34s, 4 samples, not ranked
  4. Warpbuild 8 vCPU · 32 GB p50 48s, min 43s, max 50s, 4 samples, not ranked
  5. Avrea 4 vCPU · 16 GB p50 40s, min 38s, max 45s, 4 samples, not ranked
  6. Namespace 8 vCPU · 16 GB 8x16 p50 1m 44s, min 1m 35s, max 2m 02s, 4 samples, not ranked
  7. Ubicloud 8 vCPU · 32 GB standard p50 1m 26s, min 1m 19s, max 1m 46s, 4 samples, not ranked
one run p50 min–max
  1. RunsOn 2 vCPU · 16 GB r9g.large · tmpfs p50 1m 27s, min 1m 25s, max 1m 30s, 4 samples, not ranked
  2. Blacksmith 2 vCPU · 6 GB p50 5m 01s, min 4m 33s, max 5m 13s, 4 samples, not ranked
  3. Avrea 2 vCPU · 8 GB p50 1m 08s, min 1m 07s, max 1m 19s, 3 samples, not ranked
  4. Warpbuild 2 vCPU · 8 GB p50 3m 06s, min 2m 28s, max 4m 48s, 4 samples, not ranked
one run p50 min–max

Under 3 runs so far: GitHub.

  1. Warpbuild 4 vCPU · 16 GB p50 1m 14s, min 1m 13s, max 1m 16s, 4 samples, not ranked
  2. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe p50 50s, min 49s, max 51s, 4 samples, not ranked
  3. GitHub 4 vCPU · 16 GB p50 1m 26s, min 1m 25s, max 1m 30s, 4 samples, not ranked
  4. Blacksmith 4 vCPU · 16 GB p50 1m 57s, min 1m 54s, max 2m 01s, 4 samples, not ranked
  5. Avrea 4 vCPU · 16 GB p50 40s, min 38s, max 45s, 4 samples, not ranked
  6. Namespace 4 vCPU · 16 GB Apple silicon 4x16 p50 43s, min 40s, max 1m 31s, 4 samples, not ranked
  7. Ubicloud 4 vCPU · 16 GB standard p50 2m 19s, min 2m 08s, max 24m 16s, 4 samples, not ranked
one run p50 min–max
  1. Avrea 8 vCPU · 32 GB p50 30s, min 29s, max 31s, 4 samples, not ranked
  2. Blacksmith 8 vCPU · 32 GB p50 1m 10s, min 1m 09s, max 1m 15s, 4 samples, not ranked
  3. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe p50 31s, min 30s, max 34s, 4 samples, not ranked
  4. Warpbuild 8 vCPU · 32 GB p50 48s, min 43s, max 50s, 4 samples, not ranked
  5. Namespace 8 vCPU · 16 GB 8x16 p50 1m 44s, min 1m 35s, max 2m 02s, 4 samples, not ranked
  6. Ubicloud 8 vCPU · 32 GB standard p50 1m 26s, min 1m 19s, max 1m 46s, 4 samples, not ranked
one run p50 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 4.6% (p10–p90) around a p50 of 3m 35s, min 3m 28s, max 3m 41s, 5 samples
  2. Avrea 4 vCPU · 16 GB spread 5.6% (p10–p90) around a p50 of 4m 52s, min 4m 41s, max 5m 03s, 5 samples
  3. Namespace 8 vCPU · 16 GB 8x16 spread 5.7% (p10–p90) around a p50 of 3m 37s, min 3m 27s, max 3m 41s, 5 samples
  4. StarSling 4 vCPU · 16 GB spread 9.8% (p10–p90) around a p50 of 5m 31s, min 5m 15s, max 5m 57s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 11.9% (p10–p90) around a p50 of 12m 26s, min 11m 32s, max 13m 27s, 5 samples
  6. Warpbuild 8 vCPU · 32 GB spread 28.5% (p10–p90) around a p50 of 4m 58s, min 4m 39s, max 6m 36s, 5 samples
  7. Blacksmith 4 vCPU · 16 GB spread 30.5% (p10–p90) around a p50 of 5m 01s, min 4m 38s, max 6m 49s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 31.5% (p10–p90) around a p50 of 7m 23s, min 6m 22s, max 8m 55s, 5 samples
  9. Ubicloud 4 vCPU · 16 GB standard spread 61.9% (p10–p90) around a p50 of 10m 37s, min 6m 51s, max 15m 54s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 11.9% (p10–p90) around a p50 of 11m 36s, min 11m 08s, max 13m 05s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 4 vCPU · 16 GB spread 5.6% (p10–p90) around a p50 of 4m 52s, min 4m 41s, max 5m 03s, 5 samples
  2. StarSling 4 vCPU · 16 GB spread 9.8% (p10–p90) around a p50 of 5m 31s, min 5m 15s, max 5m 57s, 5 samples
  3. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 14.2% (p10–p90) around a p50 of 4m 57s, min 4m 45s, max 5m 48s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 15.3% (p10–p90) around a p50 of 7m 27s, min 6m 35s, max 8m 06s, 5 samples
  5. GitHub 4 vCPU · 16 GB spread 24.3% (p10–p90) around a p50 of 12m 53s, min 11m 17s, max 15m 13s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 30.5% (p10–p90) around a p50 of 5m 01s, min 4m 38s, max 6m 49s, 5 samples
  7. Namespace 4 vCPU · 16 GB 4x16 spread 46.3% (p10–p90) around a p50 of 4m 40s, min 4m 30s, max 7m 56s, 5 samples
  8. Ubicloud 4 vCPU · 16 GB standard spread 61.9% (p10–p90) around a p50 of 10m 37s, min 6m 51s, max 15m 54s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 4.6% (p10–p90) around a p50 of 3m 35s, min 3m 28s, max 3m 41s, 5 samples
  2. Namespace 8 vCPU · 16 GB 8x16 spread 5.7% (p10–p90) around a p50 of 3m 37s, min 3m 27s, max 3m 41s, 5 samples
  3. StarSling 8 vCPU · 32 GB spread 6.7% (p10–p90) around a p50 of 4m 02s, min 3m 55s, max 4m 18s, 5 samples
  4. Avrea 8 vCPU · 32 GB spread 8.4% (p10–p90) around a p50 of 3m 45s, min 3m 26s, max 3m 47s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 11.9% (p10–p90) around a p50 of 12m 26s, min 11m 32s, max 13m 27s, 5 samples
  6. Ubicloud 8 vCPU · 32 GB standard spread 13.4% (p10–p90) around a p50 of 7m 38s, min 7m 01s, max 8m 17s, 5 samples
  7. Warpbuild 8 vCPU · 32 GB spread 28.5% (p10–p90) around a p50 of 4m 58s, min 4m 39s, max 6m 36s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 31.5% (p10–p90) around a p50 of 7m 23s, min 6m 22s, max 8m 55s, 5 samples
  9. Blacksmith 8 vCPU · 32 GB spread 44% (p10–p90) around a p50 of 3m 52s, min 3m 36s, max 6m 14s, 5 samples
one run p50 p10–p90 min–max
  1. GitHub 4 vCPU · 16 GB spread 8.3% (p10–p90) around a p50 of 9m 59s, min 9m 25s, max 10m 25s, 5 samples
  2. Avrea 4 vCPU · 16 GB spread 9% (p10–p90) around a p50 of 4m 46s, min 4m 39s, max 5m 09s, 5 samples
  3. Ubicloud 4 vCPU · 16 GB standard spread 13.1% (p10–p90) around a p50 of 13m 41s, min 12m 38s, max 15m 05s, 5 samples
  4. Blacksmith 4 vCPU · 16 GB spread 21.8% (p10–p90) around a p50 of 10m 26s, min 9m 46s, max 13m 03s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 28.5% (p10–p90) around a p50 of 6m 30s, min 6m 02s, max 8m 51s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 48.9% (p10–p90) around a p50 of 9m 58s, min 9m 39s, max 17m 36s, 5 samples
  7. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 49.6% (p10–p90) around a p50 of 4m 10s, min 3m 59s, max 7m 21s, 5 samples
one run p50 p10–p90 min–max

No arm64 Docker build finished at 2 vCPU.

  1. GitHub 4 vCPU · 16 GB spread 8.3% (p10–p90) around a p50 of 9m 59s, min 9m 25s, max 10m 25s, 5 samples
  2. Avrea 4 vCPU · 16 GB spread 9% (p10–p90) around a p50 of 4m 46s, min 4m 39s, max 5m 09s, 5 samples
  3. Ubicloud 4 vCPU · 16 GB standard spread 13.1% (p10–p90) around a p50 of 13m 41s, min 12m 38s, max 15m 05s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 15.9% (p10–p90) around a p50 of 10m 02s, min 9m 34s, max 12m 00s, 5 samples
  5. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 20.3% (p10–p90) around a p50 of 5m 09s, min 4m 40s, max 6m 04s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 21.8% (p10–p90) around a p50 of 10m 26s, min 9m 46s, max 13m 03s, 5 samples
  7. RunsOn 4 vCPU · 32 GB r9g.xlarge · tmpfs spread 39% (p10–p90) around a p50 of 5m 22s, min 5m 18s, max 8m 42s, 5 samples
one run p50 p10–p90 min–max
  1. Avrea 8 vCPU · 32 GB spread 9.2% (p10–p90) around a p50 of 3m 41s, min 3m 30s, max 3m 54s, 5 samples
  2. Ubicloud 8 vCPU · 32 GB standard spread 11.3% (p10–p90) around a p50 of 10m 26s, min 10m 03s, max 11m 36s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 17% (p10–p90) around a p50 of 7m 34s, min 6m 53s, max 8m 34s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 28.5% (p10–p90) around a p50 of 6m 30s, min 6m 02s, max 8m 51s, 5 samples
  5. Namespace 8 vCPU · 16 GB 8x16 spread 48.9% (p10–p90) around a p50 of 9m 58s, min 9m 39s, max 17m 36s, 5 samples
  6. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 49.6% (p10–p90) around a p50 of 4m 10s, min 3m 59s, max 7m 21s, 5 samples
one run p50 p10–p90 min–max

Is a bigger runner always worth it?

What doubling the vCPU did to each build, on each provider's own runner line: each step usually saves less time and costs more per build.

Median across providers: from 4 to 8 vCPU, each build ran 26–27% faster for 46–48% more per build; from 2 to 4 vCPU, 39–50% faster, cost per build +0% to +18%.

Median across providers: from 4 to 8 vCPU, each build ran 26–27% faster for 46–48% more per build; from 2 to 4 vCPU, 39–50% faster, cost per build +0% to +23%.

Median across providers: from 4 to 8 vCPU, each build ran 24–32% faster for 36–52% more per build; from 2 to 4 vCPU, 36–49% faster, cost per build +8% to +26%.

Median across providers: from 4 to 8 vCPU, each build ran 24–32% faster for 37–52% more per build; from 2 to 4 vCPU, 36–49% faster, cost per build +2% to +26%.

  • Each line is one provider's runner at 2, 4 and 8 vCPU, its vCPU count in each dot (hollow: under 3 runs so far); RunsOn's is the bold one (m8a / m9g, not its fastest or cheapest).
  • Right costs more per build, lower is faster: the shaded corner, bottom left, is where you want to be.
  • Dashed lines join equal time × cost. A step that drops more steeply than them bought time cheaply; one that runs flatter mostly bought cost.
Rustspotify-player
faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaNamespaceStarSlingBlacksmithRunsOnWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 27% faster, +47% cost (9 providers)
2 → 4 vCPU: 39% faster, +18% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaNamespaceStarSlingBlacksmithRunsOnWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 27% faster, +47% cost (9 providers)
2 → 4 vCPU: 39% faster, +23% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 32% faster, +36% cost (6 providers)
2 → 4 vCPU: 36% faster, +26% cost (7 providers)

faster & cheaper$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 32% faster, +37% cost (6 providers)
2 → 4 vCPU: 36% faster, +26% cost (7 providers)

TypeScriptopencode
faster & cheaper$0.0020$0.0050$0.0100$0.02001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingNamespaceWarpbuildRunsOnUbicloudBlacksmithGitHub

Median across providers: 4 → 8 vCPU: 27% faster, +46% cost (8 providers)
2 → 4 vCPU: 50% faster, +0% cost (5 providers)

faster & cheaper$0.0050$0.0100$0.02001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingNamespaceWarpbuildRunsOnUbicloudBlacksmithGitHub

Median across providers: 4 → 8 vCPU: 27% faster, +46% cost (8 providers)
2 → 4 vCPU: 50% faster, +0% cost (5 providers)

faster & cheaper$0.0020$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithUbicloudNamespace

Median across providers: 4 → 8 vCPU: 29% faster, +41% cost (6 providers)
2 → 4 vCPU: 49% faster, +8% cost (4 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubBlacksmithUbicloudNamespace

Median across providers: 4 → 8 vCPU: 29% faster, +41% cost (6 providers)
2 → 4 vCPU: 49% faster, +2% cost (4 providers)

DockerPostHog image
faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceAvreaBlacksmithStarSlingWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 26% faster, +48% cost (8 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceAvreaBlacksmithStarSlingWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 26% faster, +48% cost (8 providers)

faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 24% faster, +52% cost (6 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 24% faster, +52% cost (6 providers)

Beyond builds

The same runners, one question each.

  • CPU Fastest single-thread on x64: Namespace 4x8, 4,668 PassMark.
  • Cache 4 GB restore on x64, any size: RunsOn 8s (12 vCPU), then Blacksmith 11s (2 vCPU); GitHub's own service 31s (8 vCPU).4 GB restore on x64 at 2 vCPU: Blacksmith 11s, then Avrea 12s; GitHub's own service 1m 03s.4 GB restore on x64 at 4 vCPU: Avrea 11s, then Blacksmith 13s; GitHub's own service 46s.4 GB restore on x64 at 8 vCPU: Avrea 11s, then RunsOn 12s; GitHub's own service 31s.
  • Disk Small random reads (QD1) on x64: Namespace 2x8 leads with 104,659 per second, 59× the slowest disk (AWS CodeBuild, 1,761). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
  • CPU Fastest single-thread on arm64: Avrea 2 vCPU, 5,000 PassMark.
  • Cache 4 GB restore on arm64, any size: Avrea 11s (4 vCPU), then RunsOn 13s (8 vCPU); GitHub's own service 30s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then RunsOn 24s; GitHub's own service 53s.4 GB restore on arm64 at 4 vCPU: Avrea 11s, then RunsOn 18s; GitHub's own service 30s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 13s.
  • Disk Small random reads (QD1) on arm64: Warpbuild xfast (Apple M4 Pro) 12 vCPU leads with 29,545 per second, 3.8× the slowest disk (GitHub, 7,775). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.

Providers

Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.

Head-to-head

Two runners of the same size, build for build. Cost per build decides: a faster runner that costs twice as much per build is not the better runner.

Cost per build is variable cost only; licenses and plan fees are flat, on each provider's page. Estimate your bill