v3.5 changelog
quickstart →
Pricing Benchmarks Docs Changelog Blog Troubleshooting GitHub Install in 10 min

GitHub Actions runner benchmarks

At 4 vCPU, RunsOn is the best value on all three builds.

Rust spotify-player

RunsOn c8a.xlarge · EBS gp3

2m 04s $0.0041 per build

GitHub 4-core: 11× the cost, 1.9× the time, 8th of 9

TypeScript opencode

RunsOn r8a.xlarge · tmpfs

1m 01s $0.0034 per build

GitHub 4-core: 7.7× the cost, 2.2× the time, 7th of 7

Docker PostHog image

RunsOn r8a.xlarge · tmpfs

5m 19s $0.0134 per build

GitHub 4-core: 13× the cost, 2.8× the time, 8th of 8

Best value per provider

Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by time × cost per build.; shades are build phases in order.; build time on the right.

Rustspotify-player
  1. RunsOn4 vCPU · 8 GBc8a.xlarge · EBS gp32m 04s, cost per build spot $0.0041, on-demand $0.0097, best-value of 9 machines; phases: setup 14s, cargo test 1m 01s, clippy 34s, runner overhead 15s, ×1.00
  2. Namespace4 vCPU · 8 GB4x81m 47s, cost per build $0.0107, best-value of 2 machines; phases: setup 10s, cargo test 58s, clippy 31s, runner overhead 8s, ×2.28
  3. Ubicloud4 vCPU · 16 GBstandard3m 12s, cost per build $0.0080, best-value of 2 machines; phases: setup 28s, cargo test 1m 48s, clippy 44s, runner overhead 12s, ×3.05
  4. StarSling4 vCPU · 16 GB1m 55s, cost per build $0.0153; phases: setup 18s, cargo test 57s, clippy 30s, runner overhead 10s, ×3.50
  5. Blacksmith4 vCPU · 16 GB2m 11s, cost per build $0.0175; phases: setup 16s, cargo test 1m 05s, clippy 36s, runner overhead 14s, ×4.55
  6. Avrea4 vCPU · 16 GB2m 15s, cost per build $0.0180; phases: setup 15s, cargo test 1m 13s, clippy 38s, runner overhead 9s, ×4.83
  7. Warpbuild4 vCPU · 16 GB2m 27s, cost per build $0.0196; phases: setup 16s, cargo test 1m 22s, clippy 42s, runner overhead 7s, ×5.73
  8. GitHub4 vCPU · 16 GB3m 53s, cost per build $0.0466; phases: setup 24s, cargo test 2m 12s, clippy 1m 13s, runner overhead 4s, ×22
  9. AWS CodeBuild4 vCPU · 8 GBmedium4m 33s, cost per build $0.0455; phases: setup 32s, cargo test 2m 21s, clippy 1m 18s, runner overhead 22s, ×25

Bar shades, light to dark, in execution order: setup = checkout + packages + toolchaincargo fmtcargo testclippyrunner overhead

cost per build× = time × cost against the best

TypeScriptopencode
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs1m 01s, cost per build spot $0.0034, on-demand $0.0081, best-value of 9 machines; phases: toolchain 1s, clone 1s, bun install 8s, typecheck 36s, runner overhead 15s, ×1.00
  2. Ubicloud4 vCPU · 16 GBstandard1m 40s, cost per build $0.0042, best-value of 2 machines; phases: toolchain 1s, clone 5s, bun install 25s, typecheck 53s, runner overhead 16s, ×2.00
  3. StarSling4 vCPU · 16 GB1m 02s, cost per build $0.0083; phases: toolchain 1s, clone 2s, bun install 10s, typecheck 38s, runner overhead 11s, ×2.45
  4. Blacksmith4 vCPU · 16 GB1m 07s, cost per build $0.0089; phases: toolchain 1s, clone 2s, bun install 9s, typecheck 40s, runner overhead 15s, ×2.87
  5. Namespace4 vCPU · 16 GB4x161m 01s, cost per build $0.0122; phases: toolchain 1s, clone 1s, bun install 11s, typecheck 36s, runner overhead 12s, ×3.56
  6. Warpbuild4 vCPU · 16 GB1m 22s, cost per build $0.0109; phases: toolchain 1s, clone 1s, bun install 14s, typecheck 58s, runner overhead 8s, ×4.29
  7. GitHub4 vCPU · 16 GB2m 12s, cost per build $0.0264; phases: toolchain 1s, clone 2s, bun install 23s, typecheck 1m 28s, runner overhead 18s, ×17
  8. Avreafailed · in bun install
  9. AWS CodeBuildfailed · out of memory

Bar shades, light to dark, in execution order: toolchainclonebun installtypecheckrunner overhead

cost per build× = time × cost against the best

DockerPostHog image
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs5m 19s, cost per build spot $0.0134, on-demand $0.0322, best-value of 8 machines; phases: fetch source 19s, buildx setup 3s, docker build 4m 38s, runner overhead 19s, ×1.00
  2. Blacksmith4 vCPU · 16 GB5m 48s, cost per build $0.0464; phases: fetch source 21s, buildx setup 2s, docker build 5m 00s, runner overhead 25s, ×3.77
  3. StarSling4 vCPU · 16 GB6m 07s, cost per build $0.0489; phases: fetch source 23s, buildx setup 4s, docker build 5m 14s, runner overhead 26s, ×4.19
  4. Namespace4 vCPU · 16 GB4x165m 25s, cost per build $0.0650; phases: fetch source 22s, buildx setup 2s, docker build 4m 44s, runner overhead 17s, ×4.93
  5. Avrea4 vCPU · 16 GB7m 42s, cost per build $0.0616; phases: fetch source 21s, buildx setup 4s, docker build 6m 55s, runner overhead 22s, ×6.64
  6. Ubicloud4 vCPU · 16 GBpremium11m 10s, cost per build $0.0447, best-value of 2 machines; phases: fetch source 23s, buildx setup 12s, docker build 10m 12s, runner overhead 23s, ×6.98
  7. Warpbuild4 vCPU · 16 GB8m 00s, cost per build $0.0640; phases: fetch source 23s, buildx setup 4s, docker build 7m 15s, runner overhead 18s, ×7.17
  8. GitHub4 vCPU · 16 GB14m 53s, cost per build $0.1786; phases: fetch source 24s, buildx setup 11s, docker build 12m 26s, runner overhead 1m 52s, ×37
  9. AWS CodeBuildnot run: needs more than 8 GB of memory

Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead

cost per build× = time × cost against the best

Each provider's best-value machine (lowest time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.

Queue and consistency

How long jobs wait in a burst, and how far build times move from run to run.

Queue under a burst15 jobs at once · each provider's quickest-starting runner at the chosen size · ranked from 3 runs
  1. GitHub 2 vCPU · 8 GB 5 of 6 bursts p50 4s, p10–p90 3s to 5s, min 2s, max 40s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB 5 of 6 bursts p50 10s, p10–p90 9s to 14s, min 9s, max 15s, 75 samples
  3. Ubicloud 2 vCPU · 8 GB standard 5 of 6 bursts p50 10s, p10–p90 9s to 28s, min 7s, max 4m 37s, 75 samples
  4. StarSling 4 vCPU · 16 GB 5 of 6 bursts p50 11s, p10–p90 8s to 13s, min 7s, max 14s, 75 samples
  5. Avrea 4 vCPU · 16 GB 5 of 6 bursts p50 12s, p10–p90 9s to 14s, min 8s, max 15s, 75 samples
  6. Warpbuild 4 vCPU · 16 GB 5 of 6 bursts p50 13s, p10–p90 12s to 15s, min 11s, max 32s, 75 samples
  7. Namespace 4 vCPU · 8 GB 4x8 3 of 6 bursts p50 16s, p10–p90 12s to 20s, min 11s, max 27s, 45 samples
  8. RunsOn 2 vCPU · 8 GB m8azn.large · EBS gp3 5 of 6 bursts p50 17s, p10–p90 15s to 19s, min 15s, max 21s, 75 samples
  9. AWS CodeBuild 8 vCPU · 16 GB large 5 of 6 bursts p50 20s, p10–p90 20s to 22s, min 19s, max 24s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB 5 of 6 bursts p50 4s, p10–p90 3s to 5s, min 2s, max 40s, 75 samples
  2. Ubicloud 2 vCPU · 8 GB standard 5 of 6 bursts p50 10s, p10–p90 9s to 28s, min 7s, max 4m 37s, 75 samples
  3. Blacksmith 2 vCPU · 8 GB 5 of 6 bursts p50 11s, p10–p90 9s to 12s, min 8s, max 14s, 75 samples
  4. Avrea 2 vCPU · 8 GB 5 of 6 bursts p50 12s, p10–p90 10s to 18s, min 10s, max 20s, 75 samples
  5. Warpbuild 2 vCPU · 8 GB 5 of 6 bursts p50 13s, p10–p90 11s to 18s, min 10s, max 1m 23s, 75 samples
  6. Namespace 2 vCPU · 8 GB 2x8 3 of 6 bursts p50 16s, p10–p90 10s to 22s, min 7s, max 25s, 45 samples
  7. RunsOn 2 vCPU · 8 GB m8azn.large · EBS gp3 5 of 6 bursts p50 17s, p10–p90 15s to 19s, min 15s, max 21s, 75 samples
  8. AWS CodeBuild 2 vCPU · 3 GB small 5 of 6 bursts p50 21s, p10–p90 20s to 24s, min 19s, max 29s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB 5 of 6 bursts p50 4s, p10–p90 3s to 6s, min 2s, max 42s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB 5 of 6 bursts p50 10s, p10–p90 9s to 14s, min 9s, max 15s, 75 samples
  3. StarSling 4 vCPU · 16 GB 5 of 6 bursts p50 11s, p10–p90 8s to 13s, min 7s, max 14s, 75 samples
  4. Avrea 4 vCPU · 16 GB 5 of 6 bursts p50 12s, p10–p90 9s to 14s, min 8s, max 15s, 75 samples
  5. Warpbuild 4 vCPU · 16 GB 5 of 6 bursts p50 13s, p10–p90 12s to 15s, min 11s, max 32s, 75 samples
  6. Namespace 4 vCPU · 8 GB 4x8 3 of 6 bursts p50 16s, p10–p90 12s to 20s, min 11s, max 27s, 45 samples
  7. RunsOn 4 vCPU · 16 GB m8a.xlarge · EBS gp3 p50 18s, p10–p90 16s to 21s, min 15s, max 1m 22s, 90 samples
  8. Ubicloud 4 vCPU · 16 GB premium 5 of 6 bursts p50 20s, p10–p90 17s to 27s, min 9s, max 31s, 75 samples
  9. AWS CodeBuild 4 vCPU · 8 GB medium 5 of 6 bursts p50 22s, p10–p90 20s to 1m 21s, min 20s, max 1m 25s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB 5 of 6 bursts p50 11s, p10–p90 10s to 13s, min 8s, max 15s, 75 samples
  2. StarSling 8 vCPU · 32 GB 5 of 6 bursts p50 13s, p10–p90 9s to 1m 18s, min 7s, max 1m 35s, 75 samples
  3. Warpbuild 8 vCPU · 32 GB 5 of 6 bursts p50 13s, p10–p90 10s to 20s, min 7s, max 25s, 75 samples
  4. Namespace 8 vCPU · 32 GB 8x32 3 of 6 bursts p50 16s, p10–p90 10s to 24s, min 7s, max 1m 26s, 45 samples
  5. RunsOn 8 vCPU · 32 GB m8i-flex.2xlarge · EBS gp3 5 of 6 bursts p50 17s, p10–p90 16s to 21s, min 15s, max 23s, 75 samples
  6. Ubicloud 8 vCPU · 32 GB premium 5 of 6 bursts p50 18s, p10–p90 15s to 23s, min 14s, max 26s, 75 samples
  7. AWS CodeBuild 8 vCPU · 16 GB large 5 of 6 bursts p50 20s, p10–p90 20s to 22s, min 19s, max 24s, 75 samples
  8. GitHub 8 vCPU · 32 GB 5 of 6 bursts p50 1m 20s, p10–p90 1m 12s to 1m 34s, min 3s, max 2m 09s, 75 samples
  9. Avrea 8 vCPU · 32 GB 12 at a time: plan limit p50 11s, min 8s, max 2m 48s, 75 samples, not ranked: 12 at a time: plan limit
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB 6 of 7 bursts p50 6s, p10–p90 5s to 8s, min 4s, max 11s, 90 samples
  2. Ubicloud 2 vCPU · 8 GB standard 5 of 7 bursts p50 11s, p10–p90 9s to 23s, min 8s, max 29s, 75 samples
  3. Namespace 4 vCPU · 16 GB Apple silicon 4x16 3 of 7 bursts p50 14s, p10–p90 10s to 19s, min 2s, max 21s, 45 samples
  4. Warpbuild 6 vCPU · 14 GB xfast (Apple M4 Pro) 6 of 7 bursts p50 14s, p10–p90 12s to 44s, min 11s, max 7m 36s, 90 samples
  5. Blacksmith 8 vCPU · 32 GB 6 of 7 bursts p50 15s, p10–p90 13s to 16s, min 12s, max 21s, 90 samples
  6. RunsOn 2 vCPU · 8 GB m9g.large · EBS gp3 5 of 7 bursts p50 17s, p10–p90 15s to 20s, min 14s, max 54s, 75 samples
  7. Avrea 4 vCPU · 16 GB 6 of 7 bursts p50 22s, p10–p90 21s to 24s, min 20s, max 1m 26s, 90 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB 6 of 7 bursts p50 6s, p10–p90 5s to 8s, min 4s, max 11s, 90 samples
  2. Ubicloud 2 vCPU · 8 GB standard 5 of 7 bursts p50 11s, p10–p90 9s to 23s, min 8s, max 29s, 75 samples
  3. Blacksmith 2 vCPU · 6 GB 6 of 7 bursts p50 16s, p10–p90 14s to 18s, min 13s, max 28s, 90 samples
  4. RunsOn 2 vCPU · 8 GB m9g.large · EBS gp3 5 of 7 bursts p50 17s, p10–p90 15s to 20s, min 14s, max 54s, 75 samples
  5. Namespace 2 vCPU · 8 GB 2x8 3 of 7 bursts p50 21s, p10–p90 17s to 27s, min 14s, max 58s, 45 samples
  6. Avrea 2 vCPU · 8 GB 6 of 7 bursts p50 23s, p10–p90 20s to 28s, min 18s, max 33s, 90 samples
  7. Warpbuild 2 vCPU · 8 GB 5 of 7 bursts p50 36s, p10–p90 33s to 43s, min 6s, max 56s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB 6 of 7 bursts p50 6s, p10–p90 5s to 8s, min 4s, max 9s, 90 samples
  2. Namespace 4 vCPU · 16 GB Apple silicon 4x16 3 of 7 bursts p50 14s, p10–p90 10s to 19s, min 2s, max 21s, 45 samples
  3. Blacksmith 4 vCPU · 16 GB 6 of 7 bursts p50 15s, p10–p90 13s to 16s, min 12s, max 23s, 90 samples
  4. RunsOn 4 vCPU · 16 GB m9g.xlarge · tmpfs 5 of 7 bursts, 2 spot-interrupted p50 17s, p10–p90 15s to 24s, min 14s, max 55s, 73 samples
  5. Ubicloud 4 vCPU · 16 GB standard 5 of 7 bursts p50 21s, p10–p90 10s to 25s, min 9s, max 29s, 75 samples
  6. Avrea 4 vCPU · 16 GB 6 of 7 bursts p50 22s, p10–p90 21s to 24s, min 20s, max 1m 26s, 90 samples
  7. Warpbuild 4 vCPU · 16 GB 6 of 7 bursts p50 36s, p10–p90 17s to 1m 18s, min 6s, max 1m 35s, 90 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB 6 of 7 bursts p50 15s, p10–p90 13s to 16s, min 12s, max 21s, 90 samples
  2. RunsOn 8 vCPU · 16 GB c8g.2xlarge · EBS gp3 p50 17s, p10–p90 15s to 20s, min 14s, max 3m 08s, 105 samples
  3. Namespace 8 vCPU · 16 GB 8x16 3 of 7 bursts p50 21s, p10–p90 16s to 25s, min 14s, max 4m 20s, 45 samples
  4. Ubicloud 8 vCPU · 32 GB standard 5 of 7 bursts p50 21s, p10–p90 9s to 27s, min 8s, max 2m 07s, 75 samples
  5. Warpbuild 8 vCPU · 32 GB 5 of 7 bursts p50 33s, p10–p90 16s to 41s, min 8s, max 49s, 75 samples
  6. Avrea 8 vCPU · 32 GB 12 at a time: plan limit p50 24s, min 20s, max 2m 03s, 90 samples, not ranked: 12 at a time: plan limit
one job p50 p10–p90 min–max wait before the job starts, log scale
Build time, run to runeach provider's machine from the cards above, at the chosen size, with 3+ workflow runs · build steps time, one dot per run · % off its own p50
  1. Avrea 8 vCPU · 32 GB spread 13.2% (p10–p90) around a p50 of 1m 25s, min 1m 18s, max 1m 34s, 5 samples
  2. RunsOn 8 vCPU · 16 GB c8a.2xlarge · EBS gp3 spread 13.5% (p10–p90) around a p50 of 1m 14s, min 1m 13s, max 1m 25s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 13.7% (p10–p90) around a p50 of 1m 16s, min 1m 11s, max 1m 23s, 5 samples
  4. Namespace 4 vCPU · 8 GB 4x8 spread 18.2% (p10–p90) around a p50 of 1m 40s, min 1m 38s, max 2m 05s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 19.1% (p10–p90) around a p50 of 1m 33s, min 1m 17s, max 1m 42s, 5 samples
  6. StarSling 4 vCPU · 16 GB spread 24% (p10–p90) around a p50 of 1m 45s, min 1m 39s, max 2m 15s, 5 samples
  7. GitHub 8 vCPU · 32 GB spread 33.1% (p10–p90) around a p50 of 2m 04s, min 1m 44s, max 2m 31s, 5 samples
  8. AWS CodeBuild 8 vCPU · 16 GB large spread 35.7% (p10–p90) around a p50 of 2m 38s, min 2m 02s, max 3m 06s, 5 samples
  9. Ubicloud 2 vCPU · 8 GB standard spread 47% (p10–p90) around a p50 of 3m 58s, min 3m 44s, max 6m 29s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 7.3% (p10–p90) around a p50 of 2m 53s, min 2m 51s, max 3m 06s, 5 samples
  2. Warpbuild 2 vCPU · 8 GB spread 16.2% (p10–p90) around a p50 of 3m 42s, min 3m 32s, max 4m 24s, 5 samples
  3. GitHub 2 vCPU · 8 GB spread 17.6% (p10–p90) around a p50 of 6m 46s, min 5m 26s, max 7m 13s, 5 samples
  4. Namespace 2 vCPU · 8 GB 2x8 spread 21.6% (p10–p90) around a p50 of 2m 40s, min 2m 39s, max 3m 34s, 5 samples
  5. Avrea 2 vCPU · 8 GB spread 21.7% (p10–p90) around a p50 of 3m 46s, min 3m 08s, max 4m 19s, 5 samples
  6. Blacksmith 2 vCPU · 8 GB spread 25.9% (p10–p90) around a p50 of 3m 12s, min 2m 48s, max 3m 57s, 5 samples
  7. AWS CodeBuild 2 vCPU · 3 GB small spread 33.7% (p10–p90) around a p50 of 7m 47s, min 6m 08s, max 9m 11s, 5 samples
  8. Ubicloud 2 vCPU · 8 GB standard spread 47% (p10–p90) around a p50 of 3m 58s, min 3m 44s, max 6m 29s, 5 samples
one run p50 p10–p90 min–max
  1. Warpbuild 4 vCPU · 16 GB spread 2.9% (p10–p90) around a p50 of 2m 20s, min 2m 16s, max 2m 22s, 5 samples
  2. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 spread 14.8% (p10–p90) around a p50 of 1m 52s, min 1m 46s, max 2m 07s, 5 samples
  3. Namespace 4 vCPU · 8 GB 4x8 spread 18.2% (p10–p90) around a p50 of 1m 40s, min 1m 38s, max 2m 05s, 5 samples
  4. Blacksmith 4 vCPU · 16 GB spread 18.3% (p10–p90) around a p50 of 1m 59s, min 1m 41s, max 2m 12s, 5 samples
  5. Avrea 4 vCPU · 16 GB spread 20.8% (p10–p90) around a p50 of 2m 05s, min 1m 56s, max 2m 30s, 5 samples
  6. GitHub 4 vCPU · 16 GB spread 23.3% (p10–p90) around a p50 of 3m 47s, min 2m 31s, max 3m 58s, 5 samples
  7. StarSling 4 vCPU · 16 GB spread 24% (p10–p90) around a p50 of 1m 45s, min 1m 39s, max 2m 15s, 5 samples
  8. AWS CodeBuild 4 vCPU · 8 GB medium spread 28.5% (p10–p90) around a p50 of 4m 24s, min 3m 11s, max 4m 43s, 5 samples
  9. Ubicloud 4 vCPU · 16 GB standard spread 47% (p10–p90) around a p50 of 2m 57s, min 2m 26s, max 4m 04s, 5 samples
one run p50 p10–p90 min–max
  1. StarSling 8 vCPU · 32 GB spread 10.5% (p10–p90) around a p50 of 1m 16s, min 1m 11s, max 1m 21s, 5 samples
  2. Avrea 8 vCPU · 32 GB spread 13.2% (p10–p90) around a p50 of 1m 25s, min 1m 18s, max 1m 34s, 5 samples
  3. RunsOn 8 vCPU · 16 GB c8a.2xlarge · EBS gp3 spread 13.5% (p10–p90) around a p50 of 1m 14s, min 1m 13s, max 1m 25s, 5 samples
  4. Blacksmith 8 vCPU · 32 GB spread 13.7% (p10–p90) around a p50 of 1m 16s, min 1m 11s, max 1m 23s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 19.1% (p10–p90) around a p50 of 1m 33s, min 1m 17s, max 1m 42s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 30.4% (p10–p90) around a p50 of 1m 11s, min 1m 08s, max 1m 36s, 5 samples
  7. Ubicloud 8 vCPU · 32 GB standard spread 30.9% (p10–p90) around a p50 of 1m 56s, min 1m 44s, max 2m 33s, 5 samples
  8. GitHub 8 vCPU · 32 GB spread 33.1% (p10–p90) around a p50 of 2m 04s, min 1m 44s, max 2m 31s, 5 samples
  9. AWS CodeBuild 8 vCPU · 16 GB large spread 35.7% (p10–p90) around a p50 of 2m 38s, min 2m 02s, max 3m 06s, 5 samples
one run p50 p10–p90 min–max
  1. Blacksmith 8 vCPU · 32 GB spread 8.8% (p10–p90) around a p50 of 3m 10s, min 2m 53s, max 3m 15s, 5 samples
  2. Avrea 2 vCPU · 8 GB spread 9% (p10–p90) around a p50 of 2m 14s, min 1m 58s, max 2m 16s, 5 samples
  3. GitHub 4 vCPU · 16 GB spread 9.2% (p10–p90) around a p50 of 2m 58s, min 2m 53s, max 3m 11s, 5 samples
  4. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe spread 12.4% (p10–p90) around a p50 of 1m 48s, min 1m 46s, max 2m 03s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 13.1% (p10–p90) around a p50 of 2m 04s, min 1m 51s, max 2m 10s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 15.4% (p10–p90) around a p50 of 3m 28s, min 3m 24s, max 4m 04s, 5 samples
  7. Ubicloud 8 vCPU · 32 GB standard spread 20.3% (p10–p90) around a p50 of 3m 29s, min 3m 13s, max 4m 23s, 5 samples
one run p50 p10–p90 min–max
  1. GitHub 2 vCPU · 8 GB spread 6.1% (p10–p90) around a p50 of 5m 17s, min 5m 06s, max 5m 27s, 5 samples
  2. RunsOn 2 vCPU · 8 GB m9gd.large · local NVMe spread 6.6% (p10–p90) around a p50 of 2m 53s, min 2m 51s, max 3m 04s, 5 samples
  3. Warpbuild 2 vCPU · 8 GB spread 8.2% (p10–p90) around a p50 of 4m 37s, min 4m 20s, max 4m 54s, 5 samples
  4. Avrea 2 vCPU · 8 GB spread 9% (p10–p90) around a p50 of 2m 14s, min 1m 58s, max 2m 16s, 5 samples
  5. Blacksmith 2 vCPU · 6 GB spread 14.9% (p10–p90) around a p50 of 7m 43s, min 6m 57s, max 8m 43s, 5 samples
  6. Ubicloud 2 vCPU · 8 GB standard spread 15.6% (p10–p90) around a p50 of 7m 37s, min 7m 18s, max 8m 42s, 5 samples
  7. Namespace 2 vCPU · 8 GB 2x8 spread 54.7% (p10–p90) around a p50 of 8m 54s, min 8m 24s, max 16m 04s, 5 samples
one run p50 p10–p90 min–max
  1. Warpbuild 4 vCPU · 16 GB spread 4.4% (p10–p90) around a p50 of 3m 00s, min 2m 54s, max 3m 06s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 9.2% (p10–p90) around a p50 of 2m 58s, min 2m 53s, max 3m 11s, 5 samples
  3. RunsOn 4 vCPU · 16 GB m9gd.xlarge · local NVMe spread 12.4% (p10–p90) around a p50 of 1m 48s, min 1m 46s, max 2m 03s, 5 samples
  4. Blacksmith 4 vCPU · 16 GB spread 14.8% (p10–p90) around a p50 of 4m 36s, min 4m 04s, max 5m 04s, 5 samples
  5. Ubicloud 4 vCPU · 16 GB standard spread 20.2% (p10–p90) around a p50 of 5m 31s, min 4m 13s, max 5m 45s, 5 samples
  6. Avrea 4 vCPU · 16 GB spread 20.4% (p10–p90) around a p50 of 1m 37s, min 1m 22s, max 1m 53s, 5 samples
  7. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 44.2% (p10–p90) around a p50 of 1m 25s, min 1m 19s, max 2m 11s, 5 samples
one run p50 p10–p90 min–max
  1. Blacksmith 8 vCPU · 32 GB spread 8.8% (p10–p90) around a p50 of 3m 10s, min 2m 53s, max 3m 15s, 5 samples
  2. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe spread 10.3% (p10–p90) around a p50 of 1m 18s, min 1m 12s, max 1m 20s, 5 samples
  3. Warpbuild 8 vCPU · 32 GB spread 13.1% (p10–p90) around a p50 of 2m 04s, min 1m 51s, max 2m 10s, 5 samples
  4. Namespace 8 vCPU · 16 GB 8x16 spread 15.4% (p10–p90) around a p50 of 3m 28s, min 3m 24s, max 4m 04s, 5 samples
  5. Avrea 8 vCPU · 32 GB spread 16.4% (p10–p90) around a p50 of 1m 18s, min 59s, max 1m 19s, 5 samples
  6. Ubicloud 8 vCPU · 32 GB standard spread 20.3% (p10–p90) around a p50 of 3m 29s, min 3m 13s, max 4m 23s, 5 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 14.5% (p10–p90) around a p50 of 44s, min 35s, max 45s, 5 samples
  2. Ubicloud 4 vCPU · 16 GB standard spread 15.5% (p10–p90) around a p50 of 1m 25s, min 1m 12s, max 1m 30s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 17% (p10–p90) around a p50 of 33s, min 32s, max 38s, 5 samples
  4. GitHub 8 vCPU · 32 GB spread 26.2% (p10–p90) around a p50 of 1m 01s, min 55s, max 1m 15s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 30.1% (p10–p90) around a p50 of 1m 17s, min 1m 07s, max 1m 45s, 5 samples
  6. StarSling 4 vCPU · 16 GB spread 33.1% (p10–p90) around a p50 of 52s, min 49s, max 1m 17s, 5 samples
  7. Namespace 8 vCPU · 16 GB 8x16 spread 56.1% (p10–p90) around a p50 of 31s, min 29s, max 54s, 5 samples
  8. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 102.8% (p10–p90) around a p50 of 29s, min 29s, max 1m 18s, 5 samples
one run p50 p10–p90 min–max
  1. Blacksmith 2 vCPU · 8 GB spread 12.4% (p10–p90) around a p50 of 1m 37s, min 1m 35s, max 1m 49s, 5 samples
  2. GitHub 2 vCPU · 8 GB p50 3m 50s, min 3m 42s, max 4m 33s, 4 samples, not ranked
  3. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 18.4% (p10–p90) around a p50 of 1m 40s, min 1m 31s, max 1m 57s, 5 samples
  4. Warpbuild 2 vCPU · 8 GB spread 48.3% (p10–p90) around a p50 of 2m 25s, min 1m 54s, max 3m 30s, 5 samples
  5. Ubicloud 2 vCPU · 8 GB premium spread 65% (p10–p90) around a p50 of 2m 28s, min 2m 09s, max 4m 36s, 5 samples
one run p50 p10–p90 min–max
  1. Ubicloud 4 vCPU · 16 GB standard spread 15.5% (p10–p90) around a p50 of 1m 25s, min 1m 12s, max 1m 30s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 21.4% (p10–p90) around a p50 of 1m 56s, min 1m 25s, max 2m 03s, 5 samples
  3. Warpbuild 4 vCPU · 16 GB spread 21.6% (p10–p90) around a p50 of 1m 14s, min 1m 05s, max 1m 29s, 5 samples
  4. Blacksmith 4 vCPU · 16 GB spread 25.2% (p10–p90) around a p50 of 54s, min 48s, max 1m 04s, 5 samples
  5. StarSling 4 vCPU · 16 GB spread 33.1% (p10–p90) around a p50 of 52s, min 49s, max 1m 17s, 5 samples
  6. Namespace 4 vCPU · 16 GB 4x16 spread 65.8% (p10–p90) around a p50 of 52s, min 45s, max 1m 34s, 5 samples
  7. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 75.7% (p10–p90) around a p50 of 46s, min 44s, max 1m 40s, 5 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 14.5% (p10–p90) around a p50 of 44s, min 35s, max 45s, 5 samples
  2. Blacksmith 8 vCPU · 32 GB spread 17% (p10–p90) around a p50 of 33s, min 32s, max 38s, 5 samples
  3. Ubicloud 8 vCPU · 32 GB premium spread 21.4% (p10–p90) around a p50 of 42s, min 37s, max 50s, 5 samples
  4. GitHub 8 vCPU · 32 GB spread 26.2% (p10–p90) around a p50 of 1m 01s, min 55s, max 1m 15s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 30.1% (p10–p90) around a p50 of 1m 17s, min 1m 07s, max 1m 45s, 5 samples
  6. StarSling 8 vCPU · 32 GB spread 32.4% (p10–p90) around a p50 of 42s, min 35s, max 53s, 5 samples
  7. Namespace 8 vCPU · 16 GB 8x16 spread 56.1% (p10–p90) around a p50 of 31s, min 29s, max 54s, 5 samples
  8. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 102.8% (p10–p90) around a p50 of 29s, min 29s, max 1m 18s, 5 samples
one run p50 p10–p90 min–max
  1. GitHub 4 vCPU · 16 GB spread 7.5% (p10–p90) around a p50 of 1m 25s, min 1m 24s, max 1m 34s, 5 samples
  2. Ubicloud 8 vCPU · 32 GB standard spread 16.6% (p10–p90) around a p50 of 1m 17s, min 1m 11s, max 1m 31s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 20% (p10–p90) around a p50 of 1m 19s, min 1m 13s, max 1m 36s, 5 samples
  4. Namespace 8 vCPU · 16 GB 8x16 spread 23.4% (p10–p90) around a p50 of 1m 27s, min 1m 23s, max 1m 55s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 70% (p10–p90) around a p50 of 46s, min 42s, max 1m 33s, 5 samples
  6. RunsOn 4 vCPU · 16 GB m9g.xlarge · tmpfs spread 428.3% (p10–p90) around a p50 of 46s, min 44s, max 6m 11s, 5 samples
one run p50 p10–p90 min–max
  1. Ubicloud 2 vCPU · 8 GB standard p50 4m 38s, min 4m 27s, max 5m 13s, 3 samples, not ranked
  2. RunsOn 2 vCPU · 8 GB m9gd.large · local NVMe spread 19.8% (p10–p90) around a p50 of 1m 43s, min 1m 39s, max 2m 07s, 5 samples
  3. GitHub 2 vCPU · 8 GB p50 3m 12s, min 2m 52s, max 4m 04s, 4 samples, not ranked
  4. Blacksmith 2 vCPU · 6 GB spread 38.9% (p10–p90) around a p50 of 6m 41s, min 5m 20s, max 8m 58s, 5 samples
  5. Warpbuild 2 vCPU · 8 GB spread 123.9% (p10–p90) around a p50 of 3m 24s, min 1m 57s, max 6m 19s, 5 samples
one run p50 p10–p90 min–max
  1. GitHub 4 vCPU · 16 GB spread 7.5% (p10–p90) around a p50 of 1m 25s, min 1m 24s, max 1m 34s, 5 samples
  2. Blacksmith 4 vCPU · 16 GB spread 7.9% (p10–p90) around a p50 of 2m 14s, min 2m 06s, max 2m 23s, 5 samples
  3. Ubicloud 4 vCPU · 16 GB standard spread 12.9% (p10–p90) around a p50 of 2m 04s, min 1m 59s, max 2m 17s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 35.1% (p10–p90) around a p50 of 1m 13s, min 1m 09s, max 1m 49s, 5 samples
  5. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 166.2% (p10–p90) around a p50 of 39s, min 36s, max 2m 18s, 5 samples
  6. RunsOn 4 vCPU · 16 GB m9g.xlarge · tmpfs spread 428.3% (p10–p90) around a p50 of 46s, min 44s, max 6m 11s, 5 samples
one run p50 p10–p90 min–max
  1. Ubicloud 8 vCPU · 32 GB standard spread 16.6% (p10–p90) around a p50 of 1m 17s, min 1m 11s, max 1m 31s, 5 samples
  2. Blacksmith 8 vCPU · 32 GB spread 20% (p10–p90) around a p50 of 1m 19s, min 1m 13s, max 1m 36s, 5 samples
  3. Namespace 8 vCPU · 16 GB 8x16 spread 23.4% (p10–p90) around a p50 of 1m 27s, min 1m 23s, max 1m 55s, 5 samples
  4. Warpbuild 8 vCPU · 32 GB spread 70% (p10–p90) around a p50 of 46s, min 42s, max 1m 33s, 5 samples
  5. RunsOn 8 vCPU · 32 GB m9gd.2xlarge · local NVMe spread 88.4% (p10–p90) around a p50 of 31s, min 29s, max 1m 14s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 7.9% (p10–p90) around a p50 of 3m 54s, min 3m 33s, max 3m 58s, 5 samples
  2. Warpbuild 8 vCPU · 32 GB spread 11.3% (p10–p90) around a p50 of 4m 52s, min 4m 34s, max 5m 22s, 7 samples
  3. Namespace 8 vCPU · 16 GB 8x16 spread 12.3% (p10–p90) around a p50 of 3m 57s, min 3m 52s, max 4m 53s, 7 samples
  4. Blacksmith 4 vCPU · 16 GB spread 14.5% (p10–p90) around a p50 of 5m 22s, min 5m 06s, max 5m 57s, 7 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 18.3% (p10–p90) around a p50 of 13m 09s, min 11m 11s, max 14m 17s, 6 samples
  6. Ubicloud 8 vCPU · 32 GB standard spread 25.7% (p10–p90) around a p50 of 7m 21s, min 7m 09s, max 9m 51s, 7 samples
  7. Avrea 8 vCPU · 32 GB spread 29.1% (p10–p90) around a p50 of 4m 47s, min 4m 23s, max 6m 40s, 7 samples
  8. StarSling 4 vCPU · 16 GB spread 34.1% (p10–p90) around a p50 of 5m 41s, min 5m 24s, max 9m 34s, 7 samples
  9. GitHub 8 vCPU · 32 GB spread 273.9% (p10–p90) around a p50 of 7m 18s, min 6m 04s, max 52m 26s, 7 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 7.4% (p10–p90) around a p50 of 11m 20s, min 11m 05s, max 12m 07s, 6 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 6% (p10–p90) around a p50 of 5m 01s, min 4m 41s, max 5m 06s, 5 samples
  2. Namespace 4 vCPU · 16 GB 4x16 spread 11% (p10–p90) around a p50 of 5m 07s, min 4m 48s, max 5m 47s, 7 samples
  3. Warpbuild 4 vCPU · 16 GB spread 11.3% (p10–p90) around a p50 of 7m 35s, min 6m 54s, max 8m 04s, 7 samples
  4. Avrea 4 vCPU · 16 GB spread 13.8% (p10–p90) around a p50 of 7m 11s, min 6m 23s, max 7m 44s, 7 samples
  5. Blacksmith 4 vCPU · 16 GB spread 14.5% (p10–p90) around a p50 of 5m 22s, min 5m 06s, max 5m 57s, 7 samples
  6. GitHub 4 vCPU · 16 GB spread 16.1% (p10–p90) around a p50 of 12m 56s, min 12m 10s, max 15m 18s, 7 samples
  7. Ubicloud 4 vCPU · 16 GB premium spread 31.6% (p10–p90) around a p50 of 10m 35s, min 9m 57s, max 13m 54s, 7 samples
  8. StarSling 4 vCPU · 16 GB spread 34.1% (p10–p90) around a p50 of 5m 41s, min 5m 24s, max 9m 34s, 7 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 7.9% (p10–p90) around a p50 of 3m 54s, min 3m 33s, max 3m 58s, 5 samples
  2. Warpbuild 8 vCPU · 32 GB spread 11.3% (p10–p90) around a p50 of 4m 52s, min 4m 34s, max 5m 22s, 7 samples
  3. Blacksmith 8 vCPU · 32 GB spread 11.8% (p10–p90) around a p50 of 3m 58s, min 3m 46s, max 4m 29s, 7 samples
  4. Namespace 8 vCPU · 16 GB 8x16 spread 12.3% (p10–p90) around a p50 of 3m 57s, min 3m 52s, max 4m 53s, 7 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 18.3% (p10–p90) around a p50 of 13m 09s, min 11m 11s, max 14m 17s, 6 samples
  6. Ubicloud 8 vCPU · 32 GB standard spread 25.7% (p10–p90) around a p50 of 7m 21s, min 7m 09s, max 9m 51s, 7 samples
  7. Avrea 8 vCPU · 32 GB spread 29.1% (p10–p90) around a p50 of 4m 47s, min 4m 23s, max 6m 40s, 7 samples
  8. StarSling 8 vCPU · 32 GB spread 92.8% (p10–p90) around a p50 of 4m 20s, min 3m 54s, max 8m 17s, 7 samples
  9. GitHub 8 vCPU · 32 GB spread 273.9% (p10–p90) around a p50 of 7m 18s, min 6m 04s, max 52m 26s, 7 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 6.6% (p10–p90) around a p50 of 6m 34s, min 6m 12s, max 6m 43s, 7 samples
  2. Avrea 4 vCPU · 16 GB spread 9.8% (p10–p90) around a p50 of 4m 57s, min 4m 30s, max 5m 26s, 7 samples
  3. RunsOn 4 vCPU · 32 GB r9g.xlarge · tmpfs p50 5m 24s, min 5m 06s, max 5m 49s, 4 samples, not ranked
  4. Namespace 8 vCPU · 16 GB 8x16 spread 20.3% (p10–p90) around a p50 of 10m 12s, min 9m 32s, max 11m 57s, 7 samples
  5. Blacksmith 8 vCPU · 32 GB spread 23.8% (p10–p90) around a p50 of 7m 59s, min 7m 49s, max 11m 31s, 7 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 28.9% (p10–p90) around a p50 of 14m 02s, min 12m 34s, max 17m 18s, 7 samples
  7. GitHub 4 vCPU · 16 GB spread 116.1% (p10–p90) around a p50 of 9m 42s, min 9m 04s, max 35m 34s, 7 samples
one run p50 p10–p90 min–max

No arm64 Docker build finished at 2 vCPU.

  1. Warpbuild 4 vCPU · 16 GB spread 7.7% (p10–p90) around a p50 of 9m 52s, min 9m 09s, max 10m 30s, 7 samples
  2. Avrea 4 vCPU · 16 GB spread 9.8% (p10–p90) around a p50 of 4m 57s, min 4m 30s, max 5m 26s, 7 samples
  3. RunsOn 4 vCPU · 32 GB r9g.xlarge · tmpfs p50 5m 24s, min 5m 06s, max 5m 49s, 4 samples, not ranked
  4. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 12.3% (p10–p90) around a p50 of 4m 34s, min 4m 19s, max 5m 16s, 7 samples
  5. Ubicloud 4 vCPU · 16 GB standard spread 28.9% (p10–p90) around a p50 of 14m 02s, min 12m 34s, max 17m 18s, 7 samples
  6. Blacksmith 4 vCPU · 16 GB spread 34% (p10–p90) around a p50 of 12m 12s, min 11m 05s, max 18m 32s, 7 samples
  7. GitHub 4 vCPU · 16 GB spread 116.1% (p10–p90) around a p50 of 9m 42s, min 9m 04s, max 35m 34s, 7 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 6.6% (p10–p90) around a p50 of 6m 34s, min 6m 12s, max 6m 43s, 7 samples
  2. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 7.5% (p10–p90) around a p50 of 4m 14s, min 3m 58s, max 4m 19s, 5 samples
  3. Namespace 8 vCPU · 16 GB 8x16 spread 20.3% (p10–p90) around a p50 of 10m 12s, min 9m 32s, max 11m 57s, 7 samples
  4. Ubicloud 8 vCPU · 32 GB standard spread 21.7% (p10–p90) around a p50 of 10m 14s, min 9m 40s, max 12m 19s, 7 samples
  5. Avrea 8 vCPU · 32 GB spread 22.8% (p10–p90) around a p50 of 3m 49s, min 3m 32s, max 4m 44s, 7 samples
  6. Blacksmith 8 vCPU · 32 GB spread 23.8% (p10–p90) around a p50 of 7m 59s, min 7m 49s, max 11m 31s, 7 samples
one run p50 p10–p90 min–max

Is a bigger runner always worth it?

What doubling the vCPU did to each build, on each provider's own runner line: each step usually saves less time and costs more per build.

Median across providers: from 4 to 8 vCPU, each build ran 32–35% faster for 31–36% more per build; from 2 to 4 vCPU, 37–47% faster, cost per build +6% to +20%.

Median across providers: from 4 to 8 vCPU, each build ran 32–35% faster for 31–36% more per build; from 2 to 4 vCPU, 37–47% faster, cost per build +6% to +26%.

Median across providers: from 4 to 8 vCPU, each build ran 29–32% faster for 36–46% more per build; from 2 to 4 vCPU, 36–54% faster, cost per build −24% to +24%.

Median across providers: from 4 to 8 vCPU, each build ran 29–32% faster for 36–41% more per build; from 2 to 4 vCPU, 36–54% faster, cost per build −24% to +24%.

  • Each line is one provider's runner at 2, 4 and 8 vCPU, its vCPU count in each dot (hollow: under 3 runs so far); RunsOn's is the bold one (m8a / m9g, not its fastest or cheapest).
  • Right costs more per build, lower is faster: the shaded corner, bottom left, is where you want to be.
  • Dashed lines join equal time × cost. A step that drops more steeply than them bought time cheaply; one that runs flatter mostly bought cost.
Rustspotify-player
faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingRunsOnBlacksmithAvreaWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 32% faster, +36% cost (9 providers)
2 → 4 vCPU: 37% faster, +20% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingRunsOnBlacksmithAvreaWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 32% faster, +36% cost (9 providers)
2 → 4 vCPU: 37% faster, +26% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.0500$0.10002m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnGitHubWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 30% faster, +39% cost (6 providers)
2 → 4 vCPU: 36% faster, +24% cost (7 providers)

faster & cheaper$0.0100$0.0200$0.0500$0.10002m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnGitHubWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 30% faster, +39% cost (6 providers)
2 → 4 vCPU: 36% faster, +24% cost (7 providers)

TypeScriptopencode
faster & cheaper$0.0020$0.0050$0.0100$0.02001m 00s2m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingBlacksmithRunsOnWarpbuildUbicloudGitHub

Median across providers: 4 → 8 vCPU: 34% faster, +31% cost (7 providers)
2 → 4 vCPU: 47% faster, +6% cost (5 providers)

faster & cheaper$0.0050$0.0100$0.02001m 00s2m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingBlacksmithRunsOnWarpbuildUbicloudGitHub

Median across providers: 4 → 8 vCPU: 34% faster, +31% cost (7 providers)
2 → 4 vCPU: 47% faster, +6% cost (5 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓RunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 32% faster, +36% cost (5 providers)
2 → 4 vCPU: 54% faster, −24% cost (5 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05001m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓RunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 32% faster, +36% cost (5 providers)
2 → 4 vCPU: 54% faster, −24% cost (5 providers)

DockerPostHog image
faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceBlacksmithStarSlingAvreaWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 35% faster, +31% cost (8 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceBlacksmithStarSlingAvreaWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 35% faster, +31% cost (8 providers)

faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +46% cost (6 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +41% cost (6 providers)

Beyond builds

The same runners, one question each.

  • CPU Fastest single-thread on x64: Namespace 4x8, 4,685 PassMark.
  • Cache 4 GB restore on x64, any size: RunsOn 9s (12 vCPU), then Blacksmith 13s (2 vCPU); GitHub's own service 28s (8 vCPU).4 GB restore on x64 at 2 vCPU: Blacksmith 13s, then Avrea 15s; GitHub's own service 50s.4 GB restore on x64 at 4 vCPU: Avrea 14s, then Blacksmith 15s; GitHub's own service 34s.4 GB restore on x64 at 8 vCPU: RunsOn 13s, then Avrea 14s; GitHub's own service 28s.
  • Disk Small random reads (QD1) on x64: Blacksmith 4 vCPU leads with 82,138 per second, 47× the slowest disk (AWS CodeBuild, 1,763). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
  • CPU Fastest single-thread on arm64: Avrea 2 vCPU, 5,028 PassMark.
  • Cache 4 GB restore on arm64, any size: Avrea 11s (4 vCPU), then RunsOn 13s (8 vCPU); GitHub's own service 31s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then RunsOn 29s; GitHub's own service 52s.4 GB restore on arm64 at 4 vCPU: Avrea 11s, then RunsOn 18s; GitHub's own service 31s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 13s.
  • Disk Small random reads (QD1) on arm64: Warpbuild xfast (Apple M4 Pro) 6 vCPU leads with 26,620 per second, 3.2× the slowest disk (GitHub, 8,202). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.

Providers

Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.

Head-to-head

Two runners of the same size, build for build. Cost per build decides: a faster runner that costs twice as much per build is not the better runner.

Cost per build is variable cost only; licenses and plan fees are flat, on each provider's page. Estimate your bill