v3.5.1 changelog
quickstart →
Pricing Benchmarks Docs Changelog Blog Troubleshooting GitHub Install in 10 min

GitHub Actions runner benchmarks

At 4 vCPU, RunsOn is the best value on all three builds.

Rust spotify-player

RunsOn c8a.xlarge · EBS gp3

2m 00s $0.0036 per build

GitHub 4-core: 13× the cost, 1.9× the time, 9th of 9

TypeScript opencode

RunsOn r8a.xlarge · tmpfs

1m 02s $0.0029 per build

GitHub 4-core: 8.8× the cost, 2.0× the time, 8th of 8

Docker PostHog image

RunsOn r8a.xlarge · tmpfs

5m 12s $0.0121 per build

GitHub 4-core: 16× the cost, 3.2× the time, 8th of 8

Best value per provider

Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by time × cost per build.; shades are build phases in order.; build time on the right.

Rustspotify-player
  1. RunsOn4 vCPU · 8 GBc8a.xlarge · EBS gp32m 00s, cost per build spot $0.0036, on-demand $0.0090, best-value of 9 machines; phases: setup 14s, cargo test 1m 01s, clippy 34s, runner overhead 11s, ×1.00
  2. Ubicloud4 vCPU · 16 GBpremium2m 26s, cost per build $0.0097, best-value of 2 machines; phases: setup 19s, cargo test 1m 15s, clippy 43s, runner overhead 9s, ×3.27
  3. StarSling4 vCPU · 16 GB1m 45s, cost per build $0.0140; phases: setup 10s, cargo test 53s, clippy 28s, runner overhead 14s, ×3.38
  4. Avrea4 vCPU · 16 GB2m 01s, cost per build $0.0161; phases: setup 12s, cargo test 1m 06s, clippy 34s, runner overhead 9s, ×4.49
  5. Namespace4 vCPU · 16 GB4x161m 43s, cost per build $0.0206, best-value of 2 machines; phases: setup 9s, cargo test 56s, clippy 30s, runner overhead 8s, ×4.88
  6. Blacksmith4 vCPU · 16 GB2m 10s, cost per build $0.0173; phases: setup 16s, cargo test 1m 09s, clippy 35s, runner overhead 10s, ×5.18
  7. Warpbuild4 vCPU · 16 GB2m 17s, cost per build $0.0183; phases: setup 15s, cargo test 1m 16s, clippy 39s, runner overhead 7s, ×5.75
  8. AWS CodeBuild4 vCPU · 8 GBmedium3m 54s, cost per build $0.0390; phases: setup 32s, cargo test 1m 58s, clippy 1m 05s, runner overhead 19s, ×21
  9. GitHub4 vCPU · 16 GB3m 53s, cost per build $0.0466; phases: setup 20s, cargo test 2m 12s, clippy 1m 12s, runner overhead 9s, ×25

Bar shades, light to dark, in execution order: setup = checkout + packages + toolchaincargo fmtcargo testclippyrunner overhead

cost per build× = time × cost against the best

TypeScriptopencode
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs1m 02s, cost per build spot $0.0029, on-demand $0.0072, best-value of 9 machines; phases: toolchain 1s, clone 1s, bun install 9s, typecheck 37s, runner overhead 14s, ×1.00
  2. Avrea4 vCPU · 16 GB43s, cost per build $0.0057; phases: toolchain 1s, clone 4s, bun install 6s, typecheck 21s, runner overhead 11s, ×1.34
  3. StarSling4 vCPU · 16 GB56s, cost per build $0.0075; phases: toolchain 1s, clone 1s, bun install 10s, typecheck 36s, runner overhead 8s, ×2.33
  4. Ubicloud4 vCPU · 16 GBstandard1m 47s, cost per build $0.0045, best-value of 2 machines; phases: toolchain 2s, clone 5s, bun install 24s, typecheck 59s, runner overhead 17s, ×2.66
  5. Blacksmith4 vCPU · 16 GB1m 05s, cost per build $0.0087; phases: toolchain 1s, clone 2s, bun install 10s, typecheck 40s, runner overhead 12s, ×3.14
  6. Warpbuild4 vCPU · 16 GB1m 25s, cost per build $0.0113; phases: toolchain 1s, clone 1s, bun install 14s, typecheck 1m 00s, runner overhead 9s, ×5.38
  7. Namespace4 vCPU · 16 GB4x161m 13s, cost per build $0.0146; phases: toolchain 1s, clone 1s, bun install 29s, typecheck 34s, runner overhead 8s, ×5.95
  8. GitHub4 vCPU · 16 GB2m 07s, cost per build $0.0254; phases: toolchain 1s, clone 3s, bun install 22s, typecheck 1m 31s, runner overhead 10s, ×18
  9. AWS CodeBuildfailed · out of memory

Bar shades, light to dark, in execution order: toolchainclonebun installtypecheckrunner overhead

cost per build× = time × cost against the best

DockerPostHog image
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs5m 12s, cost per build spot $0.0121, on-demand $0.0308, best-value of 8 machines; phases: fetch source 6s, buildx setup 3s, docker build 4m 37s, runner overhead 26s, ×1.00
  2. Blacksmith4 vCPU · 16 GB5m 44s, cost per build $0.0459; phases: fetch source 21s, buildx setup 2s, docker build 4m 55s, runner overhead 26s, ×4.19
  3. StarSling4 vCPU · 16 GB5m 55s, cost per build $0.0473; phases: fetch source 23s, buildx setup 4s, docker build 5m 04s, runner overhead 24s, ×4.47
  4. Ubicloud4 vCPU · 16 GBstandard11m 22s, cost per build $0.0284, best-value of 2 machines; phases: fetch source 18s, buildx setup 12s, docker build 10m 19s, runner overhead 33s, ×5.15
  5. Namespace4 vCPU · 16 GB4x165m 17s, cost per build $0.0634; phases: fetch source 22s, buildx setup 2s, docker build 4m 38s, runner overhead 16s, ×5.34
  6. Avrea4 vCPU · 16 GB6m 42s, cost per build $0.0535; phases: fetch source 21s, buildx setup 3s, docker build 5m 50s, runner overhead 28s, ×5.71
  7. Warpbuild4 vCPU · 16 GB7m 56s, cost per build $0.0634; phases: fetch source 23s, buildx setup 4s, docker build 7m 10s, runner overhead 19s, ×8.01
  8. GitHub4 vCPU · 16 GB16m 27s, cost per build $0.1974; phases: fetch source 25s, buildx setup 12s, docker build 13m 00s, runner overhead 2m 50s, ×52
  9. AWS CodeBuildnot run: needs more than 8 GB of memory

Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead

cost per build× = time × cost against the best

Each provider's best-value machine (lowest time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.

Queue and consistency

How long jobs wait in a burst, and how far build times move from run to run.

Queue under a burst15 jobs at once · each provider's quickest-starting runner at the chosen size · ranked from 3 runs
  1. GitHub 4 vCPU · 16 GB p50 4s, p10–p90 3s to 4s, min 2s, max 5s, 75 samples
  2. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 11s, min 7s, max 14s, 75 samples
  3. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  4. Namespace 8 vCPU · 32 GB 8x32 p50 10s, p10–p90 7s to 17s, min 6s, max 11m 43s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 11s, p10–p90 9s to 10m 10s, min 9s, max 13m 06s, 75 samples
  6. Warpbuild 4 vCPU · 16 GB p50 13s, p10–p90 11s to 13m 29s, min 11s, max 16m 36s, 75 samples
  7. RunsOn 2 vCPU · 8 GB m8azn.large · EBS gp3 p50 16s, p10–p90 15s to 20s, min 14s, max 22s, 75 samples
  8. Ubicloud 8 vCPU · 32 GB premium p50 18s, p10–p90 16s to 29s, min 15s, max 34s, 75 samples
  9. AWS CodeBuild 8 vCPU · 16 GB large p50 20s, p10–p90 18s to 14m 42s, min 17s, max 18m 27s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 4s, p10–p90 2s to 5s, min 2s, max 6s, 75 samples
  2. Blacksmith 2 vCPU · 8 GB p50 9s, p10–p90 7s to 13s, min 7s, max 14s, 75 samples
  3. Namespace 2 vCPU · 8 GB 2x8 p50 10s, p10–p90 7s to 17s, min 6s, max 20s, 75 samples
  4. Avrea 2 vCPU · 8 GB p50 12s, p10–p90 9s to 9m 08s, min 8s, max 11m 13s, 75 samples
  5. Warpbuild 2 vCPU · 8 GB p50 14s, p10–p90 13s to 12m 30s, min 11s, max 14m 42s, 75 samples
  6. RunsOn 2 vCPU · 8 GB m8azn.large · EBS gp3 p50 16s, p10–p90 15s to 20s, min 14s, max 22s, 75 samples
  7. Ubicloud 2 vCPU · 8 GB premium p50 20s, p10–p90 17s to 24s, min 15s, max 26s, 75 samples
  8. AWS CodeBuild 2 vCPU · 3 GB small p50 21s, p10–p90 18s to 10m 46s, min 17s, max 21m 34s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 4s, p10–p90 3s to 4s, min 2s, max 5s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB p50 10s, p10–p90 7s to 12s, min 7s, max 17s, 75 samples
  3. StarSling 4 vCPU · 16 GB p50 10s, p10–p90 7s to 13m 50s, min 7s, max 17m 22s, 75 samples
  4. Avrea 4 vCPU · 16 GB p50 11s, p10–p90 9s to 10m 10s, min 9s, max 13m 06s, 75 samples
  5. Namespace 4 vCPU · 8 GB 4x8 p50 12s, p10–p90 8s to 2m 04s, min 6s, max 2m 06s, 75 samples
  6. Warpbuild 4 vCPU · 16 GB p50 13s, p10–p90 11s to 13m 29s, min 11s, max 16m 36s, 75 samples
  7. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 p50 16s, p10–p90 14s to 21s, min 13s, max 28s, 75 samples
  8. Ubicloud 4 vCPU · 16 GB premium p50 20s, p10–p90 16s to 5m 51s, min 10s, max 10m 54s, 75 samples
  9. AWS CodeBuild 4 vCPU · 8 GB medium p50 21s, p10–p90 19s to 16m 12s, min 18s, max 21m 09s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 9s, p10–p90 8s to 11s, min 7s, max 14s, 75 samples
  2. StarSling 8 vCPU · 32 GB p50 9s, p10–p90 6s to 12m 28s, min 5s, max 16m 38s, 75 samples
  3. Namespace 8 vCPU · 32 GB 8x32 p50 10s, p10–p90 7s to 17s, min 6s, max 11m 43s, 75 samples
  4. Avrea 8 vCPU · 32 GB p50 12s, p10–p90 8s to 13m 21s, min 7s, max 14m 35s, 75 samples
  5. Warpbuild 8 vCPU · 32 GB p50 13s, p10–p90 11s to 14m 26s, min 10s, max 15m 35s, 75 samples
  6. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs p50 16s, p10–p90 14s to 19s, min 13s, max 22s, 75 samples
  7. Ubicloud 8 vCPU · 32 GB premium p50 18s, p10–p90 16s to 29s, min 15s, max 34s, 75 samples
  8. AWS CodeBuild 8 vCPU · 16 GB large p50 20s, p10–p90 18s to 14m 42s, min 17s, max 18m 27s, 75 samples
  9. GitHub 8 vCPU · 32 GB p50 1m 16s, p10–p90 1m 00s to 1m 34s, min 3s, max 2m 12s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 7s, min 4s, max 8s, 75 samples
  2. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 11s, p10–p90 10s to 18s, min 9s, max 21s, 75 samples
  3. Ubicloud 2 vCPU · 8 GB standard p50 12s, p10–p90 8s to 35s, min 8s, max 1m 01s, 75 samples
  4. Blacksmith 8 vCPU · 32 GB p50 14s, p10–p90 11s to 15s, min 10s, max 18s, 75 samples
  5. Warpbuild 6 vCPU · 14 GB xfast (Apple M4 Pro) p50 14s, p10–p90 13s to 16s, min 12s, max 18s, 75 samples
  6. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 19s, min 14s, max 22s, 75 samples
  7. Avrea 4 vCPU · 16 GB p50 22s, p10–p90 19s to 30s, min 18s, max 35s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 2 vCPU · 8 GB p50 7s, p10–p90 6s to 7s, min 5s, max 22s, 75 samples
  2. Namespace 2 vCPU · 8 GB Apple silicon 2x8 p50 11s, p10–p90 10s to 18s, min 9s, max 21s, 75 samples
  3. Ubicloud 2 vCPU · 8 GB standard p50 12s, p10–p90 8s to 35s, min 8s, max 1m 01s, 75 samples
  4. Blacksmith 2 vCPU · 6 GB p50 14s, p10–p90 12s to 17s, min 11s, max 21s, 75 samples
  5. RunsOn 2 vCPU · 4 GB c8g.large · EBS gp3 p50 16s, p10–p90 14s to 19s, min 14s, max 22s, 75 samples
  6. Avrea 2 vCPU · 8 GB p50 23s, p10–p90 18s to 24s, min 17s, max 28s, 75 samples
  7. Warpbuild 2 vCPU · 8 GB p50 39s, p10–p90 36s to 43s, min 8s, max 46s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. GitHub 4 vCPU · 16 GB p50 6s, p10–p90 5s to 7s, min 4s, max 8s, 75 samples
  2. Blacksmith 4 vCPU · 16 GB p50 14s, p10–p90 10s to 17s, min 10s, max 5m 13s, 75 samples
  3. Namespace 4 vCPU · 16 GB Apple silicon 4x16 p50 14s, p10–p90 10s to 32s, min 9s, max 36s, 75 samples
  4. RunsOn 4 vCPU · 8 GB c8g.xlarge · EBS gp3 p50 17s, p10–p90 15s to 21s, min 14s, max 39s, 75 samples
  5. Avrea 4 vCPU · 16 GB p50 22s, p10–p90 19s to 30s, min 18s, max 35s, 75 samples
  6. Ubicloud 4 vCPU · 16 GB standard p50 23s, p10–p90 10s to 34s, min 8s, max 44s, 75 samples
  7. Warpbuild 4 vCPU · 16 GB p50 36s, p10–p90 13s to 42s, min 6s, max 45s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
  1. Blacksmith 8 vCPU · 32 GB p50 14s, p10–p90 11s to 15s, min 10s, max 18s, 75 samples
  2. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs p50 16s, p10–p90 14s to 23s, min 13s, max 25s, 75 samples
  3. Namespace 8 vCPU · 32 GB 8x32 p50 19s, p10–p90 15s to 24s, min 14s, max 2m 40s, 75 samples
  4. Avrea 8 vCPU · 32 GB p50 22s, p10–p90 19s to 1m 52s, min 17s, max 1m 59s, 75 samples
  5. Ubicloud 8 vCPU · 32 GB standard p50 24s, p10–p90 15s to 2m 39s, min 8s, max 4m 19s, 75 samples
  6. Warpbuild 8 vCPU · 32 GB p50 36s, p10–p90 16s to 43s, min 8s, max 49s, 75 samples
one job p50 p10–p90 min–max wait before the job starts, log scale
Build time, run to runeach provider's machine from the cards above, at the chosen size, with 3+ workflow runs · build steps time, one dot per run · % off its own p50
  1. AWS CodeBuild 8 vCPU · 16 GB large spread 8% (p10–p90) around a p50 of 2m 17s, min 2m 05s, max 2m 20s, 5 samples
  2. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 spread 9.5% (p10–p90) around a p50 of 1m 51s, min 1m 47s, max 2m 02s, 5 samples
  3. Avrea 8 vCPU · 32 GB spread 12% (p10–p90) around a p50 of 1m 15s, min 1m 05s, max 1m 18s, 5 samples
  4. StarSling 4 vCPU · 16 GB spread 13.1% (p10–p90) around a p50 of 1m 39s, min 1m 28s, max 1m 43s, 5 samples
  5. Warpbuild 4 vCPU · 16 GB spread 17.7% (p10–p90) around a p50 of 2m 10s, min 1m 59s, max 2m 24s, 5 samples
  6. Blacksmith 4 vCPU · 16 GB spread 22.5% (p10–p90) around a p50 of 2m 00s, min 1m 39s, max 2m 12s, 5 samples
  7. GitHub 8 vCPU · 32 GB spread 22.5% (p10–p90) around a p50 of 2m 23s, min 1m 48s, max 2m 25s, 5 samples
  8. Namespace 8 vCPU · 16 GB 8x16 spread 32.1% (p10–p90) around a p50 of 1m 11s, min 1m 03s, max 1m 31s, 5 samples
  9. Ubicloud 2 vCPU · 8 GB standard spread 91.4% (p10–p90) around a p50 of 3m 59s, min 3m 50s, max 7m 36s, 5 samples
one run p50 p10–p90 min–max
  1. AWS CodeBuild 2 vCPU · 3 GB small spread 5.5% (p10–p90) around a p50 of 6m 32s, min 6m 24s, max 6m 55s, 5 samples
  2. GitHub 2 vCPU · 8 GB spread 8.3% (p10–p90) around a p50 of 7m 00s, min 6m 29s, max 7m 13s, 5 samples
  3. RunsOn 2 vCPU · 8 GB m8a.large · EBS gp3 p50 2m 53s, min 2m 47s, max 3m 18s, 4 samples, not ranked
  4. Warpbuild 2 vCPU · 8 GB spread 14.1% (p10–p90) around a p50 of 3m 37s, min 3m 21s, max 4m 06s, 5 samples
  5. Avrea 2 vCPU · 8 GB spread 15.8% (p10–p90) around a p50 of 3m 05s, min 2m 23s, max 3m 09s, 5 samples
  6. Namespace 2 vCPU · 8 GB 2x8 spread 22.7% (p10–p90) around a p50 of 2m 37s, min 2m 36s, max 3m 34s, 5 samples
  7. Blacksmith 2 vCPU · 8 GB spread 39.4% (p10–p90) around a p50 of 3m 24s, min 3m 01s, max 4m 47s, 5 samples
  8. Ubicloud 2 vCPU · 8 GB standard spread 91.4% (p10–p90) around a p50 of 3m 59s, min 3m 50s, max 7m 36s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 8 GB c8a.xlarge · EBS gp3 spread 9.5% (p10–p90) around a p50 of 1m 51s, min 1m 47s, max 2m 02s, 5 samples
  2. Namespace 4 vCPU · 16 GB 4x16 spread 13.1% (p10–p90) around a p50 of 1m 36s, min 1m 31s, max 1m 52s, 5 samples
  3. StarSling 4 vCPU · 16 GB spread 13.1% (p10–p90) around a p50 of 1m 39s, min 1m 28s, max 1m 43s, 5 samples
  4. AWS CodeBuild 4 vCPU · 8 GB medium spread 14.2% (p10–p90) around a p50 of 3m 35s, min 3m 11s, max 3m 50s, 5 samples
  5. GitHub 4 vCPU · 16 GB spread 15.2% (p10–p90) around a p50 of 3m 45s, min 3m 35s, max 4m 26s, 5 samples
  6. Warpbuild 4 vCPU · 16 GB spread 17.7% (p10–p90) around a p50 of 2m 10s, min 1m 59s, max 2m 24s, 5 samples
  7. Avrea 4 vCPU · 16 GB spread 18.6% (p10–p90) around a p50 of 1m 51s, min 1m 25s, max 1m 56s, 5 samples
  8. Blacksmith 4 vCPU · 16 GB spread 22.5% (p10–p90) around a p50 of 2m 00s, min 1m 39s, max 2m 12s, 5 samples
  9. Ubicloud 4 vCPU · 16 GB premium spread 78.4% (p10–p90) around a p50 of 2m 14s, min 1m 54s, max 4m 07s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · EBS gp3 spread 2.2% (p10–p90) around a p50 of 1m 13s, min 1m 12s, max 1m 14s, 5 samples
  2. AWS CodeBuild 8 vCPU · 16 GB large spread 8% (p10–p90) around a p50 of 2m 17s, min 2m 05s, max 2m 20s, 5 samples
  3. Avrea 8 vCPU · 32 GB spread 12% (p10–p90) around a p50 of 1m 15s, min 1m 05s, max 1m 18s, 5 samples
  4. Ubicloud 8 vCPU · 32 GB standard spread 15.3% (p10–p90) around a p50 of 1m 56s, min 1m 38s, max 2m 03s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 20.4% (p10–p90) around a p50 of 1m 30s, min 1m 27s, max 1m 47s, 5 samples
  6. GitHub 8 vCPU · 32 GB spread 22.5% (p10–p90) around a p50 of 2m 23s, min 1m 48s, max 2m 25s, 5 samples
  7. StarSling 8 vCPU · 32 GB spread 29.7% (p10–p90) around a p50 of 1m 12s, min 1m 01s, max 1m 30s, 5 samples
  8. Namespace 8 vCPU · 16 GB 8x16 spread 32.1% (p10–p90) around a p50 of 1m 11s, min 1m 03s, max 1m 31s, 5 samples
  9. Blacksmith 8 vCPU · 32 GB spread 47.9% (p10–p90) around a p50 of 1m 26s, min 1m 11s, max 2m 07s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m9g.xlarge · EBS gp3 spread 2% (p10–p90) around a p50 of 1m 48s, min 1m 46s, max 1m 49s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 8.9% (p10–p90) around a p50 of 3m 16s, min 2m 53s, max 3m 18s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 10% (p10–p90) around a p50 of 3m 02s, min 2m 59s, max 3m 24s, 5 samples
  4. Avrea 2 vCPU · 8 GB spread 12% (p10–p90) around a p50 of 1m 57s, min 1m 54s, max 2m 16s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 13.3% (p10–p90) around a p50 of 1m 59s, min 1m 53s, max 2m 10s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 26.1% (p10–p90) around a p50 of 4m 17s, min 4m 12s, max 5m 52s, 5 samples
  7. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 40% (p10–p90) around a p50 of 1m 29s, min 1m 19s, max 2m 11s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 8 GB m9gd.large · local NVMe spread 0.9% (p10–p90) around a p50 of 2m 52s, min 2m 52s, max 2m 54s, 5 samples
  2. Warpbuild 2 vCPU · 8 GB spread 9.9% (p10–p90) around a p50 of 4m 46s, min 4m 39s, max 5m 17s, 5 samples
  3. Avrea 2 vCPU · 8 GB spread 12% (p10–p90) around a p50 of 1m 57s, min 1m 54s, max 2m 16s, 5 samples
  4. GitHub 2 vCPU · 8 GB spread 13.1% (p10–p90) around a p50 of 5m 27s, min 5m 07s, max 6m 09s, 5 samples
  5. Namespace 2 vCPU · 8 GB Apple silicon 2x8 spread 14.3% (p10–p90) around a p50 of 2m 09s, min 2m 04s, max 2m 24s, 5 samples
  6. Blacksmith 2 vCPU · 6 GB spread 16.7% (p10–p90) around a p50 of 7m 41s, min 6m 48s, max 8m 43s, 5 samples
  7. Ubicloud 2 vCPU · 8 GB standard spread 34% (p10–p90) around a p50 of 7m 46s, min 7m 01s, max 10m 25s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 16 GB m9g.xlarge · EBS gp3 spread 2% (p10–p90) around a p50 of 1m 48s, min 1m 46s, max 1m 49s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 8.9% (p10–p90) around a p50 of 3m 16s, min 2m 53s, max 3m 18s, 5 samples
  3. Blacksmith 4 vCPU · 16 GB spread 9.6% (p10–p90) around a p50 of 4m 49s, min 4m 24s, max 4m 58s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 10% (p10–p90) around a p50 of 3m 02s, min 2m 41s, max 3m 10s, 5 samples
  5. Avrea 4 vCPU · 16 GB spread 16.3% (p10–p90) around a p50 of 1m 20s, min 1m 16s, max 1m 37s, 5 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 26.1% (p10–p90) around a p50 of 4m 17s, min 4m 12s, max 5m 52s, 5 samples
  7. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 40% (p10–p90) around a p50 of 1m 29s, min 1m 19s, max 2m 11s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m9g.2xlarge · EBS gp3 spread 7.8% (p10–p90) around a p50 of 1m 14s, min 1m 12s, max 1m 19s, 5 samples
  2. Blacksmith 8 vCPU · 32 GB spread 10% (p10–p90) around a p50 of 3m 02s, min 2m 59s, max 3m 24s, 5 samples
  3. Warpbuild 8 vCPU · 32 GB spread 13.3% (p10–p90) around a p50 of 1m 59s, min 1m 53s, max 2m 10s, 5 samples
  4. Avrea 8 vCPU · 32 GB spread 18.7% (p10–p90) around a p50 of 1m 01s, min 1m 00s, max 1m 19s, 5 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 22.3% (p10–p90) around a p50 of 3m 59s, min 3m 25s, max 4m 23s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 120% (p10–p90) around a p50 of 3m 59s, min 3m 37s, max 11m 29s, 5 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 19.1% (p10–p90) around a p50 of 45s, min 44s, max 55s, 5 samples
  2. Ubicloud 4 vCPU · 16 GB standard spread 22.9% (p10–p90) around a p50 of 1m 30s, min 1m 25s, max 1m 54s, 5 samples
  3. GitHub 8 vCPU · 32 GB spread 29% (p10–p90) around a p50 of 1m 02s, min 55s, max 1m 15s, 5 samples
  4. AWS CodeBuild 8 vCPU · 16 GB large spread 45.5% (p10–p90) around a p50 of 1m 02s, min 1m 02s, max 1m 45s, 5 samples
  5. StarSling 8 vCPU · 32 GB spread 45.8% (p10–p90) around a p50 of 31s, min 30s, max 53s, 5 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 45.9% (p10–p90) around a p50 of 41s, min 28s, max 54s, 5 samples
  7. Blacksmith 4 vCPU · 16 GB spread 50.4% (p10–p90) around a p50 of 54s, min 50s, max 1m 26s, 5 samples
  8. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 104.8% (p10–p90) around a p50 of 29s, min 28s, max 1m 18s, 5 samples
  9. Avrea 4 vCPU · 16 GB p50 32s, min 10s, max 53s, 4 samples, not ranked
one run p50 p10–p90 min–max
  1. Ubicloud 2 vCPU · 8 GB standard spread 32.5% (p10–p90) around a p50 of 3m 06s, min 2m 11s, max 3m 23s, 5 samples
  2. RunsOn 2 vCPU · 16 GB r8a.large · tmpfs spread 32.9% (p10–p90) around a p50 of 1m 25s, min 1m 25s, max 2m 11s, 5 samples
  3. Warpbuild 2 vCPU · 8 GB spread 33.5% (p10–p90) around a p50 of 2m 22s, min 1m 54s, max 3m 08s, 5 samples
  4. Blacksmith 2 vCPU · 8 GB spread 50.2% (p10–p90) around a p50 of 1m 46s, min 1m 25s, max 2m 43s, 5 samples
one run p50 p10–p90 min–max
2 providers have under 3 runs so far

Avrea and GitHub.

  1. Warpbuild 4 vCPU · 16 GB spread 12.3% (p10–p90) around a p50 of 1m 15s, min 1m 12s, max 1m 24s, 5 samples
  2. Ubicloud 4 vCPU · 16 GB standard spread 22.9% (p10–p90) around a p50 of 1m 30s, min 1m 25s, max 1m 54s, 5 samples
  3. GitHub 4 vCPU · 16 GB spread 26.8% (p10–p90) around a p50 of 1m 54s, min 1m 29s, max 2m 06s, 5 samples
  4. StarSling 4 vCPU · 16 GB spread 45.4% (p10–p90) around a p50 of 48s, min 44s, max 1m 17s, 5 samples
  5. Blacksmith 4 vCPU · 16 GB spread 50.4% (p10–p90) around a p50 of 54s, min 50s, max 1m 26s, 5 samples
  6. Namespace 4 vCPU · 16 GB 4x16 spread 50.8% (p10–p90) around a p50 of 1m 05s, min 49s, max 1m 34s, 5 samples
  7. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 67.8% (p10–p90) around a p50 of 49s, min 46s, max 1m 40s, 5 samples
  8. Avrea 4 vCPU · 16 GB p50 32s, min 10s, max 53s, 4 samples, not ranked
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 19.1% (p10–p90) around a p50 of 45s, min 44s, max 55s, 5 samples
  2. GitHub 8 vCPU · 32 GB spread 29% (p10–p90) around a p50 of 1m 02s, min 55s, max 1m 15s, 5 samples
  3. Ubicloud 8 vCPU · 32 GB premium spread 32.4% (p10–p90) around a p50 of 50s, min 38s, max 57s, 5 samples
  4. Blacksmith 8 vCPU · 32 GB spread 41.1% (p10–p90) around a p50 of 37s, min 30s, max 54s, 5 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 45.5% (p10–p90) around a p50 of 1m 02s, min 1m 02s, max 1m 45s, 5 samples
  6. StarSling 8 vCPU · 32 GB spread 45.8% (p10–p90) around a p50 of 31s, min 30s, max 53s, 5 samples
  7. Namespace 8 vCPU · 16 GB 8x16 spread 45.9% (p10–p90) around a p50 of 41s, min 28s, max 54s, 5 samples
  8. Avrea 8 vCPU · 32 GB p50 22s, min 10s, max 32s, 4 samples, not ranked
  9. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 104.8% (p10–p90) around a p50 of 29s, min 28s, max 1m 18s, 5 samples
one run p50 p10–p90 min–max
  1. GitHub 4 vCPU · 16 GB spread 9.3% (p10–p90) around a p50 of 1m 24s, min 1m 22s, max 1m 35s, 5 samples
  2. Ubicloud 8 vCPU · 32 GB standard spread 16.5% (p10–p90) around a p50 of 1m 21s, min 1m 14s, max 1m 31s, 5 samples
  3. Blacksmith 8 vCPU · 32 GB spread 16.8% (p10–p90) around a p50 of 1m 21s, min 1m 18s, max 1m 36s, 5 samples
  4. Namespace 8 vCPU · 16 GB 8x16 spread 21.6% (p10–p90) around a p50 of 1m 54s, min 1m 34s, max 2m 07s, 5 samples
  5. Warpbuild 8 vCPU · 32 GB spread 71.2% (p10–p90) around a p50 of 43s, min 42s, max 1m 33s, 5 samples
  6. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 74.5% (p10–p90) around a p50 of 29s, min 29s, max 1m 05s, 5 samples
  7. Avrea 4 vCPU · 16 GB p50 26s, min 14s, max 44s, 4 samples, not ranked
one run p50 p10–p90 min–max
  1. GitHub 2 vCPU · 8 GB p50 3m 00s, min 2m 55s, max 3m 14s, 4 samples, not ranked
  2. Blacksmith 2 vCPU · 6 GB spread 17.2% (p10–p90) around a p50 of 6m 41s, min 5m 21s, max 7m 03s, 5 samples
  3. Warpbuild 2 vCPU · 8 GB p50 5m 13s, min 3m 25s, max 7m 23s, 4 samples, not ranked
  4. RunsOn 2 vCPU · 16 GB r9g.large · tmpfs spread 72.1% (p10–p90) around a p50 of 1m 26s, min 1m 22s, max 2m 50s, 5 samples
one run p50 p10–p90 min–max
2 providers have under 3 runs so far

Avrea and Ubicloud.

  1. Blacksmith 4 vCPU · 16 GB spread 8.2% (p10–p90) around a p50 of 2m 24s, min 2m 16s, max 2m 35s, 5 samples
  2. GitHub 4 vCPU · 16 GB spread 9.3% (p10–p90) around a p50 of 1m 24s, min 1m 22s, max 1m 35s, 5 samples
  3. Ubicloud 4 vCPU · 16 GB standard spread 14% (p10–p90) around a p50 of 2m 06s, min 1m 45s, max 2m 13s, 5 samples
  4. Warpbuild 4 vCPU · 16 GB spread 40.5% (p10–p90) around a p50 of 1m 16s, min 1m 05s, max 1m 49s, 5 samples
  5. Avrea 4 vCPU · 16 GB p50 26s, min 14s, max 44s, 4 samples, not ranked
  6. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 169.5% (p10–p90) around a p50 of 55s, min 40s, max 2m 18s, 5 samples
  7. RunsOn 4 vCPU · 16 GB m9g.xlarge · tmpfs spread 418.3% (p10–p90) around a p50 of 47s, min 44s, max 6m 11s, 5 samples
one run p50 p10–p90 min–max
  1. Ubicloud 8 vCPU · 32 GB standard spread 16.5% (p10–p90) around a p50 of 1m 21s, min 1m 14s, max 1m 31s, 5 samples
  2. Blacksmith 8 vCPU · 32 GB spread 16.8% (p10–p90) around a p50 of 1m 21s, min 1m 18s, max 1m 36s, 5 samples
  3. Namespace 8 vCPU · 16 GB 8x16 spread 21.6% (p10–p90) around a p50 of 1m 54s, min 1m 34s, max 2m 07s, 5 samples
  4. Avrea 8 vCPU · 32 GB p50 25s, min 14s, max 30s, 4 samples, not ranked
  5. Warpbuild 8 vCPU · 32 GB spread 71.2% (p10–p90) around a p50 of 43s, min 42s, max 1m 33s, 5 samples
  6. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 74.5% (p10–p90) around a p50 of 29s, min 29s, max 1m 05s, 5 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 4% (p10–p90) around a p50 of 3m 37s, min 3m 32s, max 3m 45s, 5 samples
  2. Warpbuild 8 vCPU · 32 GB spread 6.4% (p10–p90) around a p50 of 4m 54s, min 4m 45s, max 5m 11s, 7 samples
  3. AWS CodeBuild 8 vCPU · 16 GB large spread 16.1% (p10–p90) around a p50 of 11m 22s, min 10m 57s, max 13m 45s, 6 samples
  4. Blacksmith 4 vCPU · 16 GB spread 17% (p10–p90) around a p50 of 5m 17s, min 4m 59s, max 5m 57s, 7 samples
  5. Namespace 8 vCPU · 16 GB 8x16 spread 20.2% (p10–p90) around a p50 of 3m 52s, min 3m 29s, max 4m 53s, 7 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 33.4% (p10–p90) around a p50 of 10m 45s, min 10m 02s, max 15m 39s, 7 samples
  7. StarSling 4 vCPU · 16 GB spread 34.2% (p10–p90) around a p50 of 5m 28s, min 5m 16s, max 9m 34s, 7 samples
  8. Avrea 8 vCPU · 32 GB spread 37.3% (p10–p90) around a p50 of 4m 23s, min 3m 26s, max 6m 40s, 7 samples
  9. GitHub 8 vCPU · 32 GB spread 286% (p10–p90) around a p50 of 6m 56s, min 6m 07s, max 52m 26s, 7 samples
one run p50 p10–p90 min–max
  1. RunsOn 2 vCPU · 16 GB r8a.large · EBS gp3 spread 10% (p10–p90) around a p50 of 11m 13s, min 10m 57s, max 12m 13s, 6 samples
one run p50 p10–p90 min–max
  1. RunsOn 4 vCPU · 32 GB r8a.xlarge · tmpfs spread 4.5% (p10–p90) around a p50 of 4m 52s, min 4m 41s, max 4m 58s, 5 samples
  2. Warpbuild 4 vCPU · 16 GB spread 5.2% (p10–p90) around a p50 of 7m 35s, min 7m 17s, max 7m 48s, 6 samples
  3. GitHub 4 vCPU · 16 GB spread 16.3% (p10–p90) around a p50 of 13m 38s, min 12m 44s, max 15m 18s, 7 samples
  4. Blacksmith 4 vCPU · 16 GB spread 17% (p10–p90) around a p50 of 5m 17s, min 4m 59s, max 5m 57s, 7 samples
  5. Namespace 4 vCPU · 16 GB 4x16 spread 18.7% (p10–p90) around a p50 of 4m 54s, min 4m 30s, max 5m 47s, 6 samples
  6. Ubicloud 4 vCPU · 16 GB standard spread 33.4% (p10–p90) around a p50 of 10m 45s, min 10m 02s, max 15m 39s, 7 samples
  7. Avrea 4 vCPU · 16 GB spread 33.4% (p10–p90) around a p50 of 6m 15s, min 4m 48s, max 7m 44s, 6 samples
  8. StarSling 4 vCPU · 16 GB spread 34.2% (p10–p90) around a p50 of 5m 28s, min 5m 16s, max 9m 34s, 7 samples
one run p50 p10–p90 min–max
  1. RunsOn 8 vCPU · 32 GB m8a.2xlarge · tmpfs spread 4% (p10–p90) around a p50 of 3m 37s, min 3m 32s, max 3m 45s, 5 samples
  2. Warpbuild 8 vCPU · 32 GB spread 6.4% (p10–p90) around a p50 of 4m 54s, min 4m 45s, max 5m 11s, 7 samples
  3. Blacksmith 8 vCPU · 32 GB spread 14.6% (p10–p90) around a p50 of 3m 56s, min 3m 39s, max 4m 29s, 7 samples
  4. StarSling 8 vCPU · 32 GB spread 15.6% (p10–p90) around a p50 of 4m 13s, min 3m 54s, max 5m 10s, 7 samples
  5. AWS CodeBuild 8 vCPU · 16 GB large spread 16.1% (p10–p90) around a p50 of 11m 22s, min 10m 57s, max 13m 45s, 6 samples
  6. Namespace 8 vCPU · 16 GB 8x16 spread 20.2% (p10–p90) around a p50 of 3m 52s, min 3m 29s, max 4m 53s, 7 samples
  7. Ubicloud 8 vCPU · 32 GB standard spread 30.2% (p10–p90) around a p50 of 7m 50s, min 7m 16s, max 9m 51s, 7 samples
  8. Avrea 8 vCPU · 32 GB spread 37.3% (p10–p90) around a p50 of 4m 23s, min 3m 26s, max 6m 40s, 7 samples
  9. GitHub 8 vCPU · 32 GB spread 286% (p10–p90) around a p50 of 6m 56s, min 6m 07s, max 52m 26s, 7 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 7.8% (p10–p90) around a p50 of 6m 34s, min 6m 02s, max 6m 43s, 7 samples
  2. Namespace 8 vCPU · 16 GB 8x16 spread 10.1% (p10–p90) around a p50 of 10m 12s, min 9m 39s, max 11m 42s, 7 samples
  3. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 10.9% (p10–p90) around a p50 of 3m 59s, min 3m 45s, max 4m 23s, 5 samples
  4. Blacksmith 8 vCPU · 32 GB spread 13.3% (p10–p90) around a p50 of 8m 08s, min 7m 34s, max 9m 01s, 7 samples
  5. Ubicloud 4 vCPU · 16 GB standard spread 15% (p10–p90) around a p50 of 13m 53s, min 13m 11s, max 17m 09s, 7 samples
  6. Avrea 4 vCPU · 16 GB spread 19.5% (p10–p90) around a p50 of 5m 09s, min 4m 30s, max 6m 01s, 7 samples
  7. GitHub 4 vCPU · 16 GB spread 120.5% (p10–p90) around a p50 of 9m 25s, min 9m 04s, max 35m 34s, 7 samples
one run p50 p10–p90 min–max

No arm64 Docker build finished at 2 vCPU.

  1. RunsOn 4 vCPU · 32 GB r9g.xlarge · tmpfs spread 3.9% (p10–p90) around a p50 of 5m 11s, min 5m 06s, max 5m 20s, 5 samples
  2. Warpbuild 4 vCPU · 16 GB spread 10.8% (p10–p90) around a p50 of 10m 02s, min 9m 09s, max 10m 52s, 7 samples
  3. Namespace 4 vCPU · 16 GB Apple silicon 4x16 spread 12.8% (p10–p90) around a p50 of 4m 31s, min 4m 19s, max 5m 05s, 6 samples
  4. Ubicloud 4 vCPU · 16 GB standard spread 15% (p10–p90) around a p50 of 13m 53s, min 13m 11s, max 17m 09s, 7 samples
  5. Avrea 4 vCPU · 16 GB spread 19.5% (p10–p90) around a p50 of 5m 09s, min 4m 30s, max 6m 01s, 7 samples
  6. Blacksmith 4 vCPU · 16 GB spread 44% (p10–p90) around a p50 of 12m 40s, min 11m 05s, max 18m 32s, 7 samples
  7. GitHub 4 vCPU · 16 GB spread 120.5% (p10–p90) around a p50 of 9m 25s, min 9m 04s, max 35m 34s, 7 samples
one run p50 p10–p90 min–max
  1. Warpbuild 8 vCPU · 32 GB spread 7.8% (p10–p90) around a p50 of 6m 34s, min 6m 02s, max 6m 43s, 7 samples
  2. Namespace 8 vCPU · 16 GB 8x16 spread 10.1% (p10–p90) around a p50 of 10m 12s, min 9m 39s, max 11m 42s, 7 samples
  3. RunsOn 8 vCPU · 32 GB m9g.2xlarge · tmpfs spread 10.9% (p10–p90) around a p50 of 3m 59s, min 3m 45s, max 4m 23s, 5 samples
  4. Blacksmith 8 vCPU · 32 GB spread 13.3% (p10–p90) around a p50 of 8m 08s, min 7m 34s, max 9m 01s, 7 samples
  5. Ubicloud 8 vCPU · 32 GB standard spread 22.3% (p10–p90) around a p50 of 10m 23s, min 9m 43s, max 12m 19s, 7 samples
  6. Avrea 8 vCPU · 32 GB spread 23.5% (p10–p90) around a p50 of 4m 16s, min 3m 36s, max 4m 44s, 7 samples
one run p50 p10–p90 min–max

Is a bigger runner always worth it?

What doubling the vCPU did to each build, on each provider's own runner line: each step usually saves less time and costs more per build.

Median across providers: from 4 to 8 vCPU, each build ran 28% faster for 43–44% more per build; from 2 to 4 vCPU, 39–46% faster, cost per build +9% to +21%.

Median across providers: from 4 to 8 vCPU, each build ran 28% faster for 43–44% more per build; from 2 to 4 vCPU, 39–46% faster, cost per build +9% to +21%.

Median across providers: from 4 to 8 vCPU, each build ran 27–31% faster for 38–46% more per build; from 2 to 4 vCPU, 38–57% faster, cost per build −24% to +14%.

Median across providers: from 4 to 8 vCPU, each build ran 27–31% faster for 38–47% more per build; from 2 to 4 vCPU, 38–57% faster, cost per build −24% to +21%.

  • Each line is one provider's runner at 2, 4 and 8 vCPU, its vCPU count in each dot (hollow: under 3 runs so far); RunsOn's is the bold one (m8a / m9g, not its fastest or cheapest).
  • Right costs more per build, lower is faster: the shaded corner, bottom left, is where you want to be.
  • Dashed lines join equal time × cost. A step that drops more steeply than them bought time cheaply; one that runs flatter mostly bought cost.
Rustspotify-player
faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingRunsOnAvreaBlacksmithWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 28% faster, +43% cost (9 providers)
2 → 4 vCPU: 39% faster, +21% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.05002m 00s5m 00ssame time × costcost per build →build time · faster ↓NamespaceStarSlingRunsOnAvreaBlacksmithWarpbuildUbicloudGitHubAWS CodeBuild

Median across providers: 4 → 8 vCPU: 28% faster, +43% cost (9 providers)
2 → 4 vCPU: 39% faster, +21% cost (8 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 27% faster, +46% cost (6 providers)
2 → 4 vCPU: 38% faster, +14% cost (7 providers)

faster & cheaper$0.0100$0.0200$0.0500$0.10001m 00s2m 00s5m 00s10m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 27% faster, +47% cost (6 providers)
2 → 4 vCPU: 38% faster, +21% cost (7 providers)

TypeScriptopencode
faster & cheaper$0.0020$0.0050$0.0100$0.020030s1m 00s2m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingBlacksmithNamespaceRunsOnWarpbuildUbicloudGitHub

Median across providers: 4 → 8 vCPU: 28% faster, +44% cost (8 providers)
2 → 4 vCPU: 46% faster, +9% cost (4 providers)

faster & cheaper$0.0020$0.0050$0.0100$0.020030s1m 00s2m 00ssame time × costcost per build →build time · faster ↓AvreaStarSlingBlacksmithNamespaceRunsOnWarpbuildUbicloudGitHub

Median across providers: 4 → 8 vCPU: 28% faster, +44% cost (8 providers)
2 → 4 vCPU: 46% faster, +9% cost (4 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.050030s1m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 31% faster, +38% cost (6 providers)
2 → 4 vCPU: 57% faster, −24% cost (4 providers)

faster & cheaper$0.0050$0.0100$0.0200$0.050030s1m 00s2m 00s5m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildGitHubUbicloudBlacksmithNamespace

Median across providers: 4 → 8 vCPU: 31% faster, +38% cost (6 providers)
2 → 4 vCPU: 57% faster, −24% cost (4 providers)

DockerPostHog image
faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceBlacksmithStarSlingAvreaWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 28% faster, +44% cost (8 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓NamespaceBlacksmithStarSlingAvreaWarpbuildRunsOnUbicloudGitHub

Median across providers: 4 → 8 vCPU: 28% faster, +44% cost (8 providers)

faster & cheaper$0.0200$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +40% cost (6 providers)

faster & cheaper$0.0500$0.1000$0.20005m 00s10m 00s15m 00ssame time × costcost per build →build time · faster ↓AvreaRunsOnWarpbuildBlacksmithNamespaceUbicloud

Median across providers: 4 → 8 vCPU: 29% faster, +40% cost (6 providers)

Beyond builds

The same runners, one question each.

  • CPU Fastest single-thread on x64: Namespace 4x8, 4,675 PassMark.
  • Cache 4 GB restore on x64, any size: RunsOn 11s (12 vCPU), then Avrea 13s (2 vCPU); GitHub's own service 30s (8 vCPU).4 GB restore on x64 at 2 vCPU: Avrea 13s, then Blacksmith 14s; GitHub's own service 1m 12s.4 GB restore on x64 at 4 vCPU: Avrea 13s, then Blacksmith 14s; GitHub's own service 38s.4 GB restore on x64 at 8 vCPU: Avrea 13s, then Blacksmith 14s; GitHub's own service 30s.
  • Disk Small random reads (QD1) on x64: Namespace 8x32 leads with 80,310 per second, 45× the slowest disk (AWS CodeBuild, 1,766). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
  • CPU Fastest single-thread on arm64: Avrea 2 vCPU, 5,028 PassMark.
  • Cache 4 GB restore on arm64, any size: Avrea 11s (8 vCPU), then RunsOn 14s (8 vCPU); GitHub's own service 29s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then Namespace 19s; GitHub's own service 51s.4 GB restore on arm64 at 4 vCPU: Avrea 12s, then RunsOn 18s; GitHub's own service 29s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 14s.
  • Disk Small random reads (QD1) on arm64: Warpbuild xfast (Apple M4 Pro) 6 vCPU leads with 27,941 per second, 3.5× the slowest disk (GitHub, 7,891). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.

Providers

Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.

Head-to-head

Two runners of the same size, build for build. Cost per build decides: a faster runner that costs twice as much per build is not the better runner.

Cost per build is variable cost only; licenses and plan fees are flat, on each provider's page. Estimate your bill