quickstart →

GitHub Actions runner build benchmarks

At 4 vCPU, RunsOn is the best value on Rust and Docker, and ties with boat on TypeScript.

Rust spotify-player

RunsOn c8a.xlarge · EBS gp3

2m 02s $0.0036 per build

GitHub 4-core: 13× the cost, 1.9× the time, 9th of 9

TypeScript opencode

boat 4 vCPU ≈ RunsOn r8a.xlarge · tmpfs

1m 36s $0.0010 per build

GitHub 4-core: 26× the cost, 1.3× the time, 8th of 8

Docker PostHog image

RunsOn r8a.xlarge · tmpfs

5m 15s $0.0087 per build

GitHub 4-core: 20× the cost, 2.8× the time, 7th of 7

Best value per provider

Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by p90 time × cost per build.; shades are build phases in order.; build time on the right.

Rustspotify-player
  1. RunsOn4 vCPU · 8 GBc8a.xlarge · EBS gp32m 02s, cost per build spot $0.0036, on-demand $0.0087, best-value of 9 machines; phases: setup 15s, cargo test 1m 01s, clippy 34s, runner overhead 12s, ×1.00
  2. Avrea4 vCPU · 16 GB1m 37s, cost per build $0.0129; phases: setup 13s, cargo test 49s, clippy 25s, runner overhead 10s, ×2.98
  3. Namespace4 vCPU · 8 GB4x81m 46s, cost per build $0.0106, best-value of 2 machines; phases: setup 10s, cargo test 52s, clippy 29s, runner overhead 15s, ×3.16
  4. Ubicloud4 vCPU · 16 GBstandard2m 38s, cost per build $0.0066, best-value of 2 machines; phases: setup 23s, cargo test 1m 18s, clippy 42s, runner overhead 15s, ×4.42
  5. boat4 vCPU · 8 GB4m 48s, cost per build $0.0029; phases: setup 48s, cargo test 2m 16s, clippy 1m 06s, runner overhead 39s, ×5.41
  6. Warpbuild4 vCPU · 16 GB2m 23s, cost per build $0.0191; phases: setup 12s, cargo test 1m 22s, clippy 43s, runner overhead 6s, ×7.41
  7. Blacksmith4 vCPU · 16 GB1m 59s, cost per build $0.0159; phases: setup 17s, cargo test 57s, clippy 32s, runner overhead 13s, ×18
  8. AWS CodeBuild4 vCPU · 8 GBmedium3m 48s, cost per build $0.0380; phases: setup 32s, cargo test 1m 55s, clippy 1m 04s, runner overhead 17s, ×20
  9. GitHub4 vCPU · 16 GB3m 48s, cost per build $0.0456; phases: setup 21s, cargo test 2m 05s, clippy 1m 10s, runner overhead 12s, ×31

Bar shades, light to dark, in execution order: setup = checkout + packages + toolchaincargo fmtcargo testclippyrunner overhead

cost per build× = p90 time × cost against the best

TypeScriptopencode
  1. boat4 vCPU · 8 GB1m 36s, cost per build $0.0010; phases: toolchain 1s, clone 4s, bun install 14s, typecheck 1m 00s, runner overhead 17s, ×1.00
  2. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs1m 03s, cost per build spot $0.0021, on-demand $0.0074, best-value of 9 machines; phases: toolchain 1s, clone 1s, bun install 9s, typecheck 38s, runner overhead 15s, ×1.02
  3. Avrea4 vCPU · 16 GB50s, cost per build $0.0067; phases: toolchain 1s, clone 3s, bun install 5s, typecheck 30s, runner overhead 11s, ×2.60
  4. Ubicloud4 vCPU · 16 GBstandard1m 52s, cost per build $0.0047, best-value of 2 machines; phases: toolchain 2s, clone 5s, bun install 25s, typecheck 1m 01s, runner overhead 21s, ×4.18
  5. Namespace4 vCPU · 16 GB4x1653s, cost per build $0.0106, best-value of 2 machines; phases: toolchain 1s, clone 1s, bun install 11s, typecheck 33s, runner overhead 8s, ×6.02
  6. Warpbuild4 vCPU · 16 GB1m 21s, cost per build $0.0108; phases: toolchain 1s, clone 1s, bun install 14s, typecheck 56s, runner overhead 10s, ×7.21
  7. Blacksmith4 vCPU · 16 GB1m 45s, cost per build $0.0139; phases: toolchain 1s, clone 2s, bun install 12s, typecheck 1m 05s, runner overhead 25s, ×14
  8. GitHub4 vCPU · 16 GB2m 05s, cost per build $0.0250; phases: toolchain 1s, clone 2s, bun install 24s, typecheck 1m 11s, runner overhead 28s, ×32
  9. AWS CodeBuildfailed · out of memory

Bar shades, light to dark, in execution order: toolchainclonebun installtypecheckrunner overhead

cost per build× = p90 time × cost against the best

DockerPostHog image
  1. RunsOn4 vCPU · 32 GBr8a.xlarge · tmpfs5m 15s, cost per build spot $0.0087, on-demand $0.0307, best-value of 8 machines; phases: fetch source 5s, buildx setup 3s, docker build 4m 42s, runner overhead 25s, ×1.00
  2. Avrea4 vCPU · 16 GB5m 26s, cost per build $0.0435; phases: fetch source 23s, buildx setup 3s, docker build 4m 32s, runner overhead 28s, ×3.43
  3. Namespace4 vCPU · 16 GB4x165m 02s, cost per build $0.0604; phases: fetch source 20s, buildx setup 2s, docker build 4m 26s, runner overhead 14s, ×6.86
  4. Blacksmith4 vCPU · 16 GB5m 53s, cost per build $0.0471; phases: fetch source 20s, buildx setup 2s, docker build 5m 01s, runner overhead 30s, ×7.18
  5. Warpbuild4 vCPU · 16 GB7m 29s, cost per build $0.0599; phases: fetch source 20s, buildx setup 4s, docker build 6m 48s, runner overhead 17s, ×7.76
  6. Ubicloud4 vCPU · 16 GBpremium11m 01s, cost per build $0.0441, best-value of 2 machines; phases: fetch source 20s, buildx setup 9s, docker build 9m 46s, runner overhead 46s, ×12
  7. GitHub4 vCPU · 16 GB14m 37s, cost per build $0.1754; phases: fetch source 24s, buildx setup 7s, docker build 12m 22s, runner overhead 1m 44s, ×42
  8. AWS CodeBuildnot run: needs more than 8 GB of memory
  9. boatnot run: needs more than 8 GB of memory

Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead

cost per build× = p90 time × cost against the best

Each provider's best-value machine (lowest p90 time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.

Beyond builds

The same runners, one question each.

  • Queue How long jobs wait for a runner in a burst, and how far build times move from run to run.
  • Bigger runners What doubling the vCPU did to each build's time and cost, on each provider's own runner line.
  • Head-to-heads Two runners of the same size, build for build, cost first.
  • CPU Fastest single-thread on x64: Namespace 8x32, 4,556 PassMark.
  • Cache 4 GB restore on x64, any size: RunsOn 6s (12 vCPU), then Blacksmith 9s (2 vCPU); GitHub's own service 31s (8 vCPU).4 GB restore on x64 at 2 vCPU: Blacksmith 9s, then Avrea 13s; GitHub's own service 1m 01s.4 GB restore on x64 at 4 vCPU: Avrea 10s, then Blacksmith 12s; GitHub's own service 44s.4 GB restore on x64 at 8 vCPU: RunsOn 11s, then Avrea 11s; GitHub's own service 31s.
  • Disk Small random reads (QD1) on x64: Namespace 8x16 leads with 107,149 per second, 60× the slowest disk (AWS CodeBuild, 1,794). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • All runners All 94 runners in one table, with the price / performance chart.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
  • Queue How long jobs wait for a runner in a burst, and how far build times move from run to run.
  • Bigger runners What doubling the vCPU did to each build's time and cost, on each provider's own runner line.
  • Head-to-heads Two runners of the same size, build for build, cost first.
  • CPU Fastest single-thread on arm64: Avrea 4 vCPU, 4,823 PassMark.
  • Cache 4 GB restore on arm64, any size: Avrea 11s (8 vCPU), then RunsOn 11s (8 vCPU); GitHub's own service 36s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then RunsOn 19s; GitHub's own service 53s.4 GB restore on arm64 at 4 vCPU: Avrea 11s, then RunsOn 16s; GitHub's own service 36s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 11s.
  • Disk Small random reads (QD1) on arm64: Blacksmith 4 vCPU leads with 32,691 per second, 4.0× the slowest disk (GitHub, 8,119). That is a disk's limit, and a build often never reaches it.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
  • All runners All 94 runners in one table, with the price / performance chart.
  • tmpfs The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.

Providers

Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.

Cost per build is variable cost only; licenses and plan fees are flat, on each provider's page. Estimate your bill