quickstart →

GitHub Actions runner benchmarks

9 providers, 94 runners, the same jobs on each: price, real builds, CPU, network, disk and cache. AWS CodeBuild and RunsOn run in your own AWS account; the others run your jobs on theirs. Run by RunsOn from an open harness, last on 15 Nov 2026. Where another runner is faster, it says so.

Compare
at
applies to every panel on this page

Scorecard

Scorecardevery provider, every categoryx64 · 2 vCPU Share Save image

RunsOn tmpfs leads on build cost, network and cache; Namespace on CPU and disk; Avrea on build time; boat on price.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00082 $0.00522 5m 19s3 4,2383 225 MiB/s2 12,4195 17s4
RunsOntmpfs your AWS account $0.00093 $0.00491 5m 04s2 3,7896 265 MiB/s1 workspace in RAM, no disk 7s1
Avrea vendor's cloud $0.00405 $0.01534 3m 50s1 4,3202 98 MiB/s7 53,0183 13s3
Blacksmith vendor's cloud $0.00406 $0.02525 6m 19s4 3,8545 99 MiB/s6 73,7792 10s2
Namespace vendor's cloud $0.006010 TypeScript: failed · out of memoryTypeScript: failed · out of memory 4,5471 151 MiB/s4 104,6751 22s5
Warpbuild vendor's cloud $0.00407 $0.02726 6m 48s5 3,9824 105 MiB/s5 51,5934 28s6
Ubicloud vendor's cloud $0.00134 $0.01263 7m 59s6 2,7128 46 MiB/s9 6,8287 57s8
boat vendor's cloud $0.00031 $0.0034 11m 20s 3,6107 42 MiB/s10 10,6636 1m 21s
AWS CodeBuild your AWS account $0.00508 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,5839 160 MiB/s3 1,7629 52s7
GitHub GitHub's cloud $0.00609 $0.06397 10m 39s7 2,29610 66 MiB/s8 6,3958 1m 01s9
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 4 vCPU Share Save image

RunsOn tmpfs leads on build cost and cache; Namespace on build time, CPU and disk; AWS CodeBuild on network; boat on price.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
Avrea vendor's cloud $0.00805 $0.06314 7m 53s2 4,3672 163 MiB/s4 49,9553 11s2
RunsOn your AWS account $0.00122 $0.04982 10m 56s5 4,2543 171 MiB/s3 11,1925 16s4
RunsOntmpfs your AWS account $0.00163 $0.01451 8m 22s3 3,8057 173 MiB/s2 workspace in RAM, no disk 5s1
Namespace vendor's cloud $0.012010 $0.08166 7m 41s1 4,4241 158 MiB/s5 105,7441 18s5
Blacksmith vendor's cloud $0.00806 $0.07695 9m 37s4 3,8376 76 MiB/s8 71,7282 11s3
Warpbuild vendor's cloud $0.00807 $0.08977 11m 13s6 4,0694 100 MiB/s6 47,4234 21s6
Ubicloud vendor's cloud $0.00254 $0.05533 15m 31s7 2,8458 48 MiB/s9 7,7757 55s9
boat vendor's cloud $0.00061 Docker: not run: needs more than 8 GB of memoryDocker: not run: needs more than 8 GB of memory 3,9325 41 MiB/s10 8,8146 2m 15s
AWS CodeBuild your AWS account $0.01008 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,6029 197 MiB/s1 1,7969 36s7
GitHub GitHub's cloud $0.01209 $0.24608 20m 30s8 2,36210 95 MiB/s7 7,2868 41s8
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 8 vCPU Share Save image

RunsOn tmpfs leads on network and cache; Namespace on build time, CPU and disk; boat on price and build cost.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
Avrea vendor's cloud $0.01605 $0.09446 5m 54s2 4,4392 142 MiB/s4 52,3833 11s3
RunsOntmpfs your AWS account $0.00273 $0.01982 6m 13s3 3,8005 184 MiB/s1 workspace in RAM, no disk 5s1
Namespace vendor's cloud $0.024010 $0.07664 5m 46s1 4,5591 137 MiB/s5 106,1911 17s5
RunsOn your AWS account $0.00252 $0.03563 7m 17s5 3,7976 160 MiB/s3 11,5845 11s4
Blacksmith vendor's cloud $0.01606 $0.10137 6m 20s4 3,8014 105 MiB/s8 70,7032 10s2
Warpbuild vendor's cloud $0.01607 $0.12518 7m 49s6 4,1273 107 MiB/s7 38,6844 25s6
boat vendor's cloud $0.00121 $0.01551 12m 57s8 3,4218 73 MiB/s9 8,1817 1m 11s10
Ubicloud vendor's cloud $0.00504 $0.08585 10m 44s7 3,5247 42 MiB/s10 8,6386 55s9
AWS CodeBuild your AWS account $0.02008 $0.342710 17m 08s10 2,5979 166 MiB/s2 1,7589 35s8
GitHub GitHub's cloud $0.02209 $0.30149 13m 42s9 2,37410 115 MiB/s6 7,5298 33s7
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 2 vCPU Share Save image

RunsOn leads on price and build cost; RunsOn tmpfs on network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00071 $0.00451 5m 05s3 2,4743 201 MiB/s2 13,5175 19s3
RunsOntmpfs your AWS account $0.00092 $0.00562 4m 55s2 2,4674 269 MiB/s1 workspace in RAM, no disk 8s1
Avrea vendor's cloud $0.01208 $0.04105 3m 25s1 4,6621 139 MiB/s5 20,9052 12s2
Blacksmith vendor's cloud $0.00254 $0.03034 12m 08s6 1,3287 88 MiB/s8 29,9681 24s4
Warpbuild vendor's cloud $0.00305 $0.02613 8m 42s4 1,9395 167 MiB/s4 1,7938 29s6
NamespaceApple silicon vendor's cloud $0.04209 TypeScript: failed · out of memoryTypeScript: failed · out of memory 4,1552 168 MiB/s3 19,8933 25s5
GitHub GitHub's cloud $0.00506 $0.04526 9m 03s5 1,8696 93 MiB/s7 7,9947 53s8
Namespace vendor's cloud $0.00607 TypeScript: failed · out of memoryTypeScript: failed · out of memory 1,1419 121 MiB/s6 15,9104 42s7
Ubicloud vendor's cloud $0.00133 $0.0193 15m 28s 1,2978 63 MiB/s9 8,1496 1m 02s9
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple silicon vendor's cloud no 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runner

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 4 vCPU Share Save image

RunsOn leads on price, build cost and network; RunsOn tmpfs on cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00141 $0.01481 9m 52s4 2,4713 172 MiB/s1 13,3755 16s3
RunsOntmpfs your AWS account $0.00162 $0.01952 8m 49s3 2,4674 170 MiB/s2 workspace in RAM, no disk 5s1
Avrea vendor's cloud $0.02408 $0.17727 7m 23s1 4,8231 83 MiB/s8 19,9512 11s2
Blacksmith vendor's cloud $0.00504 $0.08724 17m 27s7 1,3417 100 MiB/s7 32,7401 21s4
NamespaceApple silicon vendor's cloud $0.08409 $0.63919 7m 37s2 4,5222 131 MiB/s4 18,7363 22s5
Warpbuild vendor's cloud $0.00605 $0.09485 15m 48s5 1,9405 144 MiB/s3 1,8068 27s6
GitHub GitHub's cloud $0.00806 $0.13206 16m 30s6 1,8706 109 MiB/s6 6,8927 36s7
Ubicloud vendor's cloud $0.00253 $0.06173 24m 42s9 1,2868 75 MiB/s9 8,7056 57s9
Namespace vendor's cloud $0.01207 $0.24748 23m 33s8 1,1659 121 MiB/s5 16,6344 38s8
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple · 6 vCPU vendor's cloud $0.0800 $0.7647 9m 34s 2,845 70 MiB/s 29,877 29s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 8 vCPU Share Save image

RunsOn leads on price; RunsOn tmpfs on build cost, network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOntmpfs your AWS account $0.00242 $0.01741 6m 42s2 2,4703 298 MiB/s1 workspace in RAM, no disk 5s1
RunsOn your AWS account $0.00241 $0.02382 7m 02s3 2,4722 161 MiB/s3 13,4544 11s3
Avrea vendor's cloud $0.04807 $0.27527 5m 44s1 4,8151 69 MiB/s6 20,7432 10s2
Blacksmith vendor's cloud $0.01004 $0.11994 12m 00s5 1,3445 97 MiB/s5 29,3121 18s4
Warpbuild vendor's cloud $0.01205 $0.12285 10m 14s4 1,9404 189 MiB/s2 1,7976 21s5
Namespace vendor's cloud $0.02406 $0.20566 17m 08s6 1,1657 125 MiB/s4 16,7833 32s6
Ubicloud vendor's cloud $0.00503 $0.09053 18m 07s7 1,3006 57 MiB/s7 9,3885 49s7
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
GitHub GitHub's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
NamespaceApple silicon vendor's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
WarpbuildApple · 12 vCPU vendor's cloud $0.1600 $1.27 7m 56s 3,138 74 MiB/s 30,249 46s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Real builds

Real buildstime and cost per buildlower is betterx64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

boat is the best value and the cheapest; Avrea is the fastest.

  1. boat 4 GB 4m 05s $0.0012
  2. RunsOn m8a.large · EBS gp3 · 8 GB 3m 06s $0.0031
  3. RunsOntmpfs r8a.large · 16 GB 3m 21s $0.0033
  4. Avrea 8 GB 2m 24s $0.0096
  5. Ubicloud standard · 8 GB 4m 31s $0.0056
  6. Namespace 2x8 · 8 GB 2m 43s $0.0163
  7. Warpbuild 8 GB 4m 00s $0.0160
  8. Blacksmith 8 GB 4m 06s $0.0164
  9. AWS CodeBuild small · 3 GB 6m 51s $0.0342
  10. GitHub 8 GB 5m 50s $0.0350
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value; RunsOn tmpfs is the cheapest; Avrea is the fastest.

  1. RunsOn m8a.large · EBS gp3 · 8 GB 2m 13s $0.0021
  2. Avrea 8 GB 1m 26s $0.0057
  3. RunsOntmpfs r8a.large · 16 GB 1m 43s $0.0017
  4. Ubicloud premium · 8 GB 3m 28s $0.0069
  5. Blacksmith 8 GB 2m 13s $0.0088
  6. Warpbuild 8 GB 2m 48s $0.0112
  7. GitHub 8 GB 4m 49s $0.0289
  8. boat 4 GB 7m 15s $0.0022
  9. AWS CodeBuild failed · out of memory
  10. Namespace failed · out of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

Only RunsOn finished this build.

  1. RunsOn r8a.large · EBS gp3 · 16 GB 12m 34s $0.0134
  2. Avrea not run: needs more than 8 GB of memory
  3. AWS CodeBuild not run: needs more than 8 GB of memory
  4. Blacksmith not run: needs more than 8 GB of memory
  5. boat not run: needs more than 8 GB of memory
  6. GitHub not run: needs more than 8 GB of memory
  7. Namespace not run: needs more than 8 GB of memory
  8. NamespaceApple silicon arm64 only
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Ubicloud not run: needs more than 8 GB of memory
  11. Warpbuild not run: needs more than 8 GB of memory
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOn c8a.xlarge · EBS gp3 · 8 GB 2m 02s $0.0036
  2. RunsOntmpfs r8a.xlarge · 32 GB 2m 04s $0.0036
  3. Avrea 16 GB 1m 37s $0.0129
  4. Namespace 4x8 · 8 GB 1m 46s $0.0106
  5. Ubicloud standard · 16 GB 2m 38s $0.0066
  6. boat 8 GB 4m 48s $0.0029
  7. Warpbuild 16 GB 2m 23s $0.0191
  8. Blacksmith 16 GB 1m 59s $0.0159
  9. AWS CodeBuild medium · 8 GB 3m 48s $0.0380
  10. GitHub 16 GB 3m 48s $0.0456
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

boat is the best value and the cheapest; Avrea is the fastest.

  1. boat 8 GB 1m 36s $0.0010
  2. RunsOntmpfs r8a.xlarge · 32 GB 1m 03s $0.0021
  3. Avrea 16 GB 50s $0.0067
  4. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 1m 01s $0.0061
  5. Ubicloud standard · 16 GB 1m 52s $0.0047
  6. Namespace 4x16 · 16 GB 53s $0.0106
  7. Warpbuild 16 GB 1m 21s $0.0108
  8. Blacksmith 16 GB 1m 45s $0.0139
  9. GitHub 16 GB 2m 05s $0.0250
  10. AWS CodeBuild failed · out of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value and the cheapest; Namespace is the fastest.

  1. RunsOntmpfs r8a.xlarge · 32 GB 5m 15s $0.0087
  2. Avrea 16 GB 5m 26s $0.0435
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 7m 53s $0.0402
  4. Namespace 4x16 · 16 GB 5m 02s $0.0604
  5. Blacksmith 16 GB 5m 53s $0.0471
  6. Warpbuild 16 GB 7m 29s $0.0599
  7. Ubicloud premium · 16 GB 11m 01s $0.0441
  8. GitHub 16 GB 14m 37s $0.1754
  9. AWS CodeBuild not run: needs more than 8 GB of memory
  10. boat not run: needs more than 8 GB of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn tmpfs is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 1m 28s $0.0047
  2. RunsOn m8a.2xlarge · EBS gp3 · 32 GB 1m 28s $0.0046
  3. Avrea 32 GB 1m 13s $0.0195
  4. Namespace 8x16 · 16 GB 1m 16s $0.0152
  5. Ubicloud premium · 32 GB 1m 54s $0.0152
  6. Warpbuild 32 GB 1m 37s $0.0259
  7. boat 16 GB 2m 06s $0.0025
  8. Blacksmith 32 GB 1m 21s $0.0216
  9. AWS CodeBuild large · 16 GB 2m 28s $0.0493
  10. GitHub 32 GB 2m 19s $0.0510
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 50s $0.0028
  2. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 49s $0.0048
  3. boat 16 GB 1m 28s $0.0017
  4. Avrea 32 GB 37s $0.0099
  5. Blacksmith 32 GB 52s $0.0139
  6. Warpbuild 32 GB 55s $0.0147
  7. Ubicloud premium · 32 GB 1m 12s $0.0095
  8. Namespace 8x32 · 32 GB 37s $0.0148
  9. AWS CodeBuild large · 16 GB 1m 41s $0.0337
  10. GitHub 32 GB 1m 30s $0.0330
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value; boat is the cheapest; Namespace is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 3m 55s $0.0123
  2. Namespace 8x16 · 16 GB 3m 53s $0.0466
  3. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 5m 00s $0.0262
  4. Avrea 32 GB 4m 04s $0.0651
  5. boat 16 GB 9m 24s $0.0113
  6. Blacksmith 32 GB 4m 07s $0.0659
  7. Ubicloud premium · 32 GB 7m 38s $0.0611
  8. Warpbuild 32 GB 5m 17s $0.0845
  9. GitHub 32 GB 9m 53s $0.2174
  10. AWS CodeBuild large · 16 GB 12m 59s $0.2597
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.large · local NVMe · 8 GB 3m 01s $0.0026
  2. RunsOntmpfs r9g.large · 16 GB 3m 13s $0.0037
  3. Avrea 8 GB 2m 04s $0.0248
  4. Warpbuild 8 GB 5m 01s $0.0151
  5. Ubicloud standard · 8 GB 10m 27s $0.0131
  6. Blacksmith 6 GB 6m 48s $0.0170
  7. GitHub 8 GB 5m 41s $0.0284
  8. NamespaceApple silicon 2x8 · 8 GB 2m 11s $0.0917
  9. Namespace 2x8 · 8 GB 9m 44s $0.0584
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn is the cheapest; RunsOn tmpfs is the best value; Avrea is the fastest.

  1. RunsOntmpfs r9g.large · 16 GB 1m 42s $0.0019
  2. RunsOn m9gd.large · local NVMe · 8 GB 2m 04s $0.0019
  3. Avrea 8 GB 1m 21s $0.0162
  4. Warpbuild 8 GB 3m 41s $0.0110
  5. Blacksmith 6 GB 5m 20s $0.0133
  6. GitHub 8 GB 3m 22s $0.0168
  7. Ubicloud standard · 8 GB 5m 01s $0.0063
  8. AWS CodeBuild no arm64 runner
  9. boat no arm64 runner
  10. Namespace failed · out of memory
  11. NamespaceApple silicon failed · out of memory
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Docker PostHog's production image, built cold with buildx

No 2 vCPU arm64 runner finished this build.

  1. Avrea not run: needs more than 8 GB of memory
  2. AWS CodeBuild no arm64 runner
  3. Blacksmith not run: needs more than 8 GB of memory
  4. boat no arm64 runner
  5. GitHub not run: needs more than 8 GB of memory
  6. Namespace not run: needs more than 8 GB of memory
  7. NamespaceApple silicon not run: needs more than 8 GB of memory
  8. RunsOn not run: needs more than 8 GB of memory
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Ubicloud not run: needs more than 8 GB of memory
  11. Warpbuild not run: needs more than 8 GB of memory
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9g.xlarge · EBS gp3 · 16 GB 2m 01s $0.0035
  2. RunsOntmpfs m9g.xlarge · 16 GB 2m 05s $0.0035
  3. Avrea 16 GB 1m 27s $0.0348
  4. Warpbuild 16 GB 3m 14s $0.0194
  5. GitHub 16 GB 3m 18s $0.0264
  6. Ubicloud standard · 16 GB 6m 07s $0.0153
  7. Blacksmith 16 GB 4m 13s $0.0211
  8. NamespaceApple silicon 4x16 · 16 GB 1m 25s $0.1190
  9. Namespace 4x8 · 8 GB 5m 51s $0.0351
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 1m 28s $0.1173
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB 1m 03s $0.0017
  2. RunsOntmpfs m9g.xlarge · 16 GB 1m 04s $0.0021
  3. Warpbuild 16 GB 1m 37s $0.0097
  4. GitHub 16 GB 1m 44s $0.0139
  5. Blacksmith 16 GB 2m 14s $0.0111
  6. Avrea 16 GB 49s $0.0196
  7. Namespace 4x16 · 16 GB 3m 24s $0.0407
  8. NamespaceApple silicon 4x16 · 16 GB 49s $0.0679
  9. Ubicloud standard · 16 GB 2m 44s $0.0068
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 58s $0.0767
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn is the best value and the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB 6m 48s $0.0097
  2. RunsOntmpfs r9g.xlarge · 32 GB 5m 40s $0.0139
  3. Ubicloud standard · 16 GB 15m 51s $0.0396
  4. Avrea 16 GB 5m 07s $0.1228
  5. Blacksmith 16 GB 11m 00s $0.0550
  6. Warpbuild 16 GB 10m 57s $0.0657
  7. GitHub 16 GB 11m 28s $0.0917
  8. Namespace 4x16 · 16 GB 14m 18s $0.1716
  9. NamespaceApple silicon 4x16 · 16 GB 5m 23s $0.4522
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 7m 08s $0.5707
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the cheapest; RunsOn tmpfs is the best value; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 1m 27s $0.0040
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 1m 27s $0.0040
  3. Avrea 32 GB 1m 07s $0.0536
  4. Warpbuild 32 GB 2m 13s $0.0266
  5. Ubicloud standard · 32 GB 4m 04s $0.0203
  6. Blacksmith 32 GB 2m 55s $0.0292
  7. Namespace 8x16 · 16 GB 4m 08s $0.0496
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 1m 10s $0.1867
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 46s $0.0024
  2. RunsOn m9gd.2xlarge · local NVMe · 32 GB 44s $0.0031
  3. Warpbuild 32 GB 1m 03s $0.0126
  4. Ubicloud standard · 32 GB 1m 51s $0.0092
  5. Blacksmith 32 GB 1m 26s $0.0143
  6. Namespace 8x16 · 16 GB 2m 03s $0.0246
  7. Avrea 32 GB 40s $0.0320
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 47s $0.1240
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn is the best value; RunsOn tmpfs is the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.2xlarge · local NVMe · 32 GB 4m 51s $0.0167
  2. RunsOntmpfs m9g.2xlarge · 32 GB 4m 29s $0.0110
  3. Warpbuild 32 GB 6m 58s $0.0836
  4. Blacksmith 32 GB 7m 39s $0.0765
  5. Ubicloud standard · 32 GB 12m 12s $0.0610
  6. Avrea 32 GB 3m 57s $0.1896
  7. Namespace 8x16 · 16 GB 10m 57s $0.1314
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 5m 59s $0.9573
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

CPU

CPUsingle-thread PassMarkhigher is betterx64 · 2 vCPU Share Save image

Namespace has the fastest core.

RunsOn is 3rd of 10, 7% below the leader. Namespace to GitHub: 2.0× apart.

  1. Namespace 2x8 · 8 GB 4,547
  2. Avrea 8 GB 4,320
  3. RunsOn m8azn.large · EBS gp3 · 8 GB 4,238
  4. Warpbuild 8 GB 3,982
  5. Blacksmith 8 GB 3,854
  6. RunsOntmpfs r8a.large · 16 GB 3,789
  7. boat 4 GB 3,610
  8. Ubicloud standard · 8 GB 2,712
  9. AWS CodeBuild small · 3 GB 2,583
  10. GitHub 8 GB 2,296
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 4 vCPU Share Save image

RunsOn, Namespace and Avrea tie for the fastest core.

Namespace to GitHub: 1.9× apart.

  1. Namespace 4x16 · 16 GB 4,424
  2. Avrea 16 GB 4,367
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 4,254
  4. Warpbuild 16 GB 4,069
  5. boat 8 GB 3,932
  6. Blacksmith 16 GB 3,837
  7. RunsOntmpfs r8a.xlarge · 32 GB 3,805
  8. Ubicloud premium · 16 GB 2,845
  9. AWS CodeBuild medium · 8 GB 2,602
  10. GitHub 16 GB 2,362
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 8 vCPU Share Save image

Namespace and Avrea tie for the fastest core.

RunsOn is 6th of 10, 17% below the leader. Namespace to GitHub: 1.9× apart.

  1. Namespace 8x32 · 32 GB 4,559
  2. Avrea 32 GB 4,439
  3. Warpbuild 32 GB 4,127
  4. Blacksmith 32 GB 3,801
  5. RunsOntmpfs r8a.2xlarge · 64 GB 3,800
  6. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 3,797
  7. Ubicloud premium · 32 GB 3,524
  8. boat 16 GB 3,421
  9. AWS CodeBuild large · 16 GB 2,597
  10. GitHub 32 GB 2,374
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 2 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 3rd of 9, 47% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 8 GB 4,662
  2. NamespaceApple silicon 2x8 · 8 GB 4,155
  3. RunsOn m9gd.large · local NVMe · 8 GB 2,474
  4. RunsOntmpfs r9g.large · 16 GB 2,467
  5. Warpbuild 8 GB 1,939
  6. GitHub 8 GB 1,869
  7. Blacksmith 6 GB 1,328
  8. Ubicloud standard · 8 GB 1,297
  9. Namespace 2x8 · 8 GB 1,141
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 4 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 3rd of 9, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 16 GB 4,823
  2. NamespaceApple silicon 4x16 · 16 GB 4,522
  3. RunsOn m9gd.xlarge · local NVMe · 16 GB 2,471
  4. RunsOntmpfs r9g.xlarge · 32 GB 2,467
  5. Warpbuild 16 GB 1,940
  6. GitHub 16 GB 1,870
  7. Blacksmith 16 GB 1,341
  8. Ubicloud standard · 16 GB 1,286
  9. Namespace 4x16 · 16 GB 1,165
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 2,845 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 8 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 2nd of 7, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 32 GB 4,815
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 2,472
  3. RunsOntmpfs m9g.2xlarge · 32 GB 2,470
  4. Warpbuild 32 GB 1,940
  5. Blacksmith 32 GB 1,344
  6. Ubicloud standard · 32 GB 1,300
  7. Namespace 8x16 · 16 GB 1,165
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 3,138 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

Price

Priceper minutelower is betterx64 · 2 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.8× lower. boat to Namespace: 20× apart.

  1. boat 4 GB $0.0003
  2. RunsOn m8id.large · local NVMe · 8 GB $0.0008 $0.0025 on-demand
  3. RunsOntmpfs r8a.large · 16 GB $0.0009 $0.0030 on-demand
  4. Ubicloud standard · 8 GB $0.0013
  5. Avrea 8 GB $0.0040
  6. Blacksmith 8 GB $0.0040
  7. Warpbuild 8 GB $0.0040
  8. AWS CodeBuild small · 3 GB $0.0050
  9. GitHub 8 GB $0.0060
  10. Namespace 2x8 · 8 GB $0.0060
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 4 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.1× lower. boat to Namespace: 20× apart.

  1. boat 8 GB $0.0006
  2. RunsOn m8i-flex.xlarge · EBS gp3 · 16 GB $0.0012 $0.0037 on-demand
  3. RunsOntmpfs r8a.xlarge · 32 GB $0.0016 $0.0056 on-demand
  4. Ubicloud standard · 16 GB $0.0025
  5. Avrea 16 GB $0.0080
  6. Blacksmith 16 GB $0.0080
  7. Warpbuild 16 GB $0.0080
  8. AWS CodeBuild medium · 8 GB $0.0100
  9. GitHub 16 GB $0.0120
  10. Namespace 4x16 · 16 GB $0.0120
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 8 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.1× lower. boat to Namespace: 20× apart.

  1. boat 16 GB $0.0012
  2. RunsOn m8i-flex.2xlarge · EBS gp3 · 32 GB $0.0025 $0.0070 on-demand
  3. RunsOntmpfs m8a.2xlarge · 32 GB $0.0027 $0.0084 on-demand
  4. Ubicloud standard · 32 GB $0.0050
  5. Avrea 32 GB $0.0160
  6. Blacksmith 32 GB $0.0160
  7. Warpbuild 32 GB $0.0160
  8. AWS CodeBuild large · 16 GB $0.0200
  9. GitHub 32 GB $0.0220
  10. Namespace 8x32 · 32 GB $0.0240
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 2 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 57× apart.

  1. RunsOn m9g.large · EBS gp3 · 8 GB $0.0007 $0.0019 on-demand
  2. RunsOntmpfs r9g.large · 16 GB $0.0009 $0.0024 on-demand
  3. Ubicloud standard · 8 GB $0.0013
  4. Blacksmith 6 GB $0.0025
  5. Warpbuild 8 GB $0.0030
  6. GitHub 8 GB $0.0050
  7. Namespace 2x8 · 8 GB $0.0060
  8. Avrea 8 GB $0.0120
  9. NamespaceApple silicon 2x8 · 8 GB $0.0420
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 4 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 62× apart.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB $0.0014 $0.0045 on-demand
  2. RunsOntmpfs m9g.xlarge · 16 GB $0.0016 $0.0036 on-demand
  3. Ubicloud standard · 16 GB $0.0025
  4. Blacksmith 16 GB $0.0050
  5. Warpbuild 16 GB $0.0060
  6. GitHub 16 GB $0.0080
  7. Namespace 4x16 · 16 GB $0.0120
  8. Avrea 16 GB $0.0240
  9. NamespaceApple silicon 4x16 · 16 GB $0.0840
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB $0.0800 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 8 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the lowest price per minute.

RunsOn to Avrea: 20× apart.

  1. RunsOn m9g.2xlarge · EBS gp3 · 32 GB $0.0024 $0.0068 on-demand
  2. RunsOntmpfs m9g.2xlarge · 32 GB $0.0024 $0.0068 on-demand
  3. Ubicloud standard · 32 GB $0.0050
  4. Blacksmith 32 GB $0.0100
  5. Warpbuild 32 GB $0.0120
  6. Namespace 8x32 · 32 GB $0.0240
  7. Avrea 32 GB $0.0480
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB $0.1600 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Network

Networkdownload from GitHubhigher is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 2nd of 10, 15% below the leader. RunsOn tmpfs to boat: 6.3× apart.

  1. RunsOntmpfs r8a.large · 16 GB 265 MiB/s
  2. RunsOn median of 9 machines 225 MiB/s
  3. AWS CodeBuild small · 3 GB 160 MiB/s
  4. Namespace 2x8 · 8 GB 151 MiB/s
  5. Warpbuild 8 GB 105 MiB/s
  6. Blacksmith 8 GB 99 MiB/s
  7. Avrea 8 GB 98 MiB/s
  8. GitHub 8 GB 66 MiB/s
  9. Ubicloud median of 2 machines 46 MiB/s
  10. boat 4 GB 42 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterx64 · 4 vCPU Share Save image

AWS CodeBuild downloads from GitHub fastest.

RunsOn is 3rd of 10, 13% below the leader. AWS CodeBuild to boat: 4.8× apart.

  1. AWS CodeBuild medium · 8 GB 197 MiB/s
  2. RunsOntmpfs r8a.xlarge · 32 GB 173 MiB/s
  3. RunsOn median of 8 machines 171 MiB/s
  4. Avrea 16 GB 163 MiB/s
  5. Namespace median of 2 machines 158 MiB/s
  6. Warpbuild 16 GB 100 MiB/s
  7. GitHub 16 GB 95 MiB/s
  8. Blacksmith 16 GB 76 MiB/s
  9. Ubicloud median of 2 machines 48 MiB/s
  10. boat 8 GB 41 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterx64 · 8 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 3rd of 10, 13% below the leader. RunsOn tmpfs to Ubicloud: 4.4× apart.

  1. RunsOntmpfs median of 2 machines 184 MiB/s
  2. AWS CodeBuild large · 16 GB 166 MiB/s
  3. RunsOn median of 7 machines 160 MiB/s
  4. Avrea 32 GB 142 MiB/s
  5. Namespace median of 2 machines 137 MiB/s
  6. GitHub 32 GB 115 MiB/s
  7. Warpbuild 32 GB 107 MiB/s
  8. Blacksmith 32 GB 105 MiB/s
  9. boat 16 GB 73 MiB/s
  10. Ubicloud median of 2 machines 42 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 2nd of 9, 25% below the leader. RunsOn tmpfs to Ubicloud: 4.3× apart.

  1. RunsOntmpfs r9g.large · 16 GB 269 MiB/s
  2. RunsOn median of 3 machines 201 MiB/s
  3. NamespaceApple silicon 2x8 · 8 GB 168 MiB/s
  4. Warpbuild 8 GB 167 MiB/s
  5. Avrea 8 GB 139 MiB/s
  6. Namespace 2x8 · 8 GB 121 MiB/s
  7. GitHub 8 GB 93 MiB/s
  8. Blacksmith 6 GB 88 MiB/s
  9. Ubicloud standard · 8 GB 63 MiB/s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 4 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the fastest download from GitHub.

RunsOn to Ubicloud: 2.3× apart.

  1. RunsOn median of 3 machines 172 MiB/s
  2. RunsOntmpfs median of 2 machines 170 MiB/s
  3. Warpbuild 16 GB 144 MiB/s
  4. NamespaceApple silicon 4x16 · 16 GB 131 MiB/s
  5. Namespace median of 2 machines 121 MiB/s
  6. GitHub 16 GB 109 MiB/s
  7. Blacksmith 16 GB 100 MiB/s
  8. Avrea 16 GB 83 MiB/s
  9. Ubicloud standard · 16 GB 75 MiB/s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 70 MiB/s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 3rd of 7, 46% below the leader. RunsOn tmpfs to Ubicloud: 5.2× apart.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 298 MiB/s
  2. Warpbuild 32 GB 189 MiB/s
  3. RunsOn median of 3 machines 161 MiB/s
  4. Namespace median of 2 machines 125 MiB/s
  5. Blacksmith 32 GB 97 MiB/s
  6. Avrea 32 GB 69 MiB/s
  7. Ubicloud standard · 32 GB 57 MiB/s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 74 MiB/s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Disk

Diskrandom reads per secondhigher is betterx64 · 2 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 8.4× higher. Namespace to AWS CodeBuild: 59× apart.

  1. Namespace 2x8 · 8 GB 104,675
  2. Blacksmith 8 GB 73,779
  3. Avrea 8 GB 53,018
  4. Warpbuild 8 GB 51,593
  5. RunsOn i7i.large · local NVMe · 16 GB 12,419
  6. boat 4 GB 10,663
  7. Ubicloud standard · 8 GB 6,828
  8. GitHub 8 GB 6,395
  9. AWS CodeBuild small · 3 GB 1,762
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 4 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 9.4× higher. Namespace to AWS CodeBuild: 59× apart.

  1. Namespace 4x8 · 8 GB 105,744
  2. Blacksmith 16 GB 71,728
  3. Avrea 16 GB 49,955
  4. Warpbuild 16 GB 47,423
  5. RunsOn i7i.xlarge · local NVMe · 32 GB 11,192
  6. boat 8 GB 8,814
  7. Ubicloud standard · 16 GB 7,775
  8. GitHub 16 GB 7,286
  9. AWS CodeBuild medium · 8 GB 1,796
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 8 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 9.2× higher. Namespace to AWS CodeBuild: 60× apart.

  1. Namespace 8x16 · 16 GB 106,191
  2. Blacksmith 32 GB 70,703
  3. Avrea 32 GB 52,383
  4. Warpbuild 32 GB 38,684
  5. RunsOn i7i.2xlarge · local NVMe · 64 GB 11,584
  6. Ubicloud premium · 32 GB 8,638
  7. boat 16 GB 8,181
  8. GitHub 32 GB 7,529
  9. AWS CodeBuild large · 16 GB 1,758
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 2 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 5th of 8, the leader is 2.2× higher. Blacksmith to Warpbuild: 17× apart.

  1. Blacksmith 6 GB 29,968
  2. Avrea 8 GB 20,905
  3. NamespaceApple silicon 2x8 · 8 GB 19,893
  4. Namespace 2x8 · 8 GB 15,910
  5. RunsOn m9gd.large · local NVMe · 8 GB 13,517
  6. Ubicloud standard · 8 GB 8,149
  7. GitHub 8 GB 7,994
  8. Warpbuild 8 GB 1,793
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 4 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 5th of 8, the leader is 2.4× higher. Blacksmith to Warpbuild: 18× apart.

  1. Blacksmith 16 GB 32,740
  2. Avrea 16 GB 19,951
  3. NamespaceApple silicon 4x16 · 16 GB 18,736
  4. Namespace 4x8 · 8 GB 16,634
  5. RunsOn m9gd.xlarge · local NVMe · 16 GB 13,375
  6. Ubicloud standard · 16 GB 8,705
  7. GitHub 16 GB 6,892
  8. Warpbuild 16 GB 1,806
  9. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 29,877 not ranked
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 8 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 4th of 6, the leader is 2.2× higher. Blacksmith to Warpbuild: 16× apart.

  1. Blacksmith 32 GB 29,312
  2. Avrea 32 GB 20,743
  3. Namespace 8x32 · 32 GB 16,783
  4. RunsOn m9gd.2xlarge · local NVMe · 32 GB 13,454
  5. Ubicloud standard · 32 GB 9,388
  6. Warpbuild 32 GB 1,797
  7. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 30,249 not ranked
  8. AWS CodeBuild no arm64 runner
  9. boat no arm64 runner
  10. GitHub no 8 vCPU arm64 runner
  11. NamespaceApple silicon no 8 vCPU arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Cache

Cacherestoring 4 GBlower is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 4th of 9, the leader is 2.4× lower. RunsOn tmpfs to GitHub: 8.6× apart.

  1. RunsOntmpfs r8a.large · actions/cache 7s
  2. Blacksmith actions/cache 10s
  3. Avrea actions/cache 13s
  4. RunsOn r8a.large · EBS gp3 · actions/cache 17s
  5. Namespace 2x8 · actions/cache 22s
  6. Warpbuild actions/cache 28s
  7. AWS CodeBuild small · actions/cache 52s
  8. Ubicloud standard · actions/cache 57s
  9. GitHub 2-core · actions/cache 1m 01s
  10. boat actions/cache 1m 21s provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 4th of 9, the leader is 3.2× lower. RunsOn tmpfs to Ubicloud: 11× apart.

  1. RunsOntmpfs r8a.xlarge · actions/cache 5s
  2. Avrea actions/cache 11s
  3. Blacksmith actions/cache 11s
  4. RunsOn m8a.xlarge · EBS gp3 · actions/cache 16s
  5. Namespace 4x8 · actions/cache 18s
  6. Warpbuild actions/cache 21s
  7. AWS CodeBuild medium · actions/cache 36s
  8. GitHub 4-core · actions/cache 41s
  9. Ubicloud premium · actions/cache 55s
  10. boat actions/cache 2m 15s provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 4th of 10, the leader is 2.2× lower. RunsOn tmpfs to boat: 14× apart.

  1. RunsOntmpfs m8a.2xlarge · actions/cache 5s
  2. Blacksmith actions/cache 10s
  3. Avrea actions/cache 11s
  4. RunsOn m8a.2xlarge · EBS gp3 · actions/cache 11s
  5. Namespace 8x16 · actions/cache 17s
  6. Warpbuild actions/cache 25s
  7. GitHub 8-core · actions/cache 33s
  8. AWS CodeBuild large · actions/cache 35s
  9. Ubicloud standard · actions/cache 55s
  10. boat actions/cache 1m 11s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 9, the leader is 2.4× lower. RunsOn tmpfs to Ubicloud: 7.8× apart.

  1. RunsOntmpfs r9g.large · actions/cache 8s
  2. Avrea actions/cache 12s
  3. RunsOn m9g.large · EBS gp3 · actions/cache 19s
  4. Blacksmith actions/cache 24s
  5. NamespaceApple silicon 2x8 · actions/cache 25s
  6. Warpbuild actions/cache 29s
  7. Namespace 2x8 · actions/cache 42s
  8. GitHub 2-core · actions/cache 53s
  9. Ubicloud standard · actions/cache 1m 02s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 9, the leader is 3.2× lower. RunsOn tmpfs to Ubicloud: 11× apart.

  1. RunsOntmpfs r9g.xlarge · actions/cache 5s
  2. Avrea actions/cache 11s
  3. RunsOn m9g.xlarge · EBS gp3 · actions/cache 16s
  4. Blacksmith actions/cache 21s
  5. NamespaceApple silicon 4x16 · actions/cache 22s
  6. Warpbuild actions/cache 27s
  7. GitHub 4-core · actions/cache 36s
  8. Namespace 4x8 · actions/cache 38s
  9. Ubicloud standard · actions/cache 57s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · actions/cache 29s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 7, the leader is 2.2× lower. RunsOn tmpfs to Ubicloud: 9.8× apart.

  1. RunsOntmpfs m9g.2xlarge · actions/cache 5s
  2. Avrea actions/cache 10s
  3. RunsOn m9g.2xlarge · EBS gp3 · actions/cache 11s
  4. Blacksmith actions/cache 18s
  5. Warpbuild actions/cache 21s
  6. Namespace 8x16 · actions/cache 32s
  7. Ubicloud standard · actions/cache 49s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · actions/cache 46s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Predictability

Predictabilityrun to runlower is betterx64 · 2 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 3rd of 7, at +10%.

  1. Avrea 8 GB · worst run 1.1× +4%
  2. RunsOntmpfs r8a.large · 16 GB · worst run 1.1× +6%
  3. RunsOn 2 machines · worst run 1.2× +10%
  4. Ubicloud 2 machines · worst run 1.5× +23%
  5. Warpbuild 8 GB · worst run 1.5× +25%
  6. GitHub 8 GB · worst run 1.3× +28%
  7. Blacksmith 8 GB · worst run 1.5× +33%
  8. AWS CodeBuild small · 3 GB · worst run 1.1× +8% provisional
  9. boat 4 GB · worst run 1.2× +17% provisional
  10. Namespace 2x8 · 8 GB · worst run 1.3× +20% provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 4 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 3rd of 9, at +7%.

  1. Avrea 16 GB · worst run 1.1× +3%
  2. RunsOntmpfs r8a.xlarge · 32 GB · worst run 1.2× +4%
  3. RunsOn 2 machines · worst run 1.4× +7%
  4. Warpbuild 16 GB · worst run 1.2× +13%
  5. GitHub 16 GB · worst run 1.2× +15%
  6. Namespace 2 machines · worst run 1.7× +24%
  7. Ubicloud 2 machines · worst run 1.7× +30%
  8. Blacksmith 16 GB · worst run 2.8× +58%
  9. boat 8 GB · worst run 2.0× +58%
  10. AWS CodeBuild medium · 8 GB · worst run 1.1× +4% provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 8 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 4th of 10, at +9%.

  1. Avrea 32 GB · worst run 1.1× +4%
  2. AWS CodeBuild large · 16 GB · worst run 1.1× +5%
  3. RunsOntmpfs m8a.2xlarge · 32 GB · worst run 1.1× +7%
  4. RunsOn 2 machines · worst run 1.5× +9%
  5. Warpbuild 32 GB · worst run 1.3× +12%
  6. Ubicloud premium · 32 GB · worst run 1.4× +15%
  7. GitHub 32 GB · worst run 1.3× +16%
  8. Namespace 2 machines · worst run 1.6× +22%
  9. Blacksmith 32 GB · worst run 1.9× +41%
  10. boat 16 GB · worst run 4.7× +147%
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 2 vCPU Share Save image

RunsOn has the steadiest build times.

  1. RunsOn m9gd.large · local NVMe · 8 GB · worst run 1.1× +5%
  2. Blacksmith 6 GB · worst run 1.2× +8%
  3. Avrea 8 GB · worst run 1.2× +12%
  4. GitHub 8 GB · worst run 1.3× +15%
  5. RunsOntmpfs r9g.large · 16 GB · worst run 1.7× +26%
  6. Warpbuild 8 GB · worst run 1.5× +33%
  7. Ubicloud standard · 8 GB · worst run 1.0× +3% provisional
  8. Namespace 2x8 · 8 GB · worst run 1.1× +4% provisional
  9. NamespaceApple silicon 2x8 · 8 GB · worst run 1.1× +12% provisional
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 4 vCPU Share Save image

RunsOn has the steadiest build times.

  1. RunsOn 2 machines · worst run 1.1× +3%
  2. GitHub 16 GB · worst run 1.1× +5%
  3. Warpbuild 16 GB · worst run 1.2× +7%
  4. Avrea 16 GB · worst run 1.2× +9%
  5. Blacksmith 16 GB · worst run 1.3× +11%
  6. Namespace 2 machines · worst run 1.2× +12%
  7. RunsOntmpfs 2 machines · worst run 1.6× +14%
  8. NamespaceApple silicon 4x16 · 16 GB · worst run 2.1× +31%
  9. Ubicloud standard · 16 GB · worst run 10.7× +169%
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB · worst run 2.0× +34% not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 8 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 2nd of 7, at +8%.

  1. Avrea 32 GB · worst run 1.1× +6%
  2. RunsOn 2 machines · worst run 1.2× +8%
  3. Ubicloud standard · 32 GB · worst run 1.3× +11%
  4. Warpbuild 32 GB · worst run 1.4× +11%
  5. Blacksmith 32 GB · worst run 1.2× +12%
  6. Namespace 8x16 · 16 GB · worst run 1.8× +17%
  7. RunsOntmpfs m9g.2xlarge · 32 GB · worst run 1.8× +17%
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB · worst run 2.6× +29% not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Every benchmark

  • Explorer Every benchmark, provider and runner in one place: pick a benchmark, open a provider, compare its runners.
  • Builds in detail Every build phase, p50 to max, by best value, time or cost.
  • Queue How long jobs wait for a runner in a burst, and how far build times move from run to run.
  • Bigger runners What doubling the vCPU does to a build's time and cost.
  • All runners All 94 runners in one table, and the price / performance chart.
  • CPU Single and multi-thread scores, job-to-job spread, score per dollar.
  • Cache Save and restore of a 4 GB cache, per cache action and storage.
  • Disk I/O fio in the job's workspace: random and sequential, latency, sync writes.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to a build.
  • EC2 CPU Every EC2 instance family by CPU score and price.
  • Head-to-heads Two runners of the same size, build for build, cost first.
  • Methodology How every figure is measured and ranked, and what changed.
  • Calculator Your own minutes and sizes: what each provider would bill.