v3.6.2 changelog
quickstart →
Pricing Benchmarks Docs Changelog Blog Troubleshooting GitHub Install in 10 min

GitHub Actions runner benchmarks

9 providers, 94 runners, the same jobs on each: price, real builds, CPU, network, disk and cache. AWS CodeBuild and RunsOn run in your own AWS account; the others run your jobs on theirs. Run by RunsOn from an open harness, last on 13 Nov 2026. Where another runner is faster, it says so.

Compare
at
applies to every panel on this page

Scorecard

Scorecardevery provider, every categoryx64 · 2 vCPU Share Save image

RunsOn tmpfs leads on build cost, network and cache; Namespace on CPU and disk; Avrea on build time; boat on price.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00082 $0.00522 5m 19s3 4,2403 212 MiB/s2 12,5105 17s4
RunsOntmpfs your AWS account $0.00093 $0.00501 5m 04s2 3,7875 303 MiB/s1 workspace in RAM, no disk 7s1
Avrea vendor's cloud $0.00405 $0.01534 3m 50s1 4,3072 99 MiB/s7 52,4383 13s3
Blacksmith vendor's cloud $0.00406 $0.02525 6m 19s4 3,7846 102 MiB/s6 76,1322 9s2
Namespace vendor's cloud $0.006010 TypeScript: failed · out of memoryTypeScript: failed · out of memory 4,5461 157 MiB/s3 104,6591 21s5
Warpbuild vendor's cloud $0.00407 $0.02726 6m 48s5 3,9714 111 MiB/s5 22,4854 27s6
Ubicloud vendor's cloud $0.00134 $0.01263 7m 59s6 2,7088 45 MiB/s9 6,7678 59s8
boat vendor's cloud $0.00031 $0.0034 11m 20s 3,5657 38 MiB/s10 10,7836 1m 30s
AWS CodeBuild your AWS account $0.00508 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,5949 155 MiB/s4 1,7619 52s7
GitHub GitHub's cloud $0.00609 $0.06397 10m 39s7 2,29510 72 MiB/s8 6,8687 1m 01s9
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 4 vCPU Share Save image

RunsOn leads on network; RunsOn tmpfs on build cost and cache; Namespace on build time, CPU and disk; boat on price.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
Avrea vendor's cloud $0.00805 $0.06314 7m 53s2 4,3602 163 MiB/s4 49,9953 10s2
RunsOn your AWS account $0.00122 $0.04982 10m 56s5 4,2543 186 MiB/s1 11,3845 16s4
Namespace vendor's cloud $0.012010 $0.08166 7m 41s1 4,4801 184 MiB/s2 105,8081 17s5
RunsOntmpfs your AWS account $0.00163 $0.01451 8m 22s3 3,8057 173 MiB/s3 workspace in RAM, no disk 5s1
Blacksmith vendor's cloud $0.00806 $0.07695 9m 37s4 3,8816 69 MiB/s8 72,7392 12s3
Warpbuild vendor's cloud $0.00807 $0.08977 11m 13s6 4,0585 113 MiB/s6 31,2884 21s6
Ubicloud vendor's cloud $0.00254 $0.05533 15m 31s7 2,8458 51 MiB/s9 7,9827 56s9
boat vendor's cloud $0.00061 Docker: not run: needs more than 8 GB of memoryDocker: not run: needs more than 8 GB of memory 4,1294 35 MiB/s10 9,9816 no cache result
AWS CodeBuild your AWS account $0.01008 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,6019 162 MiB/s5 1,7949 35s7
GitHub GitHub's cloud $0.01209 $0.24608 20m 30s8 2,36110 100 MiB/s7 7,3328 44s8
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 8 vCPU Share Save image

RunsOn tmpfs leads on network and cache; Namespace on build time, CPU and disk; boat on price and build cost.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00252 $0.03583 7m 17s5 3,8024 152 MiB/s3 11,1135 11s3
RunsOntmpfs your AWS account $0.00273 $0.01982 6m 13s3 3,8005 175 MiB/s1 workspace in RAM, no disk 5s1
Namespace vendor's cloud $0.024010 $0.07664 5m 46s1 4,5561 138 MiB/s4 107,1491 17s5
Avrea vendor's cloud $0.01605 $0.09446 5m 54s2 4,4382 108 MiB/s7 52,6453 11s2
Blacksmith vendor's cloud $0.01606 $0.10137 6m 20s4 3,7866 105 MiB/s8 67,3112 12s4
Warpbuild vendor's cloud $0.01607 $0.12518 7m 49s6 4,1233 113 MiB/s6 37,1534 26s6
boat vendor's cloud $0.00121 $0.01551 12m 57s8 3,4198 61 MiB/s9 8,1796 1m 11s
Ubicloud vendor's cloud $0.00504 $0.08585 10m 44s7 3,4967 42 MiB/s10 8,0237 55s9
AWS CodeBuild your AWS account $0.02008 $0.342710 17m 08s10 2,5979 170 MiB/s2 1,7589 34s8
GitHub GitHub's cloud $0.02209 $0.30149 13m 42s9 2,37310 123 MiB/s5 7,4998 31s7
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 2 vCPU Share Save image

RunsOn leads on price and build cost; RunsOn tmpfs on network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00071 $0.00461 5m 05s3 2,4743 225 MiB/s2 13,5335 19s3
RunsOntmpfs your AWS account $0.00102 $0.00562 4m 55s2 2,4674 269 MiB/s1 workspace in RAM, no disk 9s1
Avrea vendor's cloud $0.01208 $0.04105 3m 25s1 4,6601 141 MiB/s5 20,9712 12s2
Blacksmith vendor's cloud $0.00254 $0.03034 12m 08s6 1,3287 84 MiB/s8 31,0281 23s4
Warpbuild vendor's cloud $0.00305 $0.02613 8m 42s4 1,9395 153 MiB/s4 1,7778 29s6
NamespaceApple silicon vendor's cloud $0.04209 TypeScript: failed · out of memoryTypeScript: failed · out of memory 4,1172 158 MiB/s3 19,9573 26s5
GitHub GitHub's cloud $0.00506 $0.04526 9m 03s5 1,8686 100 MiB/s7 8,1196 53s8
Namespace vendor's cloud $0.00607 TypeScript: failed · out of memoryTypeScript: failed · out of memory 1,1499 122 MiB/s6 15,9414 42s7
Ubicloud vendor's cloud $0.00133 $0.0193 15m 28s 1,2978 66 MiB/s9 7,8467 1m 04s9
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple silicon vendor's cloud no 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runner

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 4 vCPU Share Save image

RunsOn leads on price, build cost and network; RunsOn tmpfs on cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00141 $0.01521 9m 52s4 2,4723 179 MiB/s1 13,3825 16s3
RunsOntmpfs your AWS account $0.00162 $0.01982 8m 49s3 2,4694 172 MiB/s2 workspace in RAM, no disk 5s1
Avrea vendor's cloud $0.02408 $0.17727 7m 23s1 4,8231 80 MiB/s8 19,8242 11s2
Blacksmith vendor's cloud $0.00504 $0.08724 17m 27s7 1,3407 97 MiB/s7 32,6911 21s4
NamespaceApple silicon vendor's cloud $0.08409 $0.63919 7m 37s2 4,5082 127 MiB/s4 18,9013 22s5
Warpbuild vendor's cloud $0.00605 $0.09485 15m 48s5 1,9405 130 MiB/s3 1,8128 27s6
GitHub GitHub's cloud $0.00806 $0.13206 16m 30s6 1,8706 103 MiB/s6 5,8937 36s7
Ubicloud vendor's cloud $0.00253 $0.06173 24m 42s9 1,2868 77 MiB/s9 8,3346 58s9
Namespace vendor's cloud $0.01207 $0.24748 23m 33s8 1,1729 122 MiB/s5 16,6134 38s8
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple · 6 vCPU vendor's cloud $0.0800 $0.7647 9m 34s 2,841 68 MiB/s 30,673 27s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 8 vCPU Share Save image

RunsOn leads on price; RunsOn tmpfs on build cost, network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOntmpfs your AWS account $0.00252 $0.01841 6m 42s2 2,4713 286 MiB/s1 workspace in RAM, no disk 5s1
RunsOn your AWS account $0.00251 $0.02392 7m 02s3 2,4722 163 MiB/s3 13,4464 11s3
Avrea vendor's cloud $0.04807 $0.27527 5m 44s1 4,8151 54 MiB/s7 20,7972 11s2
Blacksmith vendor's cloud $0.01004 $0.11994 12m 00s5 1,3435 94 MiB/s5 28,6101 18s4
Warpbuild vendor's cloud $0.01205 $0.12285 10m 14s4 1,9404 180 MiB/s2 1,7816 21s5
Ubicloud vendor's cloud $0.00503 $0.09053 18m 07s7 1,2996 57 MiB/s6 9,2825 49s7
Namespace vendor's cloud $0.02406 $0.20566 17m 08s6 1,1637 127 MiB/s4 16,6353 33s6
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
GitHub GitHub's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
NamespaceApple silicon vendor's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
WarpbuildApple · 12 vCPU vendor's cloud $0.1600 $1.27 7m 56s 3,130 77 MiB/s 30,396 55s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Real builds

Real buildstime and cost per buildlower is betterx64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

boat is the best value and the cheapest; Avrea is the fastest.

  1. boat 4 GB 4m 05s $0.0012
  2. RunsOn m8a.large · EBS gp3 · 8 GB 3m 06s $0.0031
  3. RunsOntmpfs r8a.large · 16 GB 3m 21s $0.0033
  4. Avrea 8 GB 2m 24s $0.0096
  5. Ubicloud standard · 8 GB 4m 31s $0.0056
  6. Namespace 2x8 · 8 GB 2m 43s $0.0163
  7. Warpbuild 8 GB 4m 00s $0.0160
  8. Blacksmith 8 GB 4m 06s $0.0164
  9. AWS CodeBuild small · 3 GB 6m 51s $0.0342
  10. GitHub 8 GB 5m 50s $0.0350
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value; RunsOn tmpfs is the cheapest; Avrea is the fastest.

  1. RunsOn m8a.large · EBS gp3 · 8 GB 2m 13s $0.0021
  2. Avrea 8 GB 1m 26s $0.0057
  3. RunsOntmpfs r8a.large · 16 GB 1m 43s $0.0017
  4. Ubicloud premium · 8 GB 3m 28s $0.0069
  5. Blacksmith 8 GB 2m 13s $0.0088
  6. Warpbuild 8 GB 2m 48s $0.0112
  7. GitHub 8 GB 4m 49s $0.0289
  8. boat 4 GB 7m 15s $0.0022
  9. AWS CodeBuild failed · out of memory
  10. Namespace failed · out of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

Only RunsOn finished this build.

  1. RunsOn r8a.large · EBS gp3 · 16 GB 12m 34s $0.0135
  2. Avrea not run: needs more than 8 GB of memory
  3. AWS CodeBuild not run: needs more than 8 GB of memory
  4. Blacksmith not run: needs more than 8 GB of memory
  5. boat not run: needs more than 8 GB of memory
  6. GitHub not run: needs more than 8 GB of memory
  7. Namespace not run: needs more than 8 GB of memory
  8. NamespaceApple silicon arm64 only
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Ubicloud not run: needs more than 8 GB of memory
  11. Warpbuild not run: needs more than 8 GB of memory
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOn c8a.xlarge · EBS gp3 · 8 GB 2m 02s $0.0036
  2. RunsOntmpfs r8a.xlarge · 32 GB 2m 04s $0.0036
  3. Avrea 16 GB 1m 37s $0.0129
  4. Namespace 4x8 · 8 GB 1m 46s $0.0106
  5. Ubicloud standard · 16 GB 2m 38s $0.0066
  6. boat 8 GB 4m 48s $0.0029
  7. Warpbuild 16 GB 2m 23s $0.0191
  8. Blacksmith 16 GB 1m 59s $0.0159
  9. AWS CodeBuild medium · 8 GB 3m 48s $0.0380
  10. GitHub 16 GB 3m 48s $0.0456
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

boat is the best value and the cheapest; Avrea is the fastest.

  1. boat 8 GB 1m 36s $0.0010
  2. RunsOntmpfs r8a.xlarge · 32 GB 1m 03s $0.0021
  3. Avrea 16 GB 50s $0.0067
  4. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 1m 01s $0.0061
  5. Ubicloud standard · 16 GB 1m 52s $0.0047
  6. Namespace 4x16 · 16 GB 53s $0.0106
  7. Warpbuild 16 GB 1m 21s $0.0108
  8. Blacksmith 16 GB 1m 45s $0.0139
  9. GitHub 16 GB 2m 05s $0.0250
  10. AWS CodeBuild failed · out of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value and the cheapest; Namespace is the fastest.

  1. RunsOntmpfs r8a.xlarge · 32 GB 5m 15s $0.0087
  2. Avrea 16 GB 5m 26s $0.0435
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 7m 53s $0.0402
  4. Namespace 4x16 · 16 GB 5m 02s $0.0604
  5. Blacksmith 16 GB 5m 53s $0.0471
  6. Warpbuild 16 GB 7m 29s $0.0599
  7. Ubicloud premium · 16 GB 11m 01s $0.0441
  8. GitHub 16 GB 14m 37s $0.1754
  9. AWS CodeBuild not run: needs more than 8 GB of memory
  10. boat not run: needs more than 8 GB of memory
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn tmpfs is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 1m 28s $0.0047
  2. RunsOn m8a.2xlarge · EBS gp3 · 32 GB 1m 28s $0.0047
  3. Avrea 32 GB 1m 13s $0.0195
  4. Namespace 8x16 · 16 GB 1m 16s $0.0152
  5. Ubicloud premium · 32 GB 1m 54s $0.0152
  6. Warpbuild 32 GB 1m 37s $0.0259
  7. boat 16 GB 2m 06s $0.0025
  8. Blacksmith 32 GB 1m 21s $0.0216
  9. AWS CodeBuild large · 16 GB 2m 28s $0.0493
  10. GitHub 32 GB 2m 19s $0.0510
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 50s $0.0029
  2. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 49s $0.0049
  3. boat 16 GB 1m 28s $0.0017
  4. Avrea 32 GB 37s $0.0099
  5. Blacksmith 32 GB 52s $0.0139
  6. Warpbuild 32 GB 55s $0.0147
  7. Ubicloud premium · 32 GB 1m 12s $0.0095
  8. Namespace 8x32 · 32 GB 37s $0.0148
  9. AWS CodeBuild large · 16 GB 1m 41s $0.0337
  10. GitHub 32 GB 1m 30s $0.0330
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value; boat is the cheapest; Namespace is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 3m 55s $0.0123
  2. Namespace 8x16 · 16 GB 3m 53s $0.0466
  3. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 5m 00s $0.0262
  4. Avrea 32 GB 4m 04s $0.0651
  5. boat 16 GB 9m 24s $0.0113
  6. Blacksmith 32 GB 4m 07s $0.0659
  7. Ubicloud premium · 32 GB 7m 38s $0.0611
  8. Warpbuild 32 GB 5m 17s $0.0845
  9. GitHub 32 GB 9m 53s $0.2174
  10. AWS CodeBuild large · 16 GB 12m 59s $0.2597
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.large · local NVMe · 8 GB 3m 01s $0.0027
  2. RunsOntmpfs r9g.large · 16 GB 3m 13s $0.0037
  3. Avrea 8 GB 2m 04s $0.0248
  4. Warpbuild 8 GB 5m 01s $0.0151
  5. Ubicloud standard · 8 GB 10m 27s $0.0131
  6. Blacksmith 6 GB 6m 48s $0.0170
  7. GitHub 8 GB 5m 41s $0.0284
  8. NamespaceApple silicon 2x8 · 8 GB 2m 11s $0.0917
  9. Namespace 2x8 · 8 GB 9m 44s $0.0584
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs r9g.large · 16 GB 1m 42s $0.0019
  2. RunsOn m9gd.large · local NVMe · 8 GB 2m 04s $0.0019
  3. Avrea 8 GB 1m 21s $0.0162
  4. Warpbuild 8 GB 3m 41s $0.0110
  5. Blacksmith 6 GB 5m 20s $0.0133
  6. GitHub 8 GB 3m 22s $0.0168
  7. Ubicloud standard · 8 GB 5m 01s $0.0063
  8. AWS CodeBuild no arm64 runner
  9. boat no arm64 runner
  10. Namespace failed · out of memory
  11. NamespaceApple silicon failed · out of memory
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Docker PostHog's production image, built cold with buildx

No 2 vCPU arm64 runner finished this build.

  1. Avrea not run: needs more than 8 GB of memory
  2. AWS CodeBuild no arm64 runner
  3. Blacksmith not run: needs more than 8 GB of memory
  4. boat no arm64 runner
  5. GitHub not run: needs more than 8 GB of memory
  6. Namespace not run: needs more than 8 GB of memory
  7. NamespaceApple silicon not run: needs more than 8 GB of memory
  8. RunsOn not run: needs more than 8 GB of memory
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Ubicloud not run: needs more than 8 GB of memory
  11. Warpbuild not run: needs more than 8 GB of memory
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9g.xlarge · EBS gp3 · 16 GB 2m 01s $0.0036
  2. RunsOntmpfs m9g.xlarge · 16 GB 2m 05s $0.0037
  3. Avrea 16 GB 1m 27s $0.0348
  4. Warpbuild 16 GB 3m 14s $0.0194
  5. GitHub 16 GB 3m 18s $0.0264
  6. Ubicloud standard · 16 GB 6m 07s $0.0153
  7. Blacksmith 16 GB 4m 13s $0.0211
  8. NamespaceApple silicon 4x16 · 16 GB 1m 25s $0.1190
  9. Namespace 4x8 · 8 GB 5m 51s $0.0351
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 1m 28s $0.1173
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB 1m 03s $0.0018
  2. RunsOntmpfs m9g.xlarge · 16 GB 1m 04s $0.0022
  3. Warpbuild 16 GB 1m 37s $0.0097
  4. GitHub 16 GB 1m 44s $0.0139
  5. Blacksmith 16 GB 2m 14s $0.0111
  6. Avrea 16 GB 49s $0.0196
  7. Namespace 4x16 · 16 GB 3m 24s $0.0407
  8. NamespaceApple silicon 4x16 · 16 GB 49s $0.0679
  9. Ubicloud standard · 16 GB 2m 44s $0.0068
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 58s $0.0767
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn is the best value and the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB 6m 48s $0.0099
  2. RunsOntmpfs r9g.xlarge · 32 GB 5m 40s $0.0139
  3. Ubicloud standard · 16 GB 15m 51s $0.0396
  4. Avrea 16 GB 5m 07s $0.1228
  5. Blacksmith 16 GB 11m 00s $0.0550
  6. Warpbuild 16 GB 10m 57s $0.0657
  7. GitHub 16 GB 11m 28s $0.0917
  8. Namespace 4x16 · 16 GB 14m 18s $0.1716
  9. NamespaceApple silicon 4x16 · 16 GB 5m 23s $0.4522
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 7m 08s $0.5707
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the cheapest; RunsOn tmpfs is the best value; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 1m 27s $0.0043
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 1m 27s $0.0042
  3. Avrea 32 GB 1m 07s $0.0536
  4. Warpbuild 32 GB 2m 13s $0.0266
  5. Ubicloud standard · 32 GB 4m 04s $0.0203
  6. Blacksmith 32 GB 2m 55s $0.0292
  7. Namespace 8x16 · 16 GB 4m 08s $0.0496
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 1m 10s $0.1867
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 46s $0.0025
  2. RunsOn m9gd.2xlarge · local NVMe · 32 GB 44s $0.0030
  3. Warpbuild 32 GB 1m 03s $0.0126
  4. Ubicloud standard · 32 GB 1m 51s $0.0092
  5. Blacksmith 32 GB 1m 26s $0.0143
  6. Namespace 8x16 · 16 GB 2m 03s $0.0246
  7. Avrea 32 GB 40s $0.0320
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 47s $0.1240
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn is the best value; RunsOn tmpfs is the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.2xlarge · local NVMe · 32 GB 4m 51s $0.0167
  2. RunsOntmpfs m9g.2xlarge · 32 GB 4m 29s $0.0117
  3. Warpbuild 32 GB 6m 58s $0.0836
  4. Blacksmith 32 GB 7m 39s $0.0765
  5. Ubicloud standard · 32 GB 12m 12s $0.0610
  6. Avrea 32 GB 3m 57s $0.1896
  7. Namespace 8x16 · 16 GB 10m 57s $0.1314
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 5m 59s $0.9573
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

CPU

CPUsingle-thread PassMarkhigher is betterx64 · 2 vCPU Share Save image

Namespace has the fastest core.

RunsOn is 3rd of 10, 7% below the leader. Namespace to GitHub: 2.0× apart.

  1. Namespace 2x8 · 8 GB 4,546
  2. Avrea 8 GB 4,307
  3. RunsOn m8azn.large · EBS gp3 · 8 GB 4,240
  4. Warpbuild 8 GB 3,971
  5. RunsOntmpfs r8a.large · 16 GB 3,787
  6. Blacksmith 8 GB 3,784
  7. boat 4 GB 3,565
  8. Ubicloud standard · 8 GB 2,708
  9. AWS CodeBuild small · 3 GB 2,594
  10. GitHub 8 GB 2,295
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 4 vCPU Share Save image

Namespace and Avrea tie for the fastest core.

RunsOn is 3rd of 10, 5% below the leader. Namespace to GitHub: 1.9× apart.

  1. Namespace 4x16 · 16 GB 4,480
  2. Avrea 16 GB 4,360
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 4,254
  4. boat 8 GB 4,129
  5. Warpbuild 16 GB 4,058
  6. Blacksmith 16 GB 3,881
  7. RunsOntmpfs r8a.xlarge · 32 GB 3,805
  8. Ubicloud premium · 16 GB 2,845
  9. AWS CodeBuild medium · 8 GB 2,601
  10. GitHub 16 GB 2,361
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 8 vCPU Share Save image

Namespace and Avrea tie for the fastest core.

RunsOn is 4th of 10, 17% below the leader. Namespace to GitHub: 1.9× apart.

  1. Namespace 8x32 · 32 GB 4,556
  2. Avrea 32 GB 4,438
  3. Warpbuild 32 GB 4,123
  4. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 3,802
  5. RunsOntmpfs r8a.2xlarge · 64 GB 3,800
  6. Blacksmith 32 GB 3,786
  7. Ubicloud premium · 32 GB 3,496
  8. boat 16 GB 3,419
  9. AWS CodeBuild large · 16 GB 2,597
  10. GitHub 32 GB 2,373
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 2 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 3rd of 9, 47% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 8 GB 4,660
  2. NamespaceApple silicon 2x8 · 8 GB 4,117
  3. RunsOn m9gd.large · local NVMe · 8 GB 2,474
  4. RunsOntmpfs r9g.large · 16 GB 2,467
  5. Warpbuild 8 GB 1,939
  6. GitHub 8 GB 1,868
  7. Blacksmith 6 GB 1,328
  8. Ubicloud standard · 8 GB 1,297
  9. Namespace 2x8 · 8 GB 1,149
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 4 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 3rd of 9, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 16 GB 4,823
  2. NamespaceApple silicon 4x16 · 16 GB 4,508
  3. RunsOn m9gd.xlarge · local NVMe · 16 GB 2,472
  4. RunsOntmpfs m9g.xlarge · 16 GB 2,469
  5. Warpbuild 16 GB 1,940
  6. GitHub 16 GB 1,870
  7. Blacksmith 16 GB 1,340
  8. Ubicloud standard · 16 GB 1,286
  9. Namespace 4x16 · 16 GB 1,172
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 2,841 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 8 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 2nd of 7, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 32 GB 4,815
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 2,472
  3. RunsOntmpfs m9g.2xlarge · 32 GB 2,471
  4. Warpbuild 32 GB 1,940
  5. Blacksmith 32 GB 1,343
  6. Ubicloud standard · 32 GB 1,299
  7. Namespace 8x16 · 16 GB 1,163
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 3,130 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

Price

Priceper minutelower is betterx64 · 2 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.8× lower. boat to Namespace: 20× apart.

  1. boat 4 GB $0.0003
  2. RunsOn m8id.large · local NVMe · 8 GB $0.0008 $0.0025 on-demand
  3. RunsOntmpfs r8a.large · 16 GB $0.0009 $0.0030 on-demand
  4. Ubicloud standard · 8 GB $0.0013
  5. Avrea 8 GB $0.0040
  6. Blacksmith 8 GB $0.0040
  7. Warpbuild 8 GB $0.0040
  8. AWS CodeBuild small · 3 GB $0.0050
  9. GitHub 8 GB $0.0060
  10. Namespace 2x8 · 8 GB $0.0060
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 4 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.0× lower. boat to Namespace: 20× apart.

  1. boat 8 GB $0.0006
  2. RunsOn m8a.xlarge · EBS gp3 · 16 GB $0.0012 $0.0044 on-demand
  3. RunsOntmpfs r8a.xlarge · 32 GB $0.0016 $0.0056 on-demand
  4. Ubicloud standard · 16 GB $0.0025
  5. Avrea 16 GB $0.0080
  6. Blacksmith 16 GB $0.0080
  7. Warpbuild 16 GB $0.0080
  8. AWS CodeBuild medium · 8 GB $0.0100
  9. GitHub 16 GB $0.0120
  10. Namespace 4x16 · 16 GB $0.0120
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 8 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.1× lower. boat to Namespace: 20× apart.

  1. boat 16 GB $0.0012
  2. RunsOn m8i-flex.2xlarge · EBS gp3 · 32 GB $0.0025 $0.0070 on-demand
  3. RunsOntmpfs m8a.2xlarge · 32 GB $0.0027 $0.0084 on-demand
  4. Ubicloud standard · 32 GB $0.0050
  5. Avrea 32 GB $0.0160
  6. Blacksmith 32 GB $0.0160
  7. Warpbuild 32 GB $0.0160
  8. AWS CodeBuild large · 16 GB $0.0200
  9. GitHub 32 GB $0.0220
  10. Namespace 8x32 · 32 GB $0.0240
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 2 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 58× apart.

  1. RunsOn m9g.large · EBS gp3 · 8 GB $0.0007 $0.0019 on-demand
  2. RunsOntmpfs r9g.large · 16 GB $0.0010 $0.0024 on-demand
  3. Ubicloud standard · 8 GB $0.0013
  4. Blacksmith 6 GB $0.0025
  5. Warpbuild 8 GB $0.0030
  6. GitHub 8 GB $0.0050
  7. Namespace 2x8 · 8 GB $0.0060
  8. Avrea 8 GB $0.0120
  9. NamespaceApple silicon 2x8 · 8 GB $0.0420
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 4 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 60× apart.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB $0.0014 $0.0045 on-demand
  2. RunsOntmpfs m9g.xlarge · 16 GB $0.0016 $0.0036 on-demand
  3. Ubicloud standard · 16 GB $0.0025
  4. Blacksmith 16 GB $0.0050
  5. Warpbuild 16 GB $0.0060
  6. GitHub 16 GB $0.0080
  7. Namespace 4x16 · 16 GB $0.0120
  8. Avrea 16 GB $0.0240
  9. NamespaceApple silicon 4x16 · 16 GB $0.0840
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB $0.0800 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 8 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the lowest price per minute.

RunsOn to Avrea: 19× apart.

  1. RunsOn m9g.2xlarge · EBS gp3 · 32 GB $0.0025 $0.0068 on-demand
  2. RunsOntmpfs m9g.2xlarge · 32 GB $0.0025 $0.0068 on-demand
  3. Ubicloud standard · 32 GB $0.0050
  4. Blacksmith 32 GB $0.0100
  5. Warpbuild 32 GB $0.0120
  6. Namespace 8x32 · 32 GB $0.0240
  7. Avrea 32 GB $0.0480
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB $0.1600 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Network

Networkdownload from GitHubhigher is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 2nd of 10, 30% below the leader. RunsOn tmpfs to boat: 8.0× apart.

  1. RunsOntmpfs r8a.large · 16 GB 303 MiB/s
  2. RunsOn median of 9 machines 212 MiB/s
  3. Namespace 2x8 · 8 GB 157 MiB/s
  4. AWS CodeBuild small · 3 GB 155 MiB/s
  5. Warpbuild 8 GB 111 MiB/s
  6. Blacksmith 8 GB 102 MiB/s
  7. Avrea 8 GB 99 MiB/s
  8. GitHub 8 GB 72 MiB/s
  9. Ubicloud median of 2 machines 45 MiB/s
  10. boat 4 GB 38 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterx64 · 4 vCPU Share Save image

RunsOn and Namespace tie for the fastest download from GitHub.

RunsOn to boat: 5.3× apart.

  1. RunsOn median of 8 machines 186 MiB/s
  2. Namespace median of 2 machines 184 MiB/s
  3. RunsOntmpfs r8a.xlarge · 32 GB 173 MiB/s
  4. Avrea 16 GB 163 MiB/s
  5. AWS CodeBuild medium · 8 GB 162 MiB/s
  6. Warpbuild 16 GB 113 MiB/s
  7. GitHub 16 GB 100 MiB/s
  8. Blacksmith 16 GB 69 MiB/s
  9. Ubicloud median of 2 machines 51 MiB/s
  10. boat 8 GB 35 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterx64 · 8 vCPU Share Save image

RunsOn tmpfs and AWS CodeBuild tie for the fastest download from GitHub.

RunsOn is 3rd of 10, 13% below the leader. RunsOn tmpfs to Ubicloud: 4.2× apart.

  1. RunsOntmpfs median of 2 machines 175 MiB/s
  2. AWS CodeBuild large · 16 GB 170 MiB/s
  3. RunsOn median of 7 machines 152 MiB/s
  4. Namespace median of 2 machines 138 MiB/s
  5. GitHub 32 GB 123 MiB/s
  6. Warpbuild 32 GB 113 MiB/s
  7. Avrea 32 GB 108 MiB/s
  8. Blacksmith 32 GB 105 MiB/s
  9. boat 16 GB 61 MiB/s
  10. Ubicloud median of 2 machines 42 MiB/s
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 2nd of 9, 16% below the leader. RunsOn tmpfs to Ubicloud: 4.1× apart.

  1. RunsOntmpfs r9g.large · 16 GB 269 MiB/s
  2. RunsOn median of 3 machines 225 MiB/s
  3. NamespaceApple silicon 2x8 · 8 GB 158 MiB/s
  4. Warpbuild 8 GB 153 MiB/s
  5. Avrea 8 GB 141 MiB/s
  6. Namespace 2x8 · 8 GB 122 MiB/s
  7. GitHub 8 GB 100 MiB/s
  8. Blacksmith 6 GB 84 MiB/s
  9. Ubicloud standard · 8 GB 66 MiB/s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 4 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the fastest download from GitHub.

RunsOn to Ubicloud: 2.3× apart.

  1. RunsOn median of 3 machines 179 MiB/s
  2. RunsOntmpfs median of 2 machines 172 MiB/s
  3. Warpbuild 16 GB 130 MiB/s
  4. NamespaceApple silicon 4x16 · 16 GB 127 MiB/s
  5. Namespace median of 2 machines 122 MiB/s
  6. GitHub 16 GB 103 MiB/s
  7. Blacksmith 16 GB 97 MiB/s
  8. Avrea 16 GB 80 MiB/s
  9. Ubicloud standard · 16 GB 77 MiB/s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 68 MiB/s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Networkdownload from GitHubhigher is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn is 3rd of 7, 43% below the leader. RunsOn tmpfs to Avrea: 5.3× apart.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 286 MiB/s
  2. Warpbuild 32 GB 180 MiB/s
  3. RunsOn median of 3 machines 163 MiB/s
  4. Namespace median of 2 machines 127 MiB/s
  5. Blacksmith 32 GB 94 MiB/s
  6. Ubicloud standard · 32 GB 57 MiB/s
  7. Avrea 32 GB 54 MiB/s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 77 MiB/s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Network in the full table

Disk

Diskrandom reads per secondhigher is betterx64 · 2 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 8.4× higher. Namespace to AWS CodeBuild: 59× apart.

  1. Namespace 2x8 · 8 GB 104,659
  2. Blacksmith 8 GB 76,132
  3. Avrea 8 GB 52,438
  4. Warpbuild 8 GB 22,485
  5. RunsOn i7i.large · local NVMe · 16 GB 12,510
  6. boat 4 GB 10,783
  7. GitHub 8 GB 6,868
  8. Ubicloud standard · 8 GB 6,767
  9. AWS CodeBuild small · 3 GB 1,761
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 4 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 9.3× higher. Namespace to AWS CodeBuild: 59× apart.

  1. Namespace 4x8 · 8 GB 105,808
  2. Blacksmith 16 GB 72,739
  3. Avrea 16 GB 49,995
  4. Warpbuild 16 GB 31,288
  5. RunsOn i7i.xlarge · local NVMe · 32 GB 11,384
  6. boat 8 GB 9,981
  7. Ubicloud standard · 16 GB 7,982
  8. GitHub 16 GB 7,332
  9. AWS CodeBuild medium · 8 GB 1,794
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 8 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 9.6× higher. Namespace to AWS CodeBuild: 61× apart.

  1. Namespace 8x16 · 16 GB 107,149
  2. Blacksmith 32 GB 67,311
  3. Avrea 32 GB 52,645
  4. Warpbuild 32 GB 37,153
  5. RunsOn i7i.2xlarge · local NVMe · 64 GB 11,113
  6. boat 16 GB 8,179
  7. Ubicloud premium · 32 GB 8,023
  8. GitHub 32 GB 7,499
  9. AWS CodeBuild large · 16 GB 1,758
  10. NamespaceApple silicon arm64 only
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 2 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 5th of 8, the leader is 2.3× higher. Blacksmith to Warpbuild: 17× apart.

  1. Blacksmith 6 GB 31,028
  2. Avrea 8 GB 20,971
  3. NamespaceApple silicon 2x8 · 8 GB 19,957
  4. Namespace 2x8 · 8 GB 15,941
  5. RunsOn m9gd.large · local NVMe · 8 GB 13,533
  6. GitHub 8 GB 8,119
  7. Ubicloud standard · 8 GB 7,846
  8. Warpbuild 8 GB 1,777
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. RunsOntmpfs workspace in RAM, no disk
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 4 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 5th of 8, the leader is 2.4× higher. Blacksmith to Warpbuild: 18× apart.

  1. Blacksmith 16 GB 32,691
  2. Avrea 16 GB 19,824
  3. NamespaceApple silicon 4x16 · 16 GB 18,901
  4. Namespace 4x8 · 8 GB 16,613
  5. RunsOn m9gd.xlarge · local NVMe · 16 GB 13,382
  6. Ubicloud standard · 16 GB 8,334
  7. GitHub 16 GB 5,893
  8. Warpbuild 16 GB 1,812
  9. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 30,673 not ranked
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 8 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 4th of 6, the leader is 2.1× higher. Blacksmith to Warpbuild: 16× apart.

  1. Blacksmith 32 GB 28,610
  2. Avrea 32 GB 20,797
  3. Namespace 8x32 · 32 GB 16,635
  4. RunsOn m9gd.2xlarge · local NVMe · 32 GB 13,446
  5. Ubicloud standard · 32 GB 9,282
  6. Warpbuild 32 GB 1,781
  7. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 30,396 not ranked
  8. AWS CodeBuild no arm64 runner
  9. boat no arm64 runner
  10. GitHub no 8 vCPU arm64 runner
  11. NamespaceApple silicon no 8 vCPU arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Cache

Cacherestoring 4 GBlower is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 4th of 9, the leader is 2.4× lower. RunsOn tmpfs to GitHub: 8.7× apart.

  1. RunsOntmpfs r8a.large · actions/cache 7s
  2. Blacksmith actions/cache 9s
  3. Avrea actions/cache 13s
  4. RunsOn r8a.large · EBS gp3 · actions/cache 17s
  5. Namespace 2x8 · actions/cache 21s
  6. Warpbuild actions/cache 27s
  7. AWS CodeBuild small · actions/cache 52s
  8. Ubicloud standard · actions/cache 59s
  9. GitHub 2-core · actions/cache 1m 01s
  10. boat actions/cache 1m 30s provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 4th of 9, the leader is 3.2× lower. RunsOn tmpfs to Ubicloud: 11× apart.

  1. RunsOntmpfs r8a.xlarge · actions/cache 5s
  2. Avrea actions/cache 10s
  3. Blacksmith actions/cache 12s
  4. RunsOn m8azn.xlarge · EBS gp3 · actions/cache 16s
  5. Namespace 4x8 · actions/cache 17s
  6. Warpbuild actions/cache 21s
  7. AWS CodeBuild medium · actions/cache 35s
  8. GitHub 4-core · actions/cache 44s
  9. Ubicloud premium · actions/cache 56s
  10. boat no cache result
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 9, the leader is 2.2× lower. RunsOn tmpfs to Ubicloud: 11× apart.

  1. RunsOntmpfs m8a.2xlarge · actions/cache 5s
  2. Avrea actions/cache 11s
  3. RunsOn m8a.2xlarge · EBS gp3 · actions/cache 11s
  4. Blacksmith actions/cache 12s
  5. Namespace 8x16 · actions/cache 17s
  6. Warpbuild actions/cache 26s
  7. GitHub 8-core · actions/cache 31s
  8. AWS CodeBuild large · actions/cache 34s
  9. Ubicloud standard · actions/cache 55s
  10. boat actions/cache 1m 11s provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 9, the leader is 2.1× lower. RunsOn tmpfs to Ubicloud: 7.1× apart.

  1. RunsOntmpfs r9g.large · actions/cache 9s
  2. Avrea actions/cache 12s
  3. RunsOn m9g.large · EBS gp3 · actions/cache 19s
  4. Blacksmith actions/cache 23s
  5. NamespaceApple silicon 2x8 · actions/cache 26s
  6. Warpbuild actions/cache 29s
  7. Namespace 2x8 · actions/cache 42s
  8. GitHub 2-core · actions/cache 53s
  9. Ubicloud standard · actions/cache 1m 04s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 9, the leader is 3.2× lower. RunsOn tmpfs to Ubicloud: 12× apart.

  1. RunsOntmpfs r9g.xlarge · actions/cache 5s
  2. Avrea actions/cache 11s
  3. RunsOn m9g.xlarge · EBS gp3 · actions/cache 16s
  4. Blacksmith actions/cache 21s
  5. NamespaceApple silicon 4x16 · actions/cache 22s
  6. Warpbuild actions/cache 27s
  7. GitHub 4-core · actions/cache 36s
  8. Namespace 4x16 · actions/cache 38s
  9. Ubicloud standard · actions/cache 58s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · actions/cache 27s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn is 3rd of 7, the leader is 2.2× lower. RunsOn tmpfs to Ubicloud: 9.8× apart.

  1. RunsOntmpfs m9g.2xlarge · actions/cache 5s
  2. Avrea actions/cache 11s
  3. RunsOn m9g.2xlarge · EBS gp3 · actions/cache 11s
  4. Blacksmith actions/cache 18s
  5. Warpbuild actions/cache 21s
  6. Namespace 8x16 · actions/cache 33s
  7. Ubicloud standard · actions/cache 49s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · actions/cache 55s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Predictability

Predictabilityrun to runlower is betterx64 · 2 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 3rd of 7, at +10%.

  1. Avrea 8 GB · worst run 1.1× +4%
  2. RunsOntmpfs r8a.large · 16 GB · worst run 1.1× +6%
  3. RunsOn 2 machines · worst run 1.2× +10%
  4. Ubicloud 2 machines · worst run 1.5× +23%
  5. Warpbuild 8 GB · worst run 1.5× +25%
  6. GitHub 8 GB · worst run 1.3× +28%
  7. Blacksmith 8 GB · worst run 1.5× +33%
  8. AWS CodeBuild small · 3 GB · worst run 1.1× +8% provisional
  9. boat 4 GB · worst run 1.2× +17% provisional
  10. Namespace 2x8 · 8 GB · worst run 1.3× +20% provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 4 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 3rd of 9, at +7%.

  1. Avrea 16 GB · worst run 1.1× +3%
  2. RunsOntmpfs r8a.xlarge · 32 GB · worst run 1.2× +4%
  3. RunsOn 2 machines · worst run 1.4× +7%
  4. Warpbuild 16 GB · worst run 1.2× +13%
  5. GitHub 16 GB · worst run 1.2× +15%
  6. Namespace 2 machines · worst run 1.7× +24%
  7. Ubicloud 2 machines · worst run 1.7× +30%
  8. Blacksmith 16 GB · worst run 2.8× +58%
  9. boat 8 GB · worst run 2.0× +58%
  10. AWS CodeBuild medium · 8 GB · worst run 1.1× +4% provisional
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 8 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 4th of 10, at +9%.

  1. Avrea 32 GB · worst run 1.1× +4%
  2. AWS CodeBuild large · 16 GB · worst run 1.1× +5%
  3. RunsOntmpfs m8a.2xlarge · 32 GB · worst run 1.1× +7%
  4. RunsOn 2 machines · worst run 1.5× +9%
  5. Warpbuild 32 GB · worst run 1.3× +12%
  6. Ubicloud premium · 32 GB · worst run 1.4× +15%
  7. GitHub 32 GB · worst run 1.3× +16%
  8. Namespace 2 machines · worst run 1.6× +22%
  9. Blacksmith 32 GB · worst run 1.9× +41%
  10. boat 16 GB · worst run 4.7× +147%
  11. NamespaceApple silicon arm64 only
  12. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 2 vCPU Share Save image

RunsOn has the steadiest build times.

  1. RunsOn m9gd.large · local NVMe · 8 GB · worst run 1.1× +5%
  2. Blacksmith 6 GB · worst run 1.2× +8%
  3. Avrea 8 GB · worst run 1.2× +12%
  4. GitHub 8 GB · worst run 1.3× +15%
  5. RunsOntmpfs r9g.large · 16 GB · worst run 1.7× +26%
  6. Warpbuild 8 GB · worst run 1.5× +33%
  7. Ubicloud standard · 8 GB · worst run 1.0× +3% provisional
  8. Namespace 2x8 · 8 GB · worst run 1.1× +4% provisional
  9. NamespaceApple silicon 2x8 · 8 GB · worst run 1.1× +12% provisional
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. WarpbuildApple silicon no 2 vCPU arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 4 vCPU Share Save image

RunsOn has the steadiest build times.

  1. RunsOn 2 machines · worst run 1.1× +3%
  2. GitHub 16 GB · worst run 1.1× +5%
  3. Warpbuild 16 GB · worst run 1.2× +7%
  4. Avrea 16 GB · worst run 1.2× +9%
  5. Blacksmith 16 GB · worst run 1.3× +11%
  6. Namespace 2 machines · worst run 1.2× +12%
  7. RunsOntmpfs 2 machines · worst run 1.6× +14%
  8. NamespaceApple silicon 4x16 · 16 GB · worst run 2.1× +31%
  9. Ubicloud standard · 16 GB · worst run 10.7× +169%
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB · worst run 2.0× +34% not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 8 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 2nd of 7, at +8%.

  1. Avrea 32 GB · worst run 1.1× +6%
  2. RunsOn 2 machines · worst run 1.2× +8%
  3. Ubicloud standard · 32 GB · worst run 1.3× +11%
  4. Warpbuild 32 GB · worst run 1.4× +11%
  5. Blacksmith 32 GB · worst run 1.2× +12%
  6. Namespace 8x16 · 16 GB · worst run 1.8× +17%
  7. RunsOntmpfs m9g.2xlarge · 32 GB · worst run 1.8× +17%
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB · worst run 2.6× +29% not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Every benchmark

  • Explorer Every benchmark, provider and runner in one place: pick a benchmark, open a provider, compare its runners.
  • Builds in detail Every build phase, p50 to max, by best value, time or cost.
  • Queue How long jobs wait for a runner in a burst, and how far build times move from run to run.
  • Bigger runners What doubling the vCPU does to a build's time and cost.
  • All runners All 94 runners in one table, and the price / performance chart.
  • CPU Single and multi-thread scores, job-to-job spread, score per dollar.
  • Cache Save and restore of a 4 GB cache, per cache action and storage.
  • Disk I/O fio in the job's workspace: random and sequential, latency, sync writes.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to a build.
  • EC2 CPU Every EC2 instance family by CPU score and price.
  • Head-to-heads Two runners of the same size, build for build, cost first.
  • Methodology How every figure is measured and ranked, and what changed.
  • Calculator Your own minutes and sizes: what each provider would bill.