quickstart →

GitHub Actions runner benchmarks

11 providers, 102 runners, the same jobs on each: price, real builds, CPU, network, disk and cache. AWS CodeBuild and RunsOn run in your own AWS account; the others run your jobs on theirs. Run by RunsOn from an open harness, last on 22 Nov 2026. Where another runner is faster, it says so.

Compare
at
applies to every panel on this page

Scorecard

Scorecardevery provider, every categoryx64 · 2 vCPU Share Save image

RunsOn tmpfs leads on build cost, network and cache; Namespace on CPU and disk; Avrea on build time; boat on price.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00082 $0.00462 5m 24s3 4,2343 205 MiB/s2 12,5105 17s4
RunsOntmpfs your AWS account $0.00083 $0.00421 4m 50s2 3,7975 303 MiB/s1 workspace in RAM, no disk 7s1
Avrea vendor's cloud $0.00406 $0.01524 3m 49s1 4,3582 97 MiB/s6 53,5983 12s3
Blacksmith vendor's cloud $0.00407 $0.02195 5m 28s4 3,7256 95 MiB/s7 71,4262 9s2
Namespace vendor's cloud $0.006011 $0.0239 3m 59s 4,5431 163 MiB/s3 105,0541 21s5
Warpbuild vendor's cloud $0.00408 $0.02816 7m 01s5 3,9364 111 MiB/s5 24,4814 26s6
Ubicloud vendor's cloud $0.00135 $0.01043 8m 21s6 2,8667 50 MiB/s10 6,8897 59s8
boat vendor's cloud $0.00031 $0.0035 11m 47s 2,3029 72 MiB/s8 10,7836 1m 32s10
AWS CodeBuild your AWS account $0.00509 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,6028 155 MiB/s4 1,7639 52s7
GitHub GitHub's cloud $0.006010 $0.06297 10m 29s7 2,24310 72 MiB/s9 6,8688 1m 00s9
Sengi vendor's cloud $0.00104 $0.0173 5m 46s 3,705 51 MiB/s 13,893 no cache result
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
StarSling vendor's cloud no 2 vCPU x64 runnerno 2 vCPU x64 runnerno 2 vCPU x64 runnerno 2 vCPU x64 runnerno 2 vCPU x64 runnerno 2 vCPU x64 runnerno 2 vCPU x64 runner
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 4 vCPU Share Save image

RunsOn tmpfs leads on build cost and cache; Namespace on build time, CPU and disk; boat on price; StarSling on network.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
Avrea vendor's cloud $0.00806 $0.06304 7m 53s2 4,4012 163 MiB/s4 49,5504 10s2
RunsOn your AWS account $0.00122 $0.04963 10m 55s6 4,2553 186 MiB/s3 11,9506 16s4
RunsOntmpfs your AWS account $0.00153 $0.01391 8m 19s3 3,8025 159 MiB/s5 workspace in RAM, no disk 5s1
Namespace vendor's cloud $0.012012 $0.08087 7m 35s1 4,4411 124 MiB/s6 104,6461 18s5
StarSling vendor's cloud $0.00808 $0.06795 8m 29s4 3,2957 217 MiB/s1 32,0385 23s7
Blacksmith vendor's cloud $0.00807 $0.07156 8m 57s5 3,6146 83 MiB/s9 82,0002 10s3
Warpbuild vendor's cloud $0.00809 $0.09258 11m 34s7 4,0814 99 MiB/s8 63,5583 21s6
Ubicloud vendor's cloud $0.00255 $0.04262 17m 03s8 3,1468 54 MiB/s11 7,8018 56s10
boat vendor's cloud $0.00061 Docker: not run: needs more than 8 GB of memoryDocker: not run: needs more than 8 GB of memory 3,0159 61 MiB/s10 10,3387 1m 36s11
AWS CodeBuild your AWS account $0.010010 TypeScript: failed · out of memoryTypeScript: failed · out of memory 2,62110 213 MiB/s2 1,79410 35s8
GitHub GitHub's cloud $0.012011 $0.24329 20m 16s9 2,36511 100 MiB/s7 7,3329 38s9
Sengi vendor's cloud $0.00204 $0.0614 10m 14s 3,724 82 MiB/s 13,632 7s
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryx64 · 8 vCPU Share Save image

RunsOn leads on network; RunsOn tmpfs on cache; Namespace on build time, CPU and disk; boat on price and build cost.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
Avrea vendor's cloud $0.01606 $0.09476 5m 55s2 4,4342 150 MiB/s4 52,1213 10s3
RunsOntmpfs your AWS account $0.00243 $0.01712 6m 16s4 3,7984 172 MiB/s2 workspace in RAM, no disk 5s1
RunsOn your AWS account $0.00242 $0.03353 7m 15s6 3,7405 178 MiB/s1 11,0837 11s4
Namespace vendor's cloud $0.024012 $0.07584 5m 43s1 4,5621 137 MiB/s5 106,2281 18s5
Blacksmith vendor's cloud $0.01607 $0.09927 6m 12s3 3,6656 105 MiB/s8 74,0952 8s2
StarSling vendor's cloud $0.01608 $0.10328 6m 27s5 3,4128 130 MiB/s6 52,0584 24s7
boat vendor's cloud $0.00121 $0.01321 11m 00s9 2,8809 79 MiB/s10 11,3836 1m 11s11
Warpbuild vendor's cloud $0.01609 $0.12609 7m 53s7 4,0403 101 MiB/s9 37,1535 23s6
Ubicloud vendor's cloud $0.00505 $0.07615 9m 31s8 3,6167 50 MiB/s11 9,5438 55s10
AWS CodeBuild your AWS account $0.020010 $0.342711 17m 08s11 2,62610 161 MiB/s3 1,75810 35s9
GitHub GitHub's cloud $0.022011 $0.298510 13m 34s10 2,35011 107 MiB/s7 7,5599 33s8
Sengi vendor's cloud $0.00404 $0.0892 7m 26s 3,745 56 MiB/s 13,912 6s
NamespaceApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only
WarpbuildApple silicon vendor's cloud arm64 onlyRust: arm64 onlyRust: arm64 onlyarm64 onlyarm64 onlyarm64 onlyarm64 only

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 2 vCPU Share Save image

RunsOn leads on price and build cost; RunsOn tmpfs on network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost Rust + TypeScript ↓ lower Build time Rust + TypeScript ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00081 $0.00451 5m 06s3 2,4753 177 MiB/s2 13,5335 19s3
RunsOntmpfs your AWS account $0.00102 $0.00552 5m 03s2 2,4664 191 MiB/s1 workspace in RAM, no disk 8s1
Avrea vendor's cloud $0.01208 $0.04076 3m 24s1 4,7441 141 MiB/s5 20,9092 12s2
Blacksmith vendor's cloud $0.00254 $0.03045 12m 09s6 1,3307 92 MiB/s8 31,0281 22s4
Warpbuild vendor's cloud $0.00305 $0.02604 8m 39s4 1,9395 167 MiB/s3 1,7798 28s6
NamespaceApple silicon vendor's cloud $0.04209 $0.1333 3m 11s 4,5942 143 MiB/s4 19,8293 24s5
Ubicloud vendor's cloud $0.00133 $0.01763 14m 07s7 1,2978 58 MiB/s9 8,4526 1m 04s9
GitHub GitHub's cloud $0.00506 $0.04577 9m 08s5 1,8696 120 MiB/s7 8,1197 53s8
Namespace vendor's cloud $0.00607 TypeScript: failed · out of memoryTypeScript: failed · out of memory 1,1469 122 MiB/s6 15,9414 43s7
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
Sengi vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
StarSling vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple silicon vendor's cloud no 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runnerno 2 vCPU arm64 runner

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up Rust + TypeScript (the builds two providers or more finished at this size); RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 4 vCPU Share Save image

RunsOn leads on price, build cost and network; RunsOn tmpfs on cache; Avrea on CPU; Blacksmith on disk; Namespace Apple silicon on build time.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOn your AWS account $0.00131 $0.01461 9m 45s4 2,4723 172 MiB/s1 13,3685 16s3
RunsOntmpfs your AWS account $0.00172 $0.01932 8m 47s3 2,4674 167 MiB/s2 workspace in RAM, no disk 5s1
Avrea vendor's cloud $0.02408 $0.18047 7m 31s2 4,8161 86 MiB/s8 20,3012 10s2
Blacksmith vendor's cloud $0.00504 $0.08604 17m 12s6 1,3407 97 MiB/s7 32,6911 20s4
NamespaceApple silicon vendor's cloud $0.08409 $0.60979 7m 16s1 4,5842 127 MiB/s4 18,5703 21s5
Warpbuild vendor's cloud $0.00605 $0.09635 16m 03s5 1,9405 158 MiB/s3 1,8128 27s6
GitHub GitHub's cloud $0.00806 $0.13876 17m 21s7 1,8706 118 MiB/s6 7,6947 37s7
Ubicloud vendor's cloud $0.00253 $0.05803 23m 13s8 1,2868 74 MiB/s9 8,3346 1m 01s9
Namespace vendor's cloud $0.01207 $0.24418 23m 26s9 1,1629 125 MiB/s5 16,7594 39s8
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
Sengi vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
StarSling vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple · 6 vCPU vendor's cloud $0.0800 $0.7293 9m 07s 3,006 68 MiB/s 29,080 37s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Scorecardevery provider, every categoryarm64 · 8 vCPU Share Save image

RunsOn leads on price; RunsOn tmpfs on build cost, network and cache; Avrea on build time and CPU; Blacksmith on disk.

Provider Runs in Price per minute ↓ lower Build cost all 3 builds ↓ lower Build time all 3 builds ↓ lower CPU PassMark single-thread ↑ higher Network download from GitHub ↑ higher Disk random reads per second ↑ higher Cache 4 GB restore ↓ lower
RunsOntmpfs your AWS account $0.00222 $0.01611 6m 31s2 2,4713 253 MiB/s1 workspace in RAM, no disk 4s1
RunsOn your AWS account $0.00221 $0.02332 6m 53s3 2,4722 161 MiB/s3 13,4624 10s3
Avrea vendor's cloud $0.04807 $0.27567 5m 45s1 4,8151 84 MiB/s6 20,3972 10s2
Blacksmith vendor's cloud $0.01004 $0.11964 11m 58s5 1,3445 101 MiB/s5 31,9051 17s4
Warpbuild vendor's cloud $0.01205 $0.12245 10m 12s4 1,9404 198 MiB/s2 1,7816 21s5
Namespace vendor's cloud $0.02406 $0.25496 17m 10s6 1,1667 134 MiB/s4 17,4523 34s6
Ubicloud vendor's cloud $0.00503 $0.08603 17m 12s7 1,2986 57 MiB/s7 8,4535 50s7
AWS CodeBuild your AWS account no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
boat vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
GitHub GitHub's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
NamespaceApple silicon vendor's cloud no 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runnerno 8 vCPU arm64 runner
Sengi vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
StarSling vendor's cloud no arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runnerno arm64 runner
WarpbuildApple · 12 vCPU vendor's cloud $0.1600 $1.15 7m 13s 3,145 70 MiB/s 30,102 1m 06s

Each figure is the row's best machine of this size for that column, named in the panel below; the small number is its place. Yellow marks a column's leader, ties within 5% included; rose marks a figure 2 times worse than the column's median or more, RunsOn's included. Build cost and time add up all 3 builds; RunsOn is priced at EC2 spot, licenses and plan fees are flat and not in a per-minute figure. Runners of another kind have a row and a place of their own: Apple silicon machines, and RunsOn's large-memory instances with the workspace in RAM (tmpfs). A faded figure takes no place: under 3 runs, or a machine of another size. Runs in: your own AWS account your own AWS account (AWS CodeBuild and RunsOn), so jobs, code and secrets never leave it; the provider's cloud the provider's cloud. Rows are in order of average place across the columns: an order, not a score.

Real builds

Real buildstime and cost per buildlower is betterx64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

boat is the best value and the cheapest; Avrea is the fastest.

  1. boat 4 GB 4m 35s $0.0014
  2. RunsOn m8a.large · EBS gp3 · 8 GB 3m 11s $0.0026
  3. RunsOntmpfs r8a.large · 16 GB 3m 07s $0.0027
  4. Avrea 8 GB 2m 25s $0.0096
  5. Ubicloud standard · 8 GB 4m 49s $0.0060
  6. Namespace 2x8 · 8 GB 2m 36s $0.0156
  7. Blacksmith 8 GB 3m 33s $0.0142
  8. Warpbuild 8 GB 4m 10s $0.0167
  9. GitHub 8 GB 5m 53s $0.0353
  10. AWS CodeBuild small · 3 GB 6m 52s $0.0343
  11. Sengi premium · 8 GB 3m 15s $0.0097
  12. NamespaceApple silicon arm64 only
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value; RunsOn tmpfs is the cheapest; Avrea is the fastest.

  1. RunsOn m8a.large · EBS gp3 · 8 GB 2m 13s $0.0020
  2. RunsOntmpfs r8a.large · 16 GB 1m 44s $0.0015
  3. Avrea 8 GB 1m 24s $0.0056
  4. Ubicloud standard · 8 GB 3m 33s $0.0044
  5. Blacksmith 8 GB 1m 56s $0.0077
  6. Warpbuild 8 GB 2m 51s $0.0114
  7. GitHub 8 GB 4m 36s $0.0276
  8. Namespace 2x8 · 8 GB 1m 23s $0.0083
  9. boat 4 GB 7m 12s $0.0022
  10. Sengi premium · 8 GB 2m 31s $0.0076
  11. AWS CodeBuild failed · out of memory
  12. NamespaceApple silicon arm64 only
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

Only RunsOn finished this build.

  1. RunsOn r8a.large · EBS gp3 · 16 GB 12m 30s $0.0111
  2. Avrea not run: needs more than 8 GB of memory
  3. AWS CodeBuild not run: needs more than 8 GB of memory
  4. Blacksmith not run: needs more than 8 GB of memory
  5. boat not run: needs more than 8 GB of memory
  6. GitHub not run: needs more than 8 GB of memory
  7. Namespace not run: needs more than 8 GB of memory
  8. NamespaceApple silicon arm64 only
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Sengi not run: needs more than 8 GB of memory
  11. StarSling no 2 vCPU x64 runner
  12. Ubicloud not run: needs more than 8 GB of memory
  13. Warpbuild not run: needs more than 8 GB of memory
  14. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOn c8a.xlarge · EBS gp3 · 8 GB 2m 03s $0.0035
  2. RunsOntmpfs r8a.xlarge · 32 GB 2m 03s $0.0035
  3. Namespace 4x8 · 8 GB 1m 42s $0.0102
  4. Avrea 16 GB 1m 37s $0.0129
  5. StarSling 16 GB 1m 41s $0.0135
  6. Ubicloud standard · 16 GB 3m 16s $0.0082
  7. boat 8 GB 2m 42s $0.0016
  8. Warpbuild 16 GB 2m 36s $0.0208
  9. Blacksmith 16 GB 2m 00s $0.0159
  10. AWS CodeBuild medium · 8 GB 3m 46s $0.0377
  11. GitHub 16 GB 3m 48s $0.0456
  12. Sengi premium · 16 GB 1m 58s $0.0118
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value; boat is the cheapest; Avrea is the fastest.

  1. RunsOntmpfs r8a.xlarge · 32 GB 1m 01s $0.0020
  2. boat 8 GB 1m 44s $0.0010
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 59s $0.0060
  4. Avrea 16 GB 50s $0.0066
  5. StarSling 16 GB 55s $0.0073
  6. Ubicloud standard · 16 GB 1m 52s $0.0046
  7. Namespace 4x16 · 16 GB 56s $0.0112
  8. Warpbuild 16 GB 1m 20s $0.0107
  9. Blacksmith 16 GB 1m 09s $0.0091
  10. GitHub 16 GB 2m 00s $0.0240
  11. Sengi premium · 16 GB 1m 18s $0.0078
  12. AWS CodeBuild failed · out of memory
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value and the cheapest; Namespace is the fastest.

  1. RunsOntmpfs r8a.xlarge · 32 GB 5m 15s $0.0084
  2. Avrea 16 GB 5m 26s $0.0435
  3. StarSling 16 GB 5m 53s $0.0471
  4. Namespace 4x16 · 16 GB 4m 57s $0.0594
  5. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 7m 54s $0.0401
  6. Blacksmith 16 GB 5m 49s $0.0465
  7. Ubicloud standard · 16 GB 11m 56s $0.0298
  8. Warpbuild 16 GB 7m 38s $0.0611
  9. GitHub 16 GB 14m 28s $0.1736
  10. Sengi premium · 16 GB 6m 58s $0.0418
  11. AWS CodeBuild not run: needs more than 8 GB of memory
  12. boat not run: needs more than 8 GB of memory
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterx64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn tmpfs is the best value; boat is the cheapest; StarSling is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 1m 28s $0.0041
  2. RunsOn m8a.2xlarge · EBS gp3 · 32 GB 1m 28s $0.0044
  3. Namespace 8x16 · 16 GB 1m 14s $0.0147
  4. Avrea 32 GB 1m 13s $0.0195
  5. StarSling 32 GB 1m 12s $0.0192
  6. boat 16 GB 1m 39s $0.0020
  7. Ubicloud premium · 32 GB 1m 56s $0.0154
  8. Warpbuild 32 GB 1m 36s $0.0255
  9. Blacksmith 32 GB 1m 22s $0.0217
  10. AWS CodeBuild large · 16 GB 2m 34s $0.0512
  11. GitHub 32 GB 2m 32s $0.0555
  12. Sengi premium · 32 GB 1m 20s $0.0160
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value; boat is the cheapest; Namespace is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 48s $0.0025
  2. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 47s $0.0048
  3. boat 16 GB 51s $0.0010
  4. Avrea 32 GB 37s $0.0097
  5. StarSling 32 GB 44s $0.0117
  6. Ubicloud premium · 32 GB 1m 02s $0.0082
  7. Blacksmith 32 GB 44s $0.0117
  8. Warpbuild 32 GB 55s $0.0147
  9. Namespace 8x32 · 32 GB 36s $0.0144
  10. AWS CodeBuild large · 16 GB 1m 42s $0.0338
  11. GitHub 32 GB 1m 20s $0.0291
  12. Sengi premium · 32 GB 42s $0.0084
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value; boat is the cheapest; Namespace is the fastest.

  1. RunsOntmpfs m8a.2xlarge · 32 GB 4m 00s $0.0105
  2. RunsOn m8a.2xlarge · gp3 1000 MiB/s · 32 GB 5m 00s $0.0243
  3. Namespace 8x16 · 16 GB 3m 54s $0.0467
  4. boat 16 GB 8m 31s $0.0102
  5. Avrea 32 GB 4m 06s $0.0655
  6. Blacksmith 32 GB 4m 07s $0.0657
  7. StarSling 32 GB 4m 31s $0.0723
  8. Warpbuild 32 GB 5m 22s $0.0859
  9. Ubicloud premium · 32 GB 6m 34s $0.0525
  10. GitHub 32 GB 9m 43s $0.2138
  11. AWS CodeBuild large · 16 GB 12m 53s $0.2577
  12. Sengi premium · 32 GB 5m 24s $0.0648
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 2 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Avrea is the fastest.

  1. RunsOn m9gd.large · local NVMe · 8 GB 3m 03s $0.0026
  2. RunsOntmpfs r9g.large · 16 GB 3m 24s $0.0037
  3. Avrea 8 GB 2m 03s $0.0245
  4. Warpbuild 8 GB 4m 59s $0.0149
  5. Blacksmith 6 GB 6m 41s $0.0167
  6. Ubicloud standard · 8 GB 9m 06s $0.0114
  7. GitHub 8 GB 5m 42s $0.0285
  8. NamespaceApple silicon 2x8 · 8 GB 2m 03s $0.0857
  9. Namespace 2x8 · 8 GB 10m 08s $0.0608
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn is the cheapest; RunsOn tmpfs is the best value; Avrea is the fastest.

  1. RunsOntmpfs r9g.large · 16 GB 1m 39s $0.0019
  2. RunsOn m9gd.large · local NVMe · 8 GB 2m 04s $0.0018
  3. Avrea 8 GB 1m 21s $0.0162
  4. Ubicloud standard · 8 GB 5m 01s $0.0063
  5. Blacksmith 6 GB 5m 28s $0.0136
  6. GitHub 8 GB 3m 27s $0.0172
  7. Warpbuild 8 GB 3m 40s $0.0110
  8. NamespaceApple silicon 2x8 · 8 GB 1m 08s $0.0476
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. Namespace failed · out of memory
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

Docker PostHog's production image, built cold with buildx

No 2 vCPU arm64 runner finished this build.

  1. Avrea not run: needs more than 8 GB of memory
  2. AWS CodeBuild no arm64 runner
  3. Blacksmith not run: needs more than 8 GB of memory
  4. boat no arm64 runner
  5. GitHub not run: needs more than 8 GB of memory
  6. Namespace not run: needs more than 8 GB of memory
  7. NamespaceApple silicon not run: needs more than 8 GB of memory
  8. RunsOn not run: needs more than 8 GB of memory
  9. RunsOntmpfs not run: needs more than 16 GB of RAM with tmpfs, where Docker's data root lives in RAM
  10. Sengi no arm64 runner
  11. StarSling no arm64 runner
  12. Ubicloud not run: needs more than 8 GB of memory
  13. Warpbuild not run: needs more than 8 GB of memory
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 4 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9g.xlarge · EBS gp3 · 16 GB 2m 01s $0.0039
  2. RunsOntmpfs m9g.xlarge · 16 GB 2m 05s $0.0039
  3. Warpbuild 16 GB 3m 09s $0.0189
  4. Avrea 16 GB 1m 29s $0.0356
  5. Blacksmith 16 GB 4m 07s $0.0205
  6. GitHub 16 GB 3m 27s $0.0276
  7. Ubicloud standard · 16 GB 5m 47s $0.0145
  8. NamespaceApple silicon 4x16 · 16 GB 1m 20s $0.1120
  9. Namespace 4x8 · 8 GB 6m 12s $0.0372
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 1m 21s $0.1080
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn is the best value and the cheapest; Namespace Apple silicon is the fastest.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB 1m 03s $0.0017
  2. RunsOntmpfs m9g.xlarge · 16 GB 1m 02s $0.0022
  3. Warpbuild 16 GB 1m 36s $0.0095
  4. Ubicloud standard · 16 GB 2m 34s $0.0064
  5. GitHub 16 GB 1m 42s $0.0136
  6. Blacksmith 16 GB 2m 12s $0.0110
  7. Avrea 16 GB 48s $0.0190
  8. NamespaceApple silicon 4x16 · 16 GB 47s $0.0658
  9. Namespace 4x16 · 16 GB 3m 08s $0.0376
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 53s $0.0707
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn is the cheapest; RunsOn tmpfs is the best value; Namespace Apple silicon is the fastest.

  1. RunsOntmpfs r9g.xlarge · 32 GB 5m 40s $0.0132
  2. RunsOn m9gd.xlarge · local NVMe · 16 GB 6m 41s $0.0090
  3. Blacksmith 16 GB 10m 54s $0.0545
  4. Ubicloud standard · 16 GB 14m 52s $0.0372
  5. Avrea 16 GB 5m 15s $0.1258
  6. Warpbuild 16 GB 11m 18s $0.0678
  7. GitHub 16 GB 12m 12s $0.0975
  8. Namespace 4x16 · 16 GB 14m 07s $0.1693
  9. NamespaceApple silicon 4x16 · 16 GB 5m 09s $0.4319
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 6m 53s $0.5507
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

Real buildstime and cost per buildlower is betterarm64 · 8 vCPU Share Save image

Rust cold build of spotify-player: cargo test and clippy

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 1m 27s $0.0037
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 1m 28s $0.0037
  3. Warpbuild 32 GB 2m 11s $0.0262
  4. Blacksmith 32 GB 2m 51s $0.0284
  5. Ubicloud standard · 32 GB 3m 51s $0.0192
  6. Avrea 32 GB 1m 07s $0.0532
  7. Namespace 8x32 · 32 GB 4m 05s $0.0980
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 1m 10s $0.1867
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

TypeScript cold bun install and typecheck of opencode

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 46s $0.0022
  2. RunsOn m9gd.2xlarge · local NVMe · 32 GB 44s $0.0032
  3. Warpbuild 32 GB 1m 04s $0.0128
  4. Ubicloud standard · 32 GB 1m 43s $0.0086
  5. Blacksmith 32 GB 1m 24s $0.0140
  6. Namespace 8x16 · 16 GB 2m 03s $0.0246
  7. Avrea 32 GB 39s $0.0312
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 41s $0.1080
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Docker PostHog's production image, built cold with buildx

RunsOn tmpfs is the best value and the cheapest; Avrea is the fastest.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 4m 19s $0.0101
  2. RunsOn m9gd.2xlarge · local NVMe · 32 GB 4m 41s $0.0163
  3. Warpbuild 32 GB 6m 57s $0.0834
  4. Blacksmith 32 GB 7m 43s $0.0772
  5. Ubicloud standard · 32 GB 11m 38s $0.0582
  6. Avrea 32 GB 3m 59s $0.1912
  7. Namespace 8x16 · 16 GB 11m 02s $0.1323
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 5m 22s $0.8587
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Each row's best-value machine of this size for that build (lowest time × cost, ranked on p90), in that order, named under the provider. Time is the median whole job, cost is that job at the provider's per-minute price (RunsOn at EC2 spot). Yellow marks the fastest and the cheapest. Apple silicon and tmpfs (workspace in RAM) machines are rows of their own; a faded row is provisional or a machine of another size, and takes no place. A row that did not finish a build stays listed with the reason.

Full build benchmark every phase, p90 and worst runs, queue under a burst, run-to-run spread, bigger runners

CPU

CPUsingle-thread PassMarkhigher is betterx64 · 2 vCPU Share Save image

Namespace and Avrea tie for the fastest core.

RunsOn is 3rd of 9, 7% below the leader. Namespace to GitHub: 2.0× apart.

  1. Namespace 2x8 · 8 GB 4,543
  2. Avrea 8 GB 4,358
  3. RunsOn m8azn.large · EBS gp3 · 8 GB 4,234
  4. Warpbuild 8 GB 3,936
  5. RunsOntmpfs r8a.large · 16 GB 3,797
  6. Blacksmith 8 GB 3,725
  7. Ubicloud standard · 8 GB 2,866
  8. AWS CodeBuild small · 3 GB 2,602
  9. boat 4 GB 2,302
  10. GitHub 8 GB 2,243
  11. Sengi premium · 8 GB 3,705 provisional
  12. NamespaceApple silicon arm64 only
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 4 vCPU Share Save image

RunsOn, Namespace and Avrea tie for the fastest core.

Namespace to GitHub: 1.9× apart.

  1. Namespace 4x16 · 16 GB 4,441
  2. Avrea 16 GB 4,401
  3. RunsOn m8azn.xlarge · gp3 1000 MiB/s · 16 GB 4,255
  4. Warpbuild 16 GB 4,081
  5. RunsOntmpfs r8a.xlarge · 32 GB 3,802
  6. Blacksmith 16 GB 3,614
  7. StarSling 16 GB 3,295
  8. Ubicloud premium · 16 GB 3,146
  9. boat 8 GB 3,015
  10. AWS CodeBuild medium · 8 GB 2,621
  11. GitHub 16 GB 2,365
  12. Sengi premium · 16 GB too few days 3,724 provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterx64 · 8 vCPU Share Save image

Namespace and Avrea tie for the fastest core.

RunsOn is 4th of 10, 17% below the leader. Namespace to GitHub: 1.9× apart.

  1. Namespace 8x32 · 32 GB 4,562
  2. Avrea 32 GB 4,434
  3. Warpbuild 32 GB 4,040
  4. RunsOntmpfs r8a.2xlarge · 64 GB 3,798
  5. RunsOn m8a.2xlarge · EBS gp3 · 32 GB 3,740
  6. Blacksmith 32 GB 3,665
  7. Ubicloud premium · 32 GB 3,616
  8. StarSling 32 GB 3,412
  9. boat 16 GB 2,880
  10. AWS CodeBuild large · 16 GB 2,626
  11. GitHub 32 GB 2,350
  12. Sengi premium · 32 GB too few days 3,745 provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 2 vCPU Share Save image

Avrea and Namespace Apple silicon tie for the fastest core.

RunsOn is 3rd of 7, 48% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 8 GB 4,744
  2. NamespaceApple silicon 2x8 · 8 GB 4,594
  3. RunsOn m9gd.large · local NVMe · 8 GB 2,475
  4. RunsOntmpfs r9g.large · 16 GB 2,466
  5. Warpbuild 8 GB 1,939
  6. GitHub 8 GB 1,869
  7. Blacksmith 6 GB 1,330
  8. Ubicloud standard · 8 GB 1,297
  9. Namespace 2x8 · 8 GB 1,146
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 4 vCPU Share Save image

Avrea and Namespace Apple silicon tie for the fastest core.

RunsOn is 3rd of 7, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 16 GB 4,816
  2. NamespaceApple silicon 4x16 · 16 GB 4,584
  3. RunsOn m9gd.xlarge · local NVMe · 16 GB 2,472
  4. RunsOntmpfs r9g.xlarge · 32 GB 2,467
  5. Warpbuild 16 GB 1,940
  6. GitHub 16 GB 1,870
  7. Blacksmith 16 GB 1,340
  8. Ubicloud standard · 16 GB 1,286
  9. Namespace 4x16 · 16 GB 1,162
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 3,006 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

CPUsingle-thread PassMarkhigher is betterarm64 · 8 vCPU Share Save image

Avrea has the fastest core.

RunsOn is 2nd of 6, 49% below the leader. Avrea to Namespace: 4.1× apart.

  1. Avrea 32 GB 4,815
  2. RunsOn m9g.2xlarge · EBS gp3 · 32 GB 2,472
  3. RunsOntmpfs m9g.2xlarge · 32 GB 2,471
  4. Warpbuild 32 GB 1,940
  5. Blacksmith 32 GB 1,344
  6. Ubicloud standard · 32 GB 1,298
  7. Namespace 8x16 · 16 GB 1,166
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 3,145 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

The figure is the score a runner's jobs reach 9 times out of 10 (p10), so a label that sometimes lands on an older CPU ranks below one that never does. Over the bar: the lowest and the highest job (the line), 9 in 10 to 1 in 10 (the box) and the median (the tick). A long box or line is a CPU lottery. The trend is the daily median, on a scale of at least ±10%, so only a real change shows as a slope.

Full CPU benchmark

Price

Priceper minutelower is betterx64 · 2 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 10, the leader is 2.5× lower. boat to Namespace: 20× apart.

  1. boat 4 GB $0.0003
  2. RunsOn m8a.large · EBS gp3 · 8 GB $0.0008 $0.0023 on-demand
  3. RunsOntmpfs r8a.large · 16 GB $0.0008 $0.0030 on-demand
  4. Sengi basic · 8 GB $0.0010
  5. Ubicloud standard · 8 GB $0.0013
  6. Avrea 8 GB $0.0040
  7. Blacksmith 8 GB $0.0040
  8. Warpbuild 8 GB $0.0040
  9. AWS CodeBuild small · 3 GB $0.0050
  10. GitHub 8 GB $0.0060
  11. Namespace 2x8 · 8 GB $0.0060
  12. NamespaceApple silicon arm64 only
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 4 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 11, 94% above the leader. boat to Namespace: 20× apart.

  1. boat 8 GB $0.0006
  2. RunsOn m8i-flex.xlarge · EBS gp3 · 16 GB $0.0012 $0.0037 on-demand
  3. RunsOntmpfs r8a.xlarge · 32 GB $0.0015 $0.0056 on-demand
  4. Sengi basic · 16 GB $0.0020
  5. Ubicloud standard · 16 GB $0.0025
  6. Avrea 16 GB $0.0080
  7. Blacksmith 16 GB $0.0080
  8. StarSling 16 GB $0.0080
  9. Warpbuild 16 GB $0.0080
  10. AWS CodeBuild medium · 8 GB $0.0100
  11. GitHub 16 GB $0.0120
  12. Namespace 4x16 · 16 GB $0.0120
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterx64 · 8 vCPU Share Save image

boat has the lowest price per minute.

RunsOn is 2nd of 11, 98% above the leader. boat to Namespace: 20× apart.

  1. boat 16 GB $0.0012
  2. RunsOn m8a.2xlarge · EBS gp3 · 32 GB $0.0024 $0.0084 on-demand
  3. RunsOntmpfs m8a.2xlarge · 32 GB $0.0024 $0.0084 on-demand
  4. Sengi basic · 32 GB $0.0040
  5. Ubicloud standard · 32 GB $0.0050
  6. Avrea 32 GB $0.0160
  7. Blacksmith 32 GB $0.0160
  8. StarSling 32 GB $0.0160
  9. Warpbuild 32 GB $0.0160
  10. AWS CodeBuild large · 16 GB $0.0200
  11. GitHub 32 GB $0.0220
  12. Namespace 8x32 · 32 GB $0.0240
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 2 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 52× apart.

  1. RunsOn m9gd.large · local NVMe · 8 GB $0.0008 $0.0024 on-demand
  2. RunsOntmpfs r9g.large · 16 GB $0.0010 $0.0024 on-demand
  3. Ubicloud standard · 8 GB $0.0013
  4. Blacksmith 6 GB $0.0025
  5. Warpbuild 8 GB $0.0030
  6. GitHub 8 GB $0.0050
  7. Namespace 2x8 · 8 GB $0.0060
  8. Avrea 8 GB $0.0120
  9. NamespaceApple silicon 2x8 · 8 GB $0.0420
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

Pay-as-you-go price of a 2 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 4 vCPU Share Save image

RunsOn has the lowest price per minute.

RunsOn to Namespace Apple silicon: 64× apart.

  1. RunsOn m9gd.xlarge · local NVMe · 16 GB $0.0013 $0.0045 on-demand
  2. RunsOntmpfs m9g.xlarge · 16 GB $0.0017 $0.0036 on-demand
  3. Ubicloud standard · 16 GB $0.0025
  4. Blacksmith 16 GB $0.0050
  5. Warpbuild 16 GB $0.0060
  6. GitHub 16 GB $0.0080
  7. Namespace 4x16 · 16 GB $0.0120
  8. Avrea 16 GB $0.0240
  9. NamespaceApple silicon 4x16 · 16 GB $0.0840
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB $0.0800 not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Pay-as-you-go price of a 4 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Priceper minutelower is betterarm64 · 8 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the lowest price per minute.

RunsOn to Avrea: 21× apart.

  1. RunsOn m9g.2xlarge · EBS gp3 · 32 GB $0.0022 $0.0068 on-demand
  2. RunsOntmpfs m9g.2xlarge · 32 GB $0.0022 $0.0068 on-demand
  3. Ubicloud standard · 32 GB $0.0050
  4. Blacksmith 32 GB $0.0100
  5. Warpbuild 32 GB $0.0120
  6. Namespace 8x32 · 32 GB $0.0240
  7. Avrea 32 GB $0.0480
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB $0.1600 not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Pay-as-you-go price of a 8 vCPU runner. RunsOn is the EC2 + EBS bill in your own AWS account, at spot (the bar) and on-demand (the outline); its license is flat, like the other providers' plan fees.

Estimate your bill

Network

Networkdownload from GitHubhigher is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn tmpfs to Ubicloud: 6.1× apart.

  1. RunsOntmpfs r8a.large · 16 GB 303 MiB/s
  2. RunsOn median of 9 machines 205 MiB/s
  3. Namespace 2x8 · 8 GB 163 MiB/s
  4. AWS CodeBuild small · 3 GB 155 MiB/s
  5. Warpbuild 8 GB 111 MiB/s
  6. Avrea 8 GB 97 MiB/s
  7. Blacksmith 8 GB 95 MiB/s
  8. boat 4 GB 72 MiB/s
  9. GitHub 8 GB 72 MiB/s
  10. Ubicloud median of 2 machines 50 MiB/s
  11. Sengi median of 2 machines 51 MiB/s provisional
  12. NamespaceApple silicon arm64 only
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Networkdownload from GitHubhigher is betterx64 · 4 vCPU Share Save image

StarSling and AWS CodeBuild tie for the fastest download from GitHub.

RunsOn is 3rd of 10, 14% below the leader. StarSling to Ubicloud: 4.0× apart.

  1. StarSling 16 GB 217 MiB/s
  2. AWS CodeBuild medium · 8 GB 213 MiB/s
  3. RunsOn median of 8 machines 186 MiB/s
  4. Avrea 16 GB 163 MiB/s
  5. RunsOntmpfs r8a.xlarge · 32 GB 159 MiB/s
  6. Namespace median of 2 machines 124 MiB/s
  7. GitHub 16 GB 100 MiB/s
  8. Warpbuild 16 GB 99 MiB/s
  9. Blacksmith 16 GB 83 MiB/s
  10. boat 8 GB 61 MiB/s
  11. Ubicloud median of 2 machines 54 MiB/s
  12. Sengi median of 2 machines 82 MiB/s provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Networkdownload from GitHubhigher is betterx64 · 8 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the fastest download from GitHub.

RunsOn to Ubicloud: 3.6× apart.

  1. RunsOn median of 7 machines 178 MiB/s
  2. RunsOntmpfs median of 2 machines 172 MiB/s
  3. AWS CodeBuild large · 16 GB 161 MiB/s
  4. Avrea 32 GB 150 MiB/s
  5. Namespace median of 2 machines 137 MiB/s
  6. StarSling 32 GB 130 MiB/s
  7. GitHub 32 GB 107 MiB/s
  8. Blacksmith 32 GB 105 MiB/s
  9. Warpbuild 32 GB 101 MiB/s
  10. boat 16 GB 79 MiB/s
  11. Ubicloud median of 2 machines 50 MiB/s
  12. Sengi median of 2 machines 56 MiB/s provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Networkdownload from GitHubhigher is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn tmpfs to Ubicloud: 3.3× apart.

  1. RunsOntmpfs r9g.large · 16 GB 191 MiB/s
  2. RunsOn median of 3 machines 177 MiB/s
  3. Warpbuild 8 GB 167 MiB/s
  4. NamespaceApple silicon 2x8 · 8 GB 143 MiB/s
  5. Avrea 8 GB 141 MiB/s
  6. Namespace 2x8 · 8 GB 122 MiB/s
  7. GitHub 8 GB 120 MiB/s
  8. Blacksmith 6 GB 92 MiB/s
  9. Ubicloud standard · 8 GB 58 MiB/s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Networkdownload from GitHubhigher is betterarm64 · 4 vCPU Share Save image

RunsOn and RunsOn tmpfs tie for the fastest download from GitHub.

RunsOn to Ubicloud: 2.3× apart.

  1. RunsOn median of 3 machines 172 MiB/s
  2. RunsOntmpfs median of 2 machines 167 MiB/s
  3. Warpbuild 16 GB 158 MiB/s
  4. NamespaceApple silicon 4x16 · 16 GB 127 MiB/s
  5. Namespace median of 2 machines 125 MiB/s
  6. GitHub 16 GB 118 MiB/s
  7. Blacksmith 16 GB 97 MiB/s
  8. Avrea 16 GB 86 MiB/s
  9. Ubicloud standard · 16 GB 74 MiB/s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 68 MiB/s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Networkdownload from GitHubhigher is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs downloads from GitHub fastest.

RunsOn tmpfs to Ubicloud: 4.4× apart.

  1. RunsOntmpfs m9g.2xlarge · 32 GB 253 MiB/s
  2. Warpbuild 32 GB 198 MiB/s
  3. RunsOn median of 3 machines 161 MiB/s
  4. Namespace median of 2 machines 134 MiB/s
  5. Blacksmith 32 GB 101 MiB/s
  6. Avrea 32 GB 84 MiB/s
  7. Ubicloud standard · 32 GB 57 MiB/s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 70 MiB/s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

A large file fetched from GitHub inside the job. Downloads vary from machine to machine, so a bar is the median of the line's machines of this size and the mark over it runs from its slowest to its fastest.

Full network benchmark

Disk

Diskrandom reads per secondhigher is betterx64 · 2 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 5th of 9, the leader is 8.4× higher. Namespace to AWS CodeBuild: 60× apart.

  1. Namespace 2x8 · 8 GB 105,054
  2. Blacksmith 8 GB 71,426
  3. Avrea 8 GB 53,598
  4. Warpbuild 8 GB 24,481
  5. RunsOn i7i.large · local NVMe · 16 GB 12,510
  6. boat 4 GB 10,783
  7. Ubicloud standard · 8 GB 6,889
  8. GitHub 8 GB 6,868
  9. AWS CodeBuild small · 3 GB 1,763
  10. Sengi premium · 8 GB 13,893 provisional
  11. NamespaceApple silicon arm64 only
  12. RunsOntmpfs workspace in RAM, no disk
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 4 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 6th of 10, the leader is 8.8× higher. Namespace to AWS CodeBuild: 58× apart.

  1. Namespace 4x8 · 8 GB 104,646
  2. Blacksmith 16 GB 82,000
  3. Warpbuild 16 GB 63,558
  4. Avrea 16 GB 49,550
  5. StarSling 16 GB 32,038
  6. RunsOn i7i.xlarge · local NVMe · 32 GB 11,950
  7. boat 8 GB 10,338
  8. Ubicloud premium · 16 GB 7,801
  9. GitHub 16 GB 7,332
  10. AWS CodeBuild medium · 8 GB 1,794
  11. Sengi premium · 16 GB 13,632 provisional
  12. NamespaceApple silicon arm64 only
  13. RunsOntmpfs workspace in RAM, no disk
  14. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterx64 · 8 vCPU Share Save image

Namespace has the fastest disk on small reads.

RunsOn is 7th of 10, the leader is 9.6× higher. Namespace to AWS CodeBuild: 60× apart.

  1. Namespace 8x32 · 32 GB 106,228
  2. Blacksmith 32 GB 74,095
  3. Avrea 32 GB 52,121
  4. StarSling 32 GB 52,058
  5. Warpbuild 32 GB 37,153
  6. boat 16 GB 11,383
  7. RunsOn m8id.2xlarge · local NVMe · 32 GB 11,083
  8. Ubicloud premium · 32 GB 9,543
  9. GitHub 32 GB 7,559
  10. AWS CodeBuild large · 16 GB 1,758
  11. Sengi premium · 32 GB 13,912 provisional
  12. NamespaceApple silicon arm64 only
  13. RunsOntmpfs workspace in RAM, no disk
  14. WarpbuildApple silicon arm64 only

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 2 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 4th of 7, the leader is 2.3× higher. Blacksmith to Warpbuild: 17× apart.

  1. Blacksmith 6 GB 31,028
  2. Avrea 8 GB 20,909
  3. NamespaceApple silicon 2x8 · 8 GB 19,829
  4. Namespace 2x8 · 8 GB 15,941
  5. RunsOn m9gd.large · local NVMe · 8 GB 13,533
  6. Ubicloud standard · 8 GB 8,452
  7. GitHub 8 GB 8,119
  8. Warpbuild 8 GB 1,779
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. RunsOntmpfs workspace in RAM, no disk
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 4 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 4th of 7, the leader is 2.4× higher. Blacksmith to Warpbuild: 18× apart.

  1. Blacksmith 16 GB 32,691
  2. Avrea 16 GB 20,301
  3. NamespaceApple silicon 4x16 · 16 GB 18,570
  4. Namespace 4x16 · 16 GB 16,759
  5. RunsOn m9gd.xlarge · local NVMe · 16 GB 13,368
  6. Ubicloud standard · 16 GB 8,334
  7. GitHub 16 GB 7,694
  8. Warpbuild 16 GB 1,812
  9. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB 29,080 not ranked
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Diskrandom reads per secondhigher is betterarm64 · 8 vCPU Share Save image

Blacksmith has the fastest disk on small reads.

RunsOn is 4th of 6, the leader is 2.4× higher. Blacksmith to Warpbuild: 18× apart.

  1. Blacksmith 32 GB 31,905
  2. Avrea 32 GB 20,397
  3. Namespace 8x16 · 16 GB 17,452
  4. RunsOn m9gd.2xlarge · local NVMe · 32 GB 13,462
  5. Ubicloud standard · 32 GB 8,453
  6. Warpbuild 32 GB 1,781
  7. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB 30,102 not ranked
  8. AWS CodeBuild no arm64 runner
  9. boat no arm64 runner
  10. GitHub no 8 vCPU arm64 runner
  11. NamespaceApple silicon no 8 vCPU arm64 runner
  12. RunsOntmpfs workspace in RAM, no disk
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

4k random reads one at a time (fio, QD1) in the job's workspace, each provider's best disk. It is a disk's limit: a build often never reaches it, so read it next to the builds. A tmpfs workspace is RAM and has no disk figure.

Full disk benchmark

Cache

Cacherestoring 4 GBlower is betterx64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to boat: 13× apart.

  1. RunsOntmpfs r8a.large · actions/cache 7s
  2. Blacksmith actions/cache 9s
  3. Avrea actions/cache 12s
  4. RunsOn r8a.large · EBS gp3 · actions/cache 17s
  5. Namespace 2x8 · actions/cache 21s
  6. Warpbuild actions/cache 26s
  7. AWS CodeBuild small · actions/cache 52s
  8. Ubicloud standard · actions/cache 59s
  9. GitHub 2-core · actions/cache 1m 00s
  10. boat actions/cache 1m 32s
  11. NamespaceApple silicon arm64 only
  12. Sengi no cache result
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to boat: 19× apart.

  1. RunsOntmpfs r8a.xlarge · actions/cache 5s
  2. Avrea actions/cache 10s
  3. Blacksmith actions/cache 10s
  4. RunsOn m8a.xlarge · EBS gp3 · actions/cache 16s
  5. Namespace 4x8 · actions/cache 18s
  6. Warpbuild actions/cache 21s
  7. StarSling actions/cache 23s
  8. AWS CodeBuild medium · actions/cache 35s
  9. GitHub 4-core · actions/cache 38s
  10. Ubicloud premium · actions/cache 56s
  11. boat actions/cache 1m 36s
  12. Sengi premium · actions/cache 7s provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterx64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to boat: 16× apart.

  1. RunsOntmpfs m8a.2xlarge · actions/cache 5s
  2. Blacksmith actions/cache 8s
  3. Avrea actions/cache 10s
  4. RunsOn m8a.2xlarge · EBS gp3 · actions/cache 11s
  5. Namespace 8x16 · actions/cache 18s
  6. Warpbuild actions/cache 23s
  7. StarSling actions/cache 24s
  8. GitHub 8-core · actions/cache 33s
  9. AWS CodeBuild large · actions/cache 35s
  10. Ubicloud standard · actions/cache 55s
  11. boat actions/cache 1m 11s
  12. Sengi premium · actions/cache 6s provisional
  13. NamespaceApple silicon arm64 only
  14. WarpbuildApple silicon arm64 only

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 2 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to Ubicloud: 8.0× apart.

  1. RunsOntmpfs r9g.large · actions/cache 8s
  2. Avrea actions/cache 12s
  3. RunsOn m9g.large · EBS gp3 · actions/cache 19s
  4. Blacksmith actions/cache 22s
  5. NamespaceApple silicon 2x8 · actions/cache 24s
  6. Warpbuild actions/cache 28s
  7. Namespace 2x8 · actions/cache 43s
  8. GitHub 2-core · actions/cache 53s
  9. Ubicloud standard · actions/cache 1m 04s
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 4 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to Ubicloud: 12× apart.

  1. RunsOntmpfs r9g.xlarge · actions/cache 5s
  2. Avrea actions/cache 10s
  3. RunsOn m9g.xlarge · EBS gp3 · actions/cache 16s
  4. Blacksmith actions/cache 20s
  5. NamespaceApple silicon 4x16 · actions/cache 21s
  6. Warpbuild actions/cache 27s
  7. GitHub 4-core · actions/cache 37s
  8. Namespace 4x8 · actions/cache 39s
  9. Ubicloud standard · actions/cache 1m 01s
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · actions/cache 37s not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Cacherestoring 4 GBlower is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs restores a 4 GB cache fastest.

RunsOn tmpfs to Ubicloud: 13× apart.

  1. RunsOntmpfs m9g.2xlarge · actions/cache 4s
  2. Avrea actions/cache 10s
  3. RunsOn m9g.2xlarge · EBS gp3 · actions/cache 10s
  4. Blacksmith actions/cache 17s
  5. Warpbuild actions/cache 21s
  6. Namespace 8x16 · actions/cache 34s
  7. Ubicloud standard · actions/cache 50s
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · WarpBuilds/cache 1m 06s not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

Restore of a 4 GB cache: each provider's fastest cache action on its standard storage, named under the provider; the tmpfs row restores into RAM.

Full cache benchmark

Predictability

Predictabilityrun to runlower is betterx64 · 2 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 2nd of 6, at +9%.

  1. Avrea 8 GB · worst run 1.1× +3%
  2. RunsOntmpfs r8a.large · 16 GB · worst run 1.1× +9%
  3. RunsOn 2 machines · worst run 1.2× +9%
  4. GitHub 8 GB · worst run 1.3× +14%
  5. Warpbuild 8 GB · worst run 1.3× +18%
  6. Ubicloud standard · 8 GB · worst run 1.4× +20%
  7. Blacksmith 8 GB · worst run 1.7× +31%
  8. AWS CodeBuild small · 3 GB · worst run 1.1× +8% provisional
  9. boat 4 GB · worst run 1.2× +17% provisional
  10. Namespace 2x8 · 8 GB · worst run 1.3× +25% provisional
  11. NamespaceApple silicon arm64 only
  12. Sengi under 3 runs of any build
  13. StarSling no 2 vCPU x64 runner
  14. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 4 vCPU Share Save image

StarSling has the steadiest build times.

RunsOn is 2nd of 9, at +3%.

  1. StarSling 16 GB · worst run 1.0× +2%
  2. RunsOntmpfs r8a.xlarge · 32 GB · worst run 1.0× +3%
  3. RunsOn 2 machines · worst run 1.1× +3%
  4. Avrea 16 GB · worst run 1.1× +3%
  5. Warpbuild 16 GB · worst run 1.3× +8%
  6. Namespace 2 machines · worst run 1.4× +12%
  7. GitHub 16 GB · worst run 1.2× +14%
  8. Ubicloud standard · 16 GB · worst run 1.3× +16%
  9. Blacksmith 16 GB · worst run 2.8× +44%
  10. boat 8 GB · worst run 3.6× +117%
  11. AWS CodeBuild medium · 8 GB · worst run 1.1× +6% provisional
  12. NamespaceApple silicon arm64 only
  13. Sengi under 3 runs of any build
  14. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterx64 · 8 vCPU Share Save image

Avrea has the steadiest build times.

RunsOn is 3rd of 10, at +5%.

  1. Avrea 32 GB · worst run 1.1× +2%
  2. AWS CodeBuild large · 16 GB · worst run 1.1× +3%
  3. RunsOntmpfs m8a.2xlarge · 32 GB · worst run 1.2× +5%
  4. Warpbuild 32 GB · worst run 1.2× +5%
  5. RunsOn 2 machines · worst run 1.3× +6%
  6. StarSling 32 GB · worst run 1.2× +11%
  7. GitHub 32 GB · worst run 1.2× +13%
  8. Namespace 2 machines · worst run 1.6× +14%
  9. Ubicloud premium · 32 GB · worst run 1.4× +16%
  10. Blacksmith 32 GB · worst run 1.9× +31%
  11. boat 16 GB · worst run 6.1× +197%
  12. NamespaceApple silicon arm64 only
  13. Sengi under 3 runs of any build
  14. WarpbuildApple silicon arm64 only

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 2 vCPU Share Save image

RunsOn and Blacksmith tie for the steadiest build times.

  1. RunsOn m9gd.large · local NVMe · 8 GB · worst run 1.1× +5%
  2. Blacksmith 6 GB · worst run 1.1× +5%
  3. Avrea 8 GB · worst run 1.1× +10%
  4. Ubicloud standard · 8 GB · worst run 1.2× +11%
  5. GitHub 8 GB · worst run 1.3× +13%
  6. RunsOntmpfs r9g.large · 16 GB · worst run 1.6× +23%
  7. Warpbuild 8 GB · worst run 2.0× +39%
  8. Namespace 2x8 · 8 GB · worst run 1.2× +8% provisional
  9. NamespaceApple silicon 2x8 · 8 GB · worst run 1.2× +12% provisional
  10. AWS CodeBuild no arm64 runner
  11. boat no arm64 runner
  12. Sengi no arm64 runner
  13. StarSling no arm64 runner
  14. WarpbuildApple silicon no 2 vCPU arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 4 vCPU Share Save image

RunsOn has the steadiest build times.

  1. RunsOn 2 machines · worst run 1.0× +2%
  2. Blacksmith 16 GB · worst run 1.1× +3%
  3. Warpbuild 16 GB · worst run 1.1× +4%
  4. RunsOntmpfs 2 machines · worst run 1.1× +4%
  5. GitHub 16 GB · worst run 1.1× +6%
  6. Avrea 16 GB · worst run 1.2× +10%
  7. Ubicloud standard · 16 GB · worst run 1.2× +12%
  8. Namespace 2 machines · worst run 1.3× +15%
  9. NamespaceApple silicon 4x16 · 16 GB · worst run 2.2× +31%
  10. WarpbuildApple silicon 6 vCPU · xfast (Apple M4 Pro) · 14 GB · worst run 2.2× +37% not ranked
  11. AWS CodeBuild no arm64 runner
  12. boat no arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Predictabilityrun to runlower is betterarm64 · 8 vCPU Share Save image

RunsOn tmpfs and Blacksmith tie for the steadiest build times.

  1. RunsOntmpfs m9g.2xlarge · 32 GB · worst run 1.2× +5%
  2. Blacksmith 32 GB · worst run 1.1× +5%
  3. RunsOn 2 machines · worst run 1.1× +6%
  4. Warpbuild 32 GB · worst run 1.1× +6%
  5. Ubicloud standard · 32 GB · worst run 1.3× +12%
  6. Avrea 32 GB · worst run 1.3× +15%
  7. Namespace 2 machines · worst run 1.8× +16%
  8. WarpbuildApple silicon 12 vCPU · xfast (Apple M4 Pro) · 28 GB · worst run 2.6× +31% not ranked
  9. AWS CodeBuild no arm64 runner
  10. boat no arm64 runner
  11. GitHub no 8 vCPU arm64 runner
  12. NamespaceApple silicon no 8 vCPU arm64 runner
  13. Sengi no arm64 runner
  14. StarSling no arm64 runner

How much a build's time moves from run to run on the same machine: the run that 9 runs in 10 beat (p90), over the typical run (p50), for each build on the machine Real builds picks, averaged over the builds with 3 runs or more. +10% means one run in ten took more than 10% longer than usual. A line measured on one build is shown, not ranked.

Run to run, build by build

Every benchmark

  • Explorer Every benchmark, provider and runner in one place: pick a benchmark, open a provider, compare its runners.
  • Builds in detail Every build phase, p50 to max, by best value, time or cost.
  • Queue How long jobs wait for a runner in a burst, and how far build times move from run to run.
  • Bigger runners What doubling the vCPU does to a build's time and cost.
  • All runners All 102 runners in one table, and the price / performance chart.
  • CPU Single and multi-thread scores, job-to-job spread, score per dollar.
  • Cache Save and restore of a 4 GB cache, per cache action and storage.
  • Disk I/O fio in the job's workspace: random and sequential, latency, sync writes.
  • EC2 storage NVMe, gp3 and tmpfs on the same instances, and what each does to a build.
  • EC2 CPU Every EC2 instance family by CPU score and price.
  • Head-to-heads Two runners of the same size, build for build, cost first.
  • Methodology How every figure is measured and ranked, and what changed.
  • Calculator Your own minutes and sizes: what each provider would bill.