GitHub 4-core: 20× the cost, 2.8× the time, 7th of 7
Best value per providerBuild time per providerCost per build per provider
Each provider's best variantEach provider's best 2 vCPU variantEach provider's best 4 vCPU variantEach provider's best 8 vCPU variant, ranked by p90 time × cost per build.; shades are build phases in order.; build time on the right.
Rustspotify-player
↓ p90 time × costwhole job (p50)timecost / buildcost
Bar shades, light to dark, in execution order: fetch sourcebuildx setupdocker buildrunner overhead
cost per build× = p90 time × cost against the best
Each provider's best-value machine (lowest p90 time × cost among its machines within 15% of its fastest of the same size) at 4 vCPU. Rust: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; TypeScript: RunsOn tested 9, Namespace 2, Ubicloud 2, the others 1; Docker: RunsOn tested 8, Ubicloud 2, the others 1.
Beyond builds
The same runners, one question each.
Queue→How long jobs wait for a runner in a burst, and how far build times move from run to run.
Bigger runners→What doubling the vCPU did to each build's time and cost, on each provider's own runner line.
Head-to-heads→Two runners of the same size, build for build, cost first.
CPU→Fastest single-thread on x64: Namespace 8x32, 4,556 PassMark.
Cache→4 GB restore on x64, any size: RunsOn 6s (12 vCPU), then Blacksmith 9s (2 vCPU); GitHub's own service 31s (8 vCPU).4 GB restore on x64 at 2 vCPU: Blacksmith 9s, then Avrea 13s; GitHub's own service 1m 01s.4 GB restore on x64 at 4 vCPU: Avrea 10s, then Blacksmith 12s; GitHub's own service 44s.4 GB restore on x64 at 8 vCPU: RunsOn 11s, then Avrea 11s; GitHub's own service 31s.
Disk→Small random reads (QD1) on x64: Namespace 8x16 leads with 107,149 per second, 60× the slowest disk (AWS CodeBuild, 1,794). That is a disk's limit, and a build often never reaches it.
EC2 storage→NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
All runners→All 94 runners in one table, with the price / performance chart.
tmpfs→The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
Queue→How long jobs wait for a runner in a burst, and how far build times move from run to run.
Bigger runners→What doubling the vCPU did to each build's time and cost, on each provider's own runner line.
Head-to-heads→Two runners of the same size, build for build, cost first.
CPU→Fastest single-thread on arm64: Avrea 4 vCPU, 4,823 PassMark.
Cache→4 GB restore on arm64, any size: Avrea 11s (8 vCPU), then RunsOn 11s (8 vCPU); GitHub's own service 36s (4 vCPU).4 GB restore on arm64 at 2 vCPU: Avrea 12s, then RunsOn 19s; GitHub's own service 53s.4 GB restore on arm64 at 4 vCPU: Avrea 11s, then RunsOn 16s; GitHub's own service 36s.4 GB restore on arm64 at 8 vCPU: Avrea 11s, then RunsOn 11s.
Disk→Small random reads (QD1) on arm64: Blacksmith 4 vCPU leads with 32,691 per second, 4.0× the slowest disk (GitHub, 8,119). That is a disk's limit, and a build often never reaches it.
EC2 storage→NVMe, gp3 and tmpfs on the same instances, and what each does to the builds above.
All runners→All 94 runners in one table, with the price / performance chart.
tmpfs→The workspace in RAM instead of on a disk: what it does to the builds above, what it costs and where it does nothing.
Providers
Price per minute of a 4 vCPU runner on x64 (its shape beside each price); RunsOn at EC2 spot and on-demand.Price per minute of a 4 vCPU runner on arm64 (its shape beside each price); RunsOn at EC2 spot and on-demand.