Fastest GitHub Actions runners: disk I/O.
fio inside the job workspace of 102 runners from 11 providers, 2 to 12 vCPU. These are the disk's limits. A build goes through the Linux page cache and often never reaches them.
How this shows up in real builds EC2 storage: NVMe, gp3 and tmpfs on the same instances
Latency per provider
Each provider's best disk, RunsOn once per storage (the Storage pill). QD1 is one 4k read at a time: what a database or a cold read pays. Most build steps batch their I/O through the page cache. tmpfs is RAM, not a disk: add it to see the gap on the same scale.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 QD1 read: Namespace (106,228), ahead of Blacksmith (82,000).tmpfs for scale: 1,150,238 on RunsOn r8a.2xlarge · tmpfs.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 synced writes: Namespace (31,543), ahead of StarSling (24,988).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 2 vCPU QD1 read: Namespace (105,054), ahead of Blacksmith (71,426).tmpfs for scale: 1,150,238 on RunsOn r8a.2xlarge · tmpfs.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 2 vCPU synced writes: Namespace (31,543), ahead of Blacksmith (16,231).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 4 vCPU QD1 read: Namespace (104,646), ahead of Blacksmith (82,000).tmpfs for scale: 1,150,238 on RunsOn r8a.2xlarge · tmpfs.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 4 vCPU synced writes: Namespace (29,746), ahead of StarSling (24,988).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 8 vCPU QD1 read: Namespace (106,228), ahead of Blacksmith (74,095).tmpfs for scale: 1,150,238 on RunsOn r8a.2xlarge · tmpfs.
Faded: provisional, under 3 runs, not ranked.
Fastest x64 8 vCPU synced writes: Namespace (28,970), ahead of Blacksmith (18,082).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Fastest arm64 QD1 read: Blacksmith (32,691), ahead of Warpbuild (30,102).tmpfs for scale: 667,201 on RunsOn m9g.xlarge · tmpfs.
Fastest arm64 synced writes: RunsOn m9gd.2xlarge · NVMe (17,361), ahead of Blacksmith (9,981).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Fastest arm64 2 vCPU QD1 read: Blacksmith (31,028), ahead of Avrea (20,909).tmpfs for scale: 667,201 on RunsOn m9g.xlarge · tmpfs.
Fastest arm64 2 vCPU synced writes: Blacksmith (8,442), ahead of RunsOn m9gd.large · NVMe (7,267).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Fastest arm64 4 vCPU QD1 read: Blacksmith (32,691), ahead of Avrea (20,301).tmpfs for scale: 667,201 on RunsOn m9g.xlarge · tmpfs.
Fastest arm64 4 vCPU synced writes: RunsOn m9gd.xlarge · NVMe (14,867), ahead of Blacksmith (9,201).volatile and nobarrier mounts make each flush cheaper by making it less durable.
Fastest arm64 8 vCPU QD1 read: Blacksmith (31,905), ahead of Avrea (20,397).tmpfs for scale: 667,201 on RunsOn m9g.xlarge · tmpfs.
Fastest arm64 8 vCPU synced writes: RunsOn m9gd.2xlarge · NVMe (17,361), ahead of Blacksmith (9,981).volatile and nobarrier mounts make each flush cheaper by making it less durable.
What the numbers say
Extremes on x64, and RAM against disk.
- On x64, one small read at a time (4k, QD1) comes back 106,228 times a second on Namespace 8x32 and 1,511 on RunsOn m8a.2xlarge · EBS gp3, 70× apart; p99 latency is 10 µs against 930 µs.
- Synced 4k writes on x64 run from 487 per second on RunsOn m8i.2xlarge · EBS gp3 (workspace mounted nobarrier) to 31,543 on Namespace 2x8.
- r8a.large with tmpfs ($0.0008/min at spot) answers 1,100,344 QD1 reads and 428,125 synced writes per second: 90× and 33× the best local NVMe at 2 vCPU (i7i.large, $0.0009) and 610× and 791× default EBS gp3 (c8a.large, $0.0008); its workspace is 14 GiB, from RAM. (EC2 storage suite) What that does to real builds.
More findings22: arm64, NVMe vs EBS, provisioned gp3, tmpfs, storage classes
- On arm64, one small read at a time (4k, QD1) comes back 32,691 times a second on Blacksmith 4 vCPU and 1,697 on RunsOn c8g.2xlarge · EBS gp3, 19× apart; p99 latency is 37 µs against 856 µs.
- Synced 4k writes on arm64 run from 537 per second on RunsOn c8g.large · EBS gp3 (workspace mounted nobarrier) to 17,361 on RunsOn m9gd.2xlarge · local NVMe.
- Same Xeon 6975P-C CPU, different disk: m8id.2xlarge on local NVMe answers 11,083 QD1 reads and 7,619 synced writes per second, m8i.2xlarge on EBS gp3 1,568 and 487.
- Same Xeon 6975P-C CPU, different disk: m8id.large on local NVMe answers 11,102 QD1 reads and 5,555 synced writes per second, m8i-flex.large on EBS gp3 1,774 and 528.
- Same Xeon 6975P-C CPU, different disk: m8id.xlarge on local NVMe answers 10,813 QD1 reads and 7,901 synced writes per second, m8i-flex.xlarge on EBS gp3 1,756 and 525.
- Same Neoverse-V3 CPU, different disk: m9gd.2xlarge on local NVMe answers 13,462 QD1 reads and 17,361 synced writes per second, m9g.2xlarge on EBS gp3 1,723 and 543.
- Same Neoverse-V3 CPU, different disk: m9gd.xlarge on local NVMe answers 13,368 QD1 reads and 14,867 synced writes per second, m9g.xlarge on EBS gp3 1,837 and 543.
- Same Neoverse-V3 CPU, different disk: m9gd.large on local NVMe answers 13,533 QD1 reads and 7,267 synced writes per second, m9g.large on EBS gp3 1,789 and 542.
- Provisioning gp3 at 1000 MiB/s on m8a.2xlarge takes sequential reads from 401 MiB/s to 1,001 MiB/s (2.5×); QD1 reads go (1,587 → 1,776 IOPS).
- Provisioning gp3 at 1000 MiB/s on m8azn.xlarge takes sequential reads from 401 MiB/s to 1,001 MiB/s (2.5×); QD1 reads barely move (1,769 → 1,771 IOPS): it buys bandwidth, not latency.
- tmpfs on m8a.2xlarge is not a disk at all: 1,137,009 QD1 reads and 428,937 synced writes per second, from RAM. The workspace is 29 GiB, sized by memory.
- tmpfs on m9g.2xlarge is not a disk at all: 653,126 QD1 reads and 300,592 synced writes per second, from RAM. The workspace is 29 GiB, sized by memory.
- tmpfs on m9g.xlarge is not a disk at all: 667,201 QD1 reads and 303,876 synced writes per second, from RAM. The workspace is 14 GiB, sized by memory.
- tmpfs on r8a.2xlarge is not a disk at all: 1,150,238 QD1 reads and 430,608 synced writes per second, from RAM. The workspace is 59 GiB, sized by memory.
- tmpfs on r8a.xlarge is not a disk at all: 1,083,747 QD1 reads and 433,705 synced writes per second, from RAM. The workspace is 29 GiB, sized by memory.
- tmpfs on r9g.large is not a disk at all: 651,023 QD1 reads and 298,146 synced writes per second, from RAM. The workspace is 14 GiB, sized by memory.
- tmpfs on r9g.xlarge is not a disk at all: 644,142 QD1 reads and 301,942 synced writes per second, from RAM. The workspace is 29 GiB, sized by memory.
- tmpfs: 644k–1.2M QD1 reads and 298k–434k synced writes per second across 8 runners (8 RunsOn configurations).
- VM disk: 5.3k–106k QD1 reads and 1.4k–32k synced writes per second across 51 runners from 9 providers (6 Avrea runners, 6 Blacksmith runners, 3 boat x64 runners, 3 GitHub runners, 12 Namespace runners, 6 Sengi x64 runners, 2 StarSling x64 runners, 9 Ubicloud runners and 4 Warpbuild runners).
- local NVMe: 11k–14k QD1 reads and 5.6k–17k synced writes per second across 9 runners (9 RunsOn configurations).
- network disk: 7.3k–7.7k QD1 reads and 3.5k–4.2k synced writes per second across 2 runners (2 GitHub runners).
- EBS: 1.5k–37k QD1 reads and 487–3.0k synced writes per second across 32 runners from 3 providers (3 AWS CodeBuild x64 runners, 25 RunsOn configurations and 4 Warpbuild configurations).
Reading these numbers
fio measures what a disk can do. Whether a job gets there depends on the job.
What each column says about a job
- QD1 reads and synced writes: databases, SQLite-backed tests and anything that flushes each write. Every flush waits for the storage, and a network disk pays a round trip each time.
-
Random writes at depth and sequential throughput:
docker buildlayers, package installs and cache archives. They batch their writes and run into the disk's ceiling, not its latency. - None of them: compile jobs. A compiler waits for the CPU, and it reads its files back from the page cache.
Every runner
Each provider's best disk first; its other machines sit behind +N more. Bars are log scale.
| One request at a time | 4k, 32 in flight | Sequential, 1M | |||||
|---|---|---|---|---|---|---|---|
| Runner | QD1 read ↑ | p99 ↓ | Sync write ↑ | Read ↑ | Write ↑ | Read ↑ | Write ↑ |
| x64 | |||||||
| Namespace 8x32 | 10 µs | 1,096,275 | 576,011 | 5.0 GiB/s | 3.4 GiB/s | ||
| 8x16 | 10 µs | 954,329 | 543,808 | 4.6 GiB/s | 3.0 GiB/s | ||
| 4x8 | 11 µs | 1,156,765 | 575,099 | 5.0 GiB/s | 3.2 GiB/s | ||
| 4x16 | 11 µs | 1,196,382 | 553,982 | 5.0 GiB/s | 3.3 GiB/s | ||
| 2x8 | 10 µs | 839,416 | 603,279 | 4.6 GiB/s | 3.0 GiB/s | ||
| Blacksmith | 16 µs | 700,085 | 217,634 | 6.6 GiB/s | 1.6 GiB/s | ||
| 8 vCPU | 16 µs | 598,298 | 241,065 | 6.9 GiB/s | 1.8 GiB/s | ||
| 2 vCPU | 19 µs | 622,999 | 154,296 | 4.7 GiB/s | 1.7 GiB/s | ||
| Warpbuild | 22 µs | 187,737 | 70,031 | 4.1 GiB/s | 2.7 GiB/s | ||
| 8 vCPU | 29 µs | 188,027 | 67,698 | 4.3 GiB/s | 2.8 GiB/s | ||
| 2 vCPU | 45 µs | 232,952 | 88,324 | 4.3 GiB/s | 2.9 GiB/s | ||
| Avrea | 14 µs | 583,528 | 228,550 | 16.2 GiB/s | 10.8 GiB/s | ||
| 8 vCPU | 17 µs | 317,363 | 198,055 | 17.0 GiB/s | 11.1 GiB/s | ||
| 4 vCPU | 21 µs | 456,155 | 219,963 | 16.0 GiB/s | 10.8 GiB/s | ||
| StarSling | 22 µs | 296,735 | 48,228 | 2.4 GiB/s | 3.3 GiB/s | ||
| 4 vCPU | 30 µs | 342,267 | 69,942 | 4.9 GiB/s | 4.1 GiB/s | ||
| Sengi premium | 81 µs | 353,085 | 154,538 | 3.1 GiB/s | 1.4 GiB/s | ||
| basic 8 vCPU | 235 µs | 465,853 | 67,027 | 3.3 GiB/s | 2.5 GiB/s | ||
| premium 4 vCPU | 111 µs | 320,646 | 144,114 | 4.4 GiB/s | 2.6 GiB/s | ||
| basic 4 vCPU | 627 µs | 253,419 | 36,902 | 3.1 GiB/s | 2.1 GiB/s | ||
| premium 2 vCPU | 84 µs | 321,467 | 152,642 | 2.8 GiB/s | 1.2 GiB/s | ||
| basic 2 vCPU | 334 µs | 145,581 | 103,680 | 2.9 GiB/s | 2.3 GiB/s | ||
| RunsOn i7i.large | 293 µs | 75,023 | 41,229 | 503 MiB/s | 395 MiB/s | ||
| m8azn.3xlarge | 840 µs | 2,997 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| c8a.2xlarge | 897 µs | 2,995 | 2,991 | 401 MiB/s | 401 MiB/s | ||
| r8a.2xlarge | <1 µs | 4,286,650 | 3,818,921 | 18.4 GiB/s | 10.0 GiB/s | ||
| m8a.2xlarge | 930 µs | 2,995 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8a.2xlarge | 782 µs | 15,995 | 15,992 | 1,001 MiB/s | 1,001 MiB/s | ||
| m8a.2xlarge | <1 µs | 4,245,514 | 3,822,441 | 18.3 GiB/s | 10.0 GiB/s | ||
| m8azn.xlarge | 872 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8azn.xlarge | 823 µs | 15,994 | 15,991 | 1,001 MiB/s | 1,001 MiB/s | ||
| m8i.2xlarge | 1,004 µs | 2,993 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| m8id.2xlarge | 230 µs | 134,258 | 67,076 | 905 MiB/s | 432 MiB/s | ||
| m8i-flex.2xlarge | 905 µs | 2,993 | 2,995 | 401 MiB/s | 401 MiB/s | ||
| c8a.xlarge | 938 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8a.xlarge | 799 µs | 2,994 | 2,991 | 401 MiB/s | 401 MiB/s | ||
| i7i.2xlarge | 344 µs | 300,466 | 165,109 | 2.0 GiB/s | 1.5 GiB/s | ||
| r8a.xlarge | <1 µs | 4,126,468 | 3,769,822 | 18.2 GiB/s | 9.9 GiB/s | ||
| m8azn.large | 758 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8i.xlarge | 995 µs | 2,993 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| m8id.xlarge | 257 µs | 67,090 | 33,526 | 451 MiB/s | 216 MiB/s | ||
| m8i-flex.xlarge | 881 µs | 2,993 | 2,993 | 401 MiB/s | 401 MiB/s | ||
| i7i.xlarge | 210 µs | 150,053 | 82,487 | 1,010 MiB/s | 788 MiB/s | ||
| r8a.large | <1 µs | 2,115,833 | 1,927,435 | 17.0 GiB/s | 10.0 GiB/s | ||
| r8a.large | 807 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8a.large | 774 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| c8a.large | 766 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m8id.large | 185 µs | 33,541 | 16,753 | 226 MiB/s | 108 MiB/s | ||
| m8i.large | 1,004 µs | 2,993 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| m8i-flex.large | 872 µs | 2,993 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| t8i.medium | 864 µs | 3,003 | 2,998 | 401 MiB/s | 401 MiB/s | ||
| boat | 158 µs | 325,400 | 185,423 | 9.8 GiB/s | 1.0 GiB/s | ||
| 4 vCPU | 198 µs | 372,459 | 155,448 | 7.6 GiB/s | 1.3 GiB/s | ||
| 2 vCPU | 183 µs | 456,909 | 165,954 | 10.2 GiB/s | 892 MiB/s | ||
| Ubicloud premium | 338 µs | 330,381 | 21,612 | 2.9 GiB/s | 1.2 GiB/s | ||
| standard 8 vCPU | 856 µs | 255,065 | 12,701 | 2.5 GiB/s | 1.0 GiB/s | ||
| premium 4 vCPU | 725 µs | 224,391 | 23,539 | 2.3 GiB/s | 1.1 GiB/s | ||
| standard 4 vCPU | 1,106 µs | 163,941 | 17,458 | 2.2 GiB/s | 694 MiB/s | ||
| standard 2 vCPU | 436 µs | 151,057 | 26,808 | 2.5 GiB/s | 845 MiB/s | ||
| premium 2 vCPU | 1,352 µs | 117,486 | 13,095 | 2.4 GiB/s | 806 MiB/s | ||
| GitHub | 224 µs | 38,973 | 18,243 | 784 MiB/s | 545 MiB/s | ||
| 4-core | 226 µs | 19,574 | 10,264 | 394 MiB/s | 245 MiB/s | ||
| 2-core | 272 µs | 9,378 | 8,843 | 199 MiB/s | 133 MiB/s | ||
| AWS CodeBuild medium | 733 µs | 2,993 | 2,974 | 241 MiB/s | 251 MiB/s | ||
| large | 758 µs | 2,990 | 2,977 | 243 MiB/s | 251 MiB/s | ||
| small | 815 µs | 2,988 | 2,971 | 251 MiB/s | 251 MiB/s | ||
| arm64 | |||||||
| Blacksmith | 37 µs | 256,503 | 35,819 | 1.6 GiB/s | 786 MiB/s | ||
| 8 vCPU | 35 µs | 255,453 | 37,077 | 1.7 GiB/s | 893 MiB/s | ||
| 2 vCPU | 39 µs | 252,404 | 36,713 | 1.6 GiB/s | 806 MiB/s | ||
| Warpbuild xfast (Apple M4 Pro) | 33 µs | 383,165 | 5,949 | 11.5 GiB/s | 12.1 GiB/s | ||
| xfast (Apple M4 Pro) 6 vCPU | 64 µs | 347,190 | 6,482 | 12.6 GiB/s | 10.7 GiB/s | ||
| 8 vCPU | 864 µs | 4,995 | 4,993 | 401 MiB/s | 401 MiB/s | ||
| 4 vCPU | 848 µs | 4,194 | 4,193 | 302 MiB/s | 302 MiB/s | ||
| 2 vCPU | 799 µs | 4,194 | 4,193 | 302 MiB/s | 301 MiB/s | ||
| Avrea | 60 µs | 155,527 | 18,394 | 30.4 GiB/s | 14.6 GiB/s | ||
| 8 vCPU | 57 µs | 150,734 | 17,825 | 28.7 GiB/s | 16.0 GiB/s | ||
| 4 vCPU | 58 µs | 150,737 | 18,814 | 31.9 GiB/s | 15.8 GiB/s | ||
| Namespace Apple silicon 2x8 | 78 µs | 155,576 | 16,238 | 35.2 GiB/s | 11.8 GiB/s | ||
| Apple silicon 4x16 | 90 µs | 146,835 | 25,918 | 33.5 GiB/s | 13.4 GiB/s | ||
| 8x32 | 91 µs | 181,390 | 58,090 | 920 MiB/s | 769 MiB/s | ||
| 8x16 | 86 µs | 182,592 | 38,702 | 922 MiB/s | 700 MiB/s | ||
| 4x8 | 89 µs | 144,427 | 39,668 | 915 MiB/s | 628 MiB/s | ||
| 4x16 | 88 µs | 147,879 | 40,493 | 928 MiB/s | 657 MiB/s | ||
| 2x8 | 104 µs | 68,480 | 25,565 | 924 MiB/s | 654 MiB/s | ||
| RunsOn m9gd.large | 81 µs | 43,609 | 21,784 | 293 MiB/s | 141 MiB/s | ||
| m9gd.2xlarge | 82 µs | 174,573 | 87,209 | 1.1 GiB/s | 561 MiB/s | ||
| m9g.2xlarge | <1 µs | 2,616,178 | 2,495,107 | 13.3 GiB/s | 9.1 GiB/s | ||
| m9g.2xlarge | 954 µs | 2,995 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| c8g.2xlarge | 856 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| m9gd.xlarge | 116 µs | 87,221 | 43,592 | 585 MiB/s | 281 MiB/s | ||
| m9g.xlarge | 856 µs | 2,994 | 2,991 | 401 MiB/s | 401 MiB/s | ||
| r9g.xlarge | <1 µs | 2,581,865 | 2,478,450 | 13.7 GiB/s | 8.9 GiB/s | ||
| m9g.xlarge | <1 µs | 2,686,147 | 2,522,480 | 14.8 GiB/s | 9.3 GiB/s | ||
| c8g.xlarge | 864 µs | 2,992 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| m9g.large | 848 µs | 2,992 | 2,994 | 401 MiB/s | 401 MiB/s | ||
| r9g.large | <1 µs | 1,277,313 | 1,235,007 | 13.9 GiB/s | 9.5 GiB/s | ||
| c8g.large | 840 µs | 2,994 | 2,992 | 401 MiB/s | 401 MiB/s | ||
| Ubicloud standard | 424 µs | 223,224 | 43,151 | 763 MiB/s | 515 MiB/s | ||
| standard 4 vCPU | 498 µs | 149,499 | 30,170 | 734 MiB/s | 394 MiB/s | ||
| standard 2 vCPU | 461 µs | 84,409 | 27,725 | 746 MiB/s | 405 MiB/s | ||
| GitHub | 214 µs | 9,376 | 6,333 | 200 MiB/s | 164 MiB/s | ||
| 4-core | 228 µs | 19,575 | 12,279 | 394 MiB/s | 330 MiB/s | ||
- IOPS are 4k blocks, medians of finished jobs
- bold best disk measured on the arch (tmpfs is RAM, left out)
- , : cheaper, less durable flushes
Pick the disk per job on RunsOn
-
The instance type sets the storage under the workspace: EBS gp3 by default, provisioned gp3 for bandwidth, local NVMe
on the
dandifamilies, or tmpfs when the job fits in RAM. Pick it with thefamilylabel; each one, measured: EC2 storage benchmark. - What tmpfs does to real builds, what it costs and where it does nothing: Disks? Where we're going, we don't need disks.
How it's measured
fio runs inside the job, in GITHUB_WORKSPACE: the disk a build writes to, on the filesystem the provider
mounted there. Every runner is compared, whatever its shape.
fio settingsblock sizes, queue depths, mounts
- QD1 random read: 4k blocks, one request in flight; IOPS and p99 latency.
- Sync write: 4k writes, each followed by
fdatasync. - Random read and write: 4k blocks, 4 jobs with 32 requests in flight each.
- Sequential read and write: 1M blocks, 32 requests in flight.
- The workspace filesystem, its size and its mount options come from the job's own mount table.
- Earlier versions of this page used a different harness and other fio settings on 2 vCPU runners; those numbers are gone rather than mixed in.