AWS CodeBuild runners for GitHub Actions vs RunsOn: same AWS, older machines

AWS CodeBuild can host your GitHub Actions runners. For an AWS shop it looks like the obvious choice: no third party, one webhook, one label, and the minutes land on the AWS invoice you already pay.

We ran it through the same benchmark harness as eight other runner providers. On CodeBuild’s 4 vCPU medium size, a cold Rust build costs $0.050 and takes 4m42s. On RunsOn, in the same AWS region, an m8azn.xlarge does it for $0.0079 in 1m53s. GitHub’s own 4-core runner sits in between at $0.047 and 3m47s. Our TypeScript typecheck never finished on medium: it ran out of memory in all five attempts.

CodeBuild starts jobs about as fast as RunsOn. It loses on the machine behind each size.

On demand, you pick a size and AWS picks the machine#

Sizes and prices come from AWS’s compute types reference ↗ and the CodeBuild price list ↗.

Across almost 300 jobs, every one landed on an Intel Xeon Platinum 8124M (Skylake, 2017), or less often an 8275CL or 8223CL (Cascade Lake, 2019): the CPUs behind EC2’s c5 family. Each job runs in a Docker container on an Amazon Linux 2 host with a 4.14 kernel. No label asks for a CPU generation, a different memory ratio or spot capacity. Four vCPUs always come with 8 GB, and the only way to more memory is large, which doubles cores and price.

Cost and time per build#

The same pinned workloads ran on every runner in us-east-1. Percentages compare each runner with CodeBuild large (8 vCPU, 16 GiB), the cheapest on-demand CodeBuild size that finishes all three builds.

Cost per build

Build time

RunsOn’s 4 vCPU machines beat CodeBuild’s 8 vCPU large on every build, at 84 to 92% less per build. The gap is the CPU: large scores 8,615 on PassMark with all 8 threads, less than a 4 vCPU RunsOn c8a.xlarge (13,440), and its single-thread score (2,040) is under half the m8azn’s (4,273). The like-for-like RunsOn shape, c8a.2xlarge, builds the Rust project in 1m26s for $0.0072.

CodeBuild and GitHub costs are job time multiplied by the list rate. RunsOn costs are what the instance actually billed, as reported by the RunsOn control plane: EC2 from boot to termination plus EBS, on spot with an occasional on-demand fallback. At on-demand prices the m8azn Rust build costs $0.018, still 72% under large. RunsOn’s license is a flat yearly fee and isn’t in these figures. CodeBuild bills from build submission ↗, rounded up to the minute, so its real charges run higher than shown: a 5-minute medium Rust build bills 6 minutes.

medium runs out of memory#

The opencode typecheck was killed out of memory (exit 137) on medium in all five attempts, after 1.5 to 10 minutes that CodeBuild billed in full. The PostHog build hit fatal error: runtime: out of memory in our first runs, and the harness now builds it on large only.

8 GB is tight for this typecheck anywhere: Namespace’s 4 vCPU / 8 GB shape passed one of five attempts, RunsOn’s c8a.xlarge three of five. The difference is what you can do about it. On RunsOn, 4 vCPU with 16 GB is ram=16, and ram=32 or an r8a family goes further. On CodeBuild on demand, the next shape with more memory is large.

Starting jobs is a draw, caching is not#

We queue 15 identical jobs at the same instant and measure each one’s wait. CodeBuild started its median job after 21 seconds on both sizes; large started every job within 25 seconds, while medium had one in ten wait 81 seconds or more. RunsOn, which boots a fresh EC2 instance per job, started its median job after 17 seconds on the m8a. GitHub took about 4. RunsOn’s warm pools bring that under 10 seconds if it matters.

On a CodeBuild runner, actions/cache goes to GitHub’s cache service, like any self-hosted runner. Saving a 4 GB entry took 2m21s, the slowest save of any provider in the cache benchmark, and restoring it 42 seconds. On RunsOn the same steps go to S3 in your account through Magic Cache: 16 seconds to save and 18 to restore.

Reserved capacity fleets fix the hardware and change the bill#

A reserved capacity fleet ↗ is a set of EC2 instances CodeBuild keeps running for you. You can name an exact instance type (CUSTOM_INSTANCE_TYPE) or describe one by vCPU, memory and disk (ATTRIBUTE_BASED_COMPUTE), and a workflow routes to it with a fleet:<name> label.

The newest x86 families a fleet accepts ↗ are 7th generation: M7a, C7a, M7i, C7i, R7a and R7i. We asked for an m8a.xlarge anyway, and CodeBuild refused: The specified instance type is not supported. On ARM it stops at Graviton4.

So we ran the closest thing, a fleet of one m7a.xlarge (AMD Genoa, 4 vCPU, 16 GiB) with the same container image. Percentages here compare RunsOn with the fleet:

The fleet passes every build medium couldn’t and comes within about 15% of RunsOn. It still runs in a container on the old 4.14 kernel, and its cache still goes to GitHub, so caching stays slow.

The catch is the bill. AWS bills a fleet instance ↗ from request to termination, whether a job runs or not, with a 60-minute minimum. An m7a.xlarge slot costs $0.00696 a minute: $301 a month for one job at a time, about 1.8 times the plain EC2 on-demand rate. A slot busy every minute of a ten-hour day, 22 days a month, is still idle 69% of the time it bills, which puts it at $0.023 per busy minute, more than large on demand. Every instance a fleet scales out to bills at least an hour, and deleting one doesn’t cut that short: CodeBuild’s answer to our delete call was that “fleets are deleted after all instances have run for 1 hour”. Our test fleet cost about $0.75.

Fleet instances also persist between builds, and AWS warns that their cached data can be accessible to other projects in the same account ↗. RunsOn gives every job a fresh VM.

Setup takes about the same time on day one#

On CodeBuild, the runner tutorial ↗ has you connect GitHub once per account and region, then create a “Runner project” with a repository, an image, a compute size and an IAM service role. The console registers a webhook filtered on WORKFLOW_JOB_QUEUED, and jobs target the project by name:

jobs:
  build:
    runs-on: codebuild-myproject-${{ github.run_id }}-${{ github.run_attempt }}-ubuntu-8.0-large

The details show up later. A project listens to one repository unless you set an organization webhook ↗ scope. A mistyped project name leaves the job hanging in the queue. With multi-label syntax, jobs in one run need a unique extra label each, or one job can take another’s runner ↗. The images are CodeBuild’s own, not GitHub’s runner images. And modern hardware means a fleet, with its own capacity and scaling settings.

On RunsOn, you create one CloudFormation stack with your GitHub organization, a license key and an email, then register a private GitHub App from the stack’s entry point. Jobs describe the machine they want:

jobs:
  build:
    runs-on: runs-on=${{ github.run_id }}/cpu=4/ram=16/family=m8azn+m8a

Any EC2 instance type is available per job, spot by default with on-demand fallback, on images that replicate GitHub’s runner images. In exchange you own the stack and its upgrades, and a large burst can hit your EC2 vCPU quotas until you raise them. CodeBuild has nothing to maintain.

When CodeBuild is the right call#

Below about $30 a month of CodeBuild minutes, switching won’t pay for RunsOn’s $350-a-year license (RunsOn is free for non-commercial use). CodeBuild also fits when GitHub Actions only triggers pipelines that already live in CodeBuild and CodePipeline, when you need macOS on AWS (RunsOn doesn’t support macOS yet), or when policy only allows AWS-managed services. Procurement alone isn’t a reason: RunsOn Enterprise licenses can go through AWS Marketplace.

Most teams don’t fit those cases. Their builds take minutes, need more than 2 GB per core, and run hundreds of times a day. For them, CodeBuild on demand costs more per build than GitHub’s own runners and finishes later. RunsOn runs in the same AWS account on current AMD Turin hardware, at a sixth to a tenth of CodeBuild’s price per build.

How we measured#

Every number comes from our open benchmark harness ↗, which feeds the runner benchmarks. It runs the same pinned workloads on every provider, takes timings from the GitHub Jobs API rather than from inside the job, and records the hardware each job actually got. These results cover September 30 to October 5, 2026, in us-east-1, with five runs of each build suite per runner. The fleet ran on October 6 on a separate harness branch. Current medians are on the CodeBuild and RunsOn profiles.