Docker Hub pull-through cache

CI jobs that docker pull public images eventually hit Docker Hub’s anonymous pull-rate limit (429 Too Many Requests / toomanyrequests). Because every runner in a stack shares one NAT egress IP, a busy organization trips that limit quickly.

RunsOn can mirror Docker Hub through an ECR pull-through cache ↗ in your own AWS account. The first pull of an image fetches it from Docker Hub into ECR; every later pull — across all your runners — is served from ECR over the VPC. That removes the rate limit and speeds up pulls.

Availability#

The pull-through cache works with Fleet and Flex-on-Terraform stacks. It is not available on the CloudFormation install path. Linux runners get transparent Docker Hub mirroring and automatic ECR authentication. Windows runners can use the same cached repositories after authenticating to ECR and changing image references to explicit ECR paths.

Requirements#

Three things must be in place:

1. For Docker Hub, a Secrets Manager secret named ecr-pullthroughcache/... in the same AWS account and Region as the rule. It must contain Docker Hub credentials.
2. An ECR pull-through cache rule for the upstream registry, created in your AWS account.
3. The ecr-pull-through runner extra, so the agent logs the runner into ECR (and, for Docker Hub, configures the daemon mirror) before your job starts.

1. Create the pull-through rule#

RunsOn references existing pull-through cache rules; it does not create them (creating one needs ecr:CreatePullThroughCacheRule, which the runner role intentionally does not hold). AWS requires Docker Hub rules to use a Secrets Manager credential, even when you only pull public images. Create the secret and regional rule once, outside the RunsOn module:

variable "dockerhub_username" {
  type = string
}

variable "dockerhub_access_token" {
  type      = string
  sensitive = true
}

resource "aws_secretsmanager_secret" "docker_hub" {
  name = "ecr-pullthroughcache/docker-hub"

  # ECR requires the default aws/secretsmanager key.
}

resource "aws_secretsmanager_secret_version" "docker_hub" {
  secret_id = aws_secretsmanager_secret.docker_hub.id

  secret_string = jsonencode({
    username    = var.dockerhub_username
    accessToken = var.dockerhub_access_token
  })
}

resource "aws_ecr_pull_through_cache_rule" "docker_hub" {
  ecr_repository_prefix = "docker-hub"
  upstream_registry_url = "registry-1.docker.io"
  credential_arn        = aws_secretsmanager_secret.docker_hub.arn
}

Use a named prefix such as docker-hub. The special ROOT prefix is rejected in RunsOn v3.2.0 and later because it would grant runners access to every ECR repository in the account.¹ RunsOn maps normal Docker Hub references through the named prefix with a runner-local registry mirror, so Docker Hub remains transparent. Other upstreams work too, but you reference those images through the ECR path explicitly. ECR Public, Quay, and registry.k8s.io do not need a credential; GHCR and other authenticated upstreams need their own Secrets Manager secret.

2. Reference the rule in the RunsOn stack#

Pass the rule to the Fleet or Flex module via ecr_pull_through_cache_rules:

module "runs_on" {
  source  = "runs-on/runs-on/aws//modules/fleet"
  # ...

  ecr_pull_through_cache_rules = {
    docker_hub = aws_ecr_pull_through_cache_rule.docker_hub
  }
}

This grants runners the EcrPullThroughCacheAccess IAM policy (ecr:GetAuthorizationToken, BatchImportUpstreamImage, BatchGetImage, CreateRepository, …) so ECR can lazily create the cache repository and import the upstream image on first pull. See ecr_pull_through_cache_rules in the configuration reference.

3. Enable the extra on runners#

Add ecr-pull-through to the runner’s extras.

jobs:
  build:
    runs-on: runs-on=${{ github.run_id }}/runner=2cpu-linux-x64/extras=ecr-pull-through
    steps:
      - uses: runs-on/action@v2
      - uses: actions/checkout@v7
      - run: docker pull node:22          # transparently served from your ECR mirror
      - run: docker build .               # FROM docker.io/... images are mirrored too

# Fleet: bake it into the runner definition
runners = {
  linux-docker = {
    cpu    = 8
    ram    = 16
    family = ["c8i"]
    image  = "ubuntu24-full-x64"
    extras = ["s3-cache", "ecr-pull-through"]
  }
}

For Docker Hub, no image references change. On Linux, the agent starts a local registry mirror that maps Docker Hub paths onto your named ECR prefix, authenticates it to ECR, and adds that local mirror to Docker’s registry-mirrors. A plain docker pull node:22 or a FROM node:22 in a Dockerfile therefore goes through ECR automatically.

Environment variables#

When the extra is active on Linux, one variable is available to your job:

• RUNS_ON_ECR_PULL_THROUGH_CACHE — the ECR registry host (for example 123456789012.dkr.ecr.us-east-1.amazonaws.com). Use it to build an explicit path for a non-Docker Hub upstream: ${RUNS_ON_ECR_PULL_THROUGH_CACHE}/<ecr_repository_prefix>/<image>.

For example, a GHCR rule with prefix ghcr lets a Linux workflow pull ${RUNS_ON_ECR_PULL_THROUGH_CACHE}/ghcr/my-org/my-tool:v2 after the extra signs Docker into ECR.

Limitations#

• Docker Hub credentials. AWS requires a same-account, same-Region Secrets Manager secret for every Docker Hub pull-through rule, including public images. Name it ecr-pullthroughcache/... and attach its ARN as credential_arn when you create the rule. RunsOn authenticates the runner to your ECR, not directly to Docker Hub.
• One transparent Docker Hub rule. Configure at most one Docker Hub rule without an upstream_repository_prefix. Additional Docker Hub rules must scope an upstream prefix and use explicit ECR image paths.
• BuildKit. The agent writes a default BuildKit configuration when it does not already exist. It covers docker-container Buildx builders that do not supply an explicit configuration. Builders with an explicit configuration must include the same Docker Hub mirror; remote-driver builders must remove $HOME/.docker/buildx/buildkitd.default.toml before creation because their daemon runs elsewhere.
• Windows. Windows runners do not receive automatic ECR Docker credentials or transparent Docker Hub mirroring. Authenticate and use explicit ECR paths in the workflow instead.
• No automatic expiry. Cached upstream images accumulate in the ECR cache repositories. Add your own ECR lifecycle policy ↗ if you want them pruned.
• Private networking. If runners launch in private subnets, they need a route to ECR (ECR + S3 endpoints, via NAT or interface VPC endpoints) the same as any other ECR access.