Speeding up docker builds with the ephemeral registry

Since v2.8.2, you can configure RunsOn to automatically create an ephemeral registry (using an ECR repository in your AWS account). This registry can be used as a temporary storage for images that are needed across your build jobs, without the need to configure an external registry by yourself.

Use cases

The ephemeral registry can be used to:

1. Push and pull images that need to be temporarily shared across workflow jobs. For instance, if you need to build an image in one job for a specific use case (e.g. integration tests), and then use it in one or many dependent jobs.
2. Cache docker layers across workflow jobs.

In my tests, docker layer caching using type=registry was slightly faster than using the S3 backend (type=s3) or the Magic Cache (type=gha).

Accessing the ephemeral registry

Runners automatically get push/pull access to this registry, with the EC2 instance profile assigned to the runners.

The RUNS_ON_ECR_CACHE environment variable is available in your steps, and contains the registry URL (e.g. 123456789012.dkr.ecr.us-east-1.amazonaws.com/runs-on/ephemeral-registry).

Note: The ephemeral registry is only available if the RunsOn stack has the Ephemeral Registry feature enabled.

Using the ephemeral registry in your workflow

You can use the ephemeral registry in your workflows by using the RUNS_ON_ECR_CACHE environment variable in your steps.

If you include ecr-cache in the extras parameter of your workflow job, the RunsOn agent will automatically log into the registry before your job starts.

For instance, the following workflow will build an image, push it to the ephemeral registry, and is configured to cache the layers in the ephemeral registry, so that future builds will be much faster:

jobs:
  ecr-cache:
    runs-on: runs-on=${{ github.run_id }}/runner=2cpu-linux-x64/extras=ecr-cache
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v4
        env:
          TAG: ${{ env.RUNS_ON_ECR_CACHE }}:my-app-latest
        with:
          context: .
          push: true
          tags: ${{ env.TAG }}
          cache-from: type=registry,ref=${{ env.TAG }}
          cache-to: type=registry,ref=${{ env.TAG }} }},mode=max

Cleanup

The registry is configured so that images are automatically cleaned up after 10 days.

Considerations

If you make heavy use of the registry, it might be a good idea to enable the VPC endpoint for ECR in your RunsOn stack. This comes at additional cost, but might be cheaper than paying for bandwidth costs in the end.

Security-wise, the registry is shared across all runners launched by the same RunsOn stack. So make sure you only store images and layers that can be shared across all runners. You can create multiple RunsOn environments if you need better isolation.

Limitations

• The ephemeral registry is only available on Linux runners.