How Magic Cache works

Every Magic Cache page relies on the same pieces: the RunsOn agent on the runner, the cache bucket of your stack, and GitHub’s cache rules. This page describes them once.

The agent on each runner#

With Magic Cache enabled, the RunsOn agent starts a cache server on the runner and points the job’s caching clients at it: actions/cache and the setup actions, the gha backend of Docker Buildx, and the build tools listed in the remote_cache input of runs-on/action.

The agent forwards every request to the S3 cache bucket of your stack, in your AWS account, and streams hits straight back. Nothing leaves your account, and there is no size cap.

Only the storage backend changes, so the same workflow keeps working on GitHub-hosted runners, where the cache points at GitHub’s backend instead. If the cache server fails to start within 5 seconds, the job uses GitHub’s cache backend and gets a warning annotation. The job does not fail.

Enabling it#

Magic Cache is opt-in. Turn it on per job (Flex) or per runner (Fleet):

runs-on:
  - runs-on=${{ github.run_id }}
  - runner=2cpu-linux-x64
  - extras=s3-cache
steps:
  - uses: runs-on/action@v2

runners = {
  linux-cache = {
    cpu    = 4
    family = ["c8i"]
    extras = ["s3-cache"]
  }
}

The git checkout and apt package caches have extras of their own (git-cache, apt-cache) and do not need s3-cache. Action downloads are cached on every runner.

Build tool caches#

Build tools cache their work by key: Bazel’s actions, Turborepo’s tasks, Gradle’s task outputs, compiled Go packages, sccache’s compilations. Since v3.6.0, the agent serves those caches too, on Linux and Windows runners:

• Nothing to deploy: no cache server, no credentials to manage. Entries stay in your AWS account.
• Shared while jobs run: parallel jobs, matrix legs and later jobs reuse each other’s entries as soon as they are written. Nothing is restored or saved as one large archive.
• GitHub’s branch rules: see Scope and isolation.

List the tools in the remote_cache input of runs-on/action@v2, one per line, on a runner with extras=s3-cache:

- uses: runs-on/action@v2
  with:
    remote_cache: |
      go
      gradle

Without Magic Cache, or on a RunsOn version before v3.6.0, the action warns and the tools build without the remote cache. A line with an unknown tool or option is skipped with a warning.

The sticky_cache input of the same action is different: it keeps a tool’s directories on the job’s sticky disk, restored when the same lineage runs again. remote_cache shares entries through S3 with every job and branch that can read them, while they build.

How the job reaches its cache#

None of these tools sends GitHub credentials. The action registers the job’s runtime token with the agent once, and the agent answers with addresses derived from it, one per protocol, such as http://<runner IP>:6871/_runs-on/tools/<id>/kv:

• The same token always gets the same addresses, nothing that lacks the token can compute them, and they reveal nothing of the token.
• They work on that runner only, until the job’s token expires.
• The action masks <id> in the job’s logs.

Retention#

Every Magic Cache entry lives in the cache bucket of your stack and expires with the bucket’s lifecycle rule, 10 days by default. S3 counts from when the entry was written and rounds up to the next midnight UTC. Restoring an entry does not extend it, and tools rebuild what has expired.

Use a longer window for slow-changing dependency caches, and a shorter one for high-churn branches or very large artifacts.

Scope and isolation#

Magic Cache follows GitHub’s cache rules:

• A job writes to its own branch or pull request, and reads its own entries, then those of its base branch (for a pull request) and of the default branch. Bazel reads the base or default branch first: its keys digest each action’s inputs, so the branch’s own scope only holds what the branch built.
• A pull request can use the default branch’s entries but never change them. The default branch warms the cache for every branch.
• Pull requests from forks read like any pull request. Turborepo jobs from forks read without uploading.

Cache isolation enforces these rules with credentials scoped to the job. Without it, a stack keeps the access model of earlier v3 releases: each repository has its own prefix in the bucket, but the runner’s IAM role can read and write every repository’s entries under cache/*, and git checkouts are shared across the stack. It scopes the Magic Cache protocol, but it is not a trust boundary for untrusted job code: run public or otherwise untrusted repositories in a separate RunsOn stack from trusted workloads.

Isolate GitHub Actions caches#

We recommend cache isolation on every stack. Since v3.2, it gives each job brokered credentials, scoped to its repository from the job’s GitHub-signed runtime token, for actions/cache, the setup actions, the Buildx type=gha backend, and since v3.6.0 the build tool caches and git checkouts. Action downloads and apt packages hold only public files and stay shared across the stack.

Isolation stores entries under scoped-cache/* instead of cache/*. Expect one cold cache per repository and branch when you enable it, from the one-time move to the new prefix. Existing objects keep their normal retention and expire without migration.

Fleet capacity-backed runners also get one cold cache after a v3.2 upgrade, even if isolation stays off: v3.2 corrects their old empty repository prefix and repopulates the matching cache/v1/<owner>/<repo>/... namespace automatically.

The v3.2 upgrade guide covers a safe rollout, and the sticky-disk isolation that removes the legacy snapshot permissions.