Bazel remote cache

RunsOn can serve a Bazel remote cache ↗ from the agent that already runs on every runner. Action results and build outputs are stored in the cache bucket of your RunsOn stack, so there is no cache service to deploy, and nothing leaves your AWS account.

Unlike a --disk_cache saved with actions/cache, a remote cache is shared while jobs run:

• Parallel jobs and shards reuse each other’s results instead of overwriting one another’s cache archive.
• Branches and pull requests reuse the default branch’s results, and download only what their build needs.
• Nothing is restored or saved as one large archive at the start and end of the job.

Availability#

Since v3.6.0, on Linux runners with Magic Cache, with Flex and Fleet. Bazel 6.3 or later.

Enabling it#

List bazel in the remote_cache input of runs-on/action@v2 ↗ on a runner with Magic Cache, before Bazel runs:

jobs:
  build:
    runs-on:
      - runs-on=${{ github.run_id }}
      - runner=16cpu-linux-x64
      - extras=s3-cache
    steps:
      - uses: runs-on/action@v2
        with:
          remote_cache: bazel
      - uses: actions/checkout@v7
      - run: bazel test //...

runners = {
  linux-bazel = {
    cpu    = 16
    family = ["c8i"]
    extras = ["s3-cache"]
  }
}

That is all. The action writes the cache address into the system Bazel configuration (/etc/bazel.bazelrc), so Bazel picks it up without any flag:

common --remote_cache=http://<runner private IP>:6871/_runs-on/tools/<id>/bazel

• A project’s own --remote_cache wins, because Bazel reads the workspace .bazelrc after the system one.
• Container jobs work the same way: the action runs inside the container and writes the container’s /etc/bazel.bazelrc. This needs root or passwordless sudo in the container.
• When the cache cannot be set up, for example without Magic Cache or on an older RunsOn version, the action warns and Bazel builds without it.

Recommended flags#

• --remote_download_minimal: by default, Bazel downloads the outputs of every top-level target, even when nothing needs rebuilding. In our benchmark, that was 82 GB for a fully cached build. With this flag Bazel downloads only what it needs to run actions.
• A higher --jobs: a cached build spends its time on cache lookups, and --jobs bounds how many run at once. Local actions stay bounded by --local_resources. In our benchmark, --jobs=200 took a fully cached build from 64 to 54 seconds.

common:ci --remote_download_minimal
common:ci --jobs=200

Scope#

The cache follows the same rules as GitHub’s cache and Magic Cache, taken from the job’s GitHub-signed token:

• Action results are written only to the job’s own branch or pull request. They are read from the base or default branch first, then from the job’s own scope, which on a branch holds only what the branch built. A pull request can use the default branch’s results but can never change them.
• Build outputs are content-addressed and verified by their SHA-256 digest, by the agent on upload and by Bazel on download. They are shared across the repository. With cache isolation enabled, they follow the same per-branch rules as action results.
• The cache address stands for the job’s token. Bazel sends no credentials of its own, so the action registers the job’s token with the agent, which answers with an address derived from it. Nothing that lacks the token can compute the address, and the address reveals nothing of the token. The address itself gives the job’s cache access to whatever can reach the runner while the job runs. The action masks it in the job’s logs, but Bazel includes it in build events ↗ it uploads.

Storage#

• Location: the cache bucket of your stack, next to Magic Cache entries.
• Compression: outputs are compressed with zstd before they are stored. In our benchmark, a build producing 89 GB of outputs stored 18 GB.
• Expiry: entries expire with the bucket’s cache lifecycle rule (10 days by default), counted from when they were written. Bazel rebuilds what has expired.

Benchmark#

openai/codex ↗ built and tested with Bazel 9.2 on a 32 vCPU runner with local NVMe: 161 Rust crates, about 11,000 actions and 89 GB of outputs. Median job time over three rounds:

After four parallel shards built one branch, a full build of the same commit re-ran 253 actions with the archive (the last shard to save overwrote the others) and 20 with the remote cache.

Limitations#

• Caching only: this is a remote cache, not remote execution.
• Bazel’s HTTP cache protocol only: tools that need the gRPC Remote Execution API, such as Buck2 and Pants, cannot use it.
• Large outputs: an output larger than 5 GB is not cached.

Troubleshooting#

Bazel’s own summary line says how many actions came from the cache, for example INFO: 11653 processes: 11458 remote cache hit, ....

• Bazel does not use the cache: check the runs-on/action step’s log for a warning. The input needs Magic Cache (extras=s3-cache) and RunsOn v3.6.0 or later.
• WARNING: Remote Cache: 401 Unauthorized: the agent no longer knows the job’s token, for example after the token expired in a very long job. Bazel continues without the cache.