Git checkout caching

Since v3.6.0, the git-cache extra stores the first checkout of a commit in your stack’s S3 cache bucket. Every later checkout of that commit, on any runner of the stack, streams from S3 instead of GitHub. Matrix jobs, test shards, the jobs of one workflow, and re-runs all check out the same commit, so most of them hit the cache.

Nothing changes in your workflows: actions/checkout and any other git client on the runner keep using https://github.com/ URLs and their usual credentials.

Availability#

It works with github.com and GHE.com (data residency), including on stacks with cache isolation (see Security). It is not available on GitHub Enterprise Server.

Enable it#

runs-on:
  - runs-on=${{ github.run_id }}
  - runner=4cpu-linux-x64
  - extras=s3-cache+git-cache

runners = {
  linux-build = {
    cpu    = 4
    family = ["c8i"]
    extras = ["s3-cache", "git-cache"]
  }
}

A checkout served from the cache prints a line like this in the actions/checkout step:

remote: RunsOn git cache: serving 453.1 MiB from S3

What to expect#

On an m8a.xlarge runner, with actions/checkout@v4:

The gain is largest for shallow checkouts of large repositories. A full-history checkout is bound by the runner’s CPU: git indexes every object it receives, and that takes longer than the transfer. Storing a checkout on a miss adds no measurable time.

Concurrent checkouts#

Jobs of a matrix start their checkouts within seconds of each other, before the first one has stored anything. So when several runners miss the same commit at once, the first one fetches it from GitHub and stores it, and the others wait for it and check out from the cache. If that runner stops making progress, the others fetch from GitHub themselves, so a checkout is never stuck waiting.

In a 20-job matrix checking out elastic/kibana, this turned 20 fetches from GitHub into 1: 425 MB instead of about 8.5 GB. Checkout times stayed about the same as fetching from GitHub directly.

Private subnets and NAT gateways#

When runners use private networking, every fetch from GitHub goes through the NAT gateway, which charges for each GB it processes. Cache traffic to S3 does not, as long as the VPC has an S3 gateway endpoint; the Flex CloudFormation stack creates one. With the cache on, a commit crosses the NAT gateway once per stack, however many jobs check it out.

What is cached#

• Fetches in a fresh workspace, which is what actions/checkout does, at any fetch-depth and with filter. Later identical fetches are served from S3.
• A full-history fetch asks for every branch and tag, so its cache entry changes whenever a ref moves: it hits while the repository’s refs are unchanged.
• Everything else goes to GitHub unchanged: fetches into a workspace that already has objects, ref listings, pushes, and Git LFS.
• Jobs in a container: and Docker builds run their own git and go straight to GitHub.

Security#

• GitHub stays in charge of access. The runner serves a cached checkout only to credentials that GitHub accepted for that repository moments before. A job cannot use the cache to read a repository its token cannot read.
• The cache cannot change your code. Git verifies every object it receives against the commit you asked for. A corrupted or tampered entry makes the fetch fail instead of changing the checkout; actions/checkout retries, and the retry is fetched from GitHub and replaces the entry.
• Without cache isolation, entries are shared within the stack. They live under cache/git/v1/ in the cache bucket, which every runner of the stack can read with its instance role, as with other Magic Cache data. Job code on one repository could read the cached source of another repository of the same stack. Enable cache isolation, or run untrusted or sensitive repositories in a separate stack.
• With cache isolation, entries follow GitHub’s cache rules. Each repository’s entries live in its own isolated storage, split by branch as with actions/cache. A job stores checkouts only for its own branch, and reads those of its branch, the default branch and, for a pull request, its base branch. So a pull request reuses its base branch’s checkouts, but cannot store anything the base branch’s jobs read. The job’s own repository is cached there, and so are public repositories it checks out, such as an open-source dependency fetched at a pinned commit. Checkouts of other private repositories go straight to GitHub: job code can read its repository’s cache storage directly, so a private repository stored there could be read by later jobs that have no access to it, such as pull requests from forks. This relies on the RunsOn build of the GitHub Actions runner, which hands the job’s credentials to the cache; on images with the stock runner, every checkout goes to GitHub.

Retention#

Entries expire with the cache bucket’s lifecycle rule (S3CacheExpirationInDays in CloudFormation, cache_expiration_days in Terraform), 10 days by default.

Troubleshooting#

• If the cache cannot start, the job continues with a warning annotation, and git fetches go directly to GitHub.
• The cache stays off when the image already routes git to GitHub through a proxy (http.proxy in /etc/gitconfig or the runner user’s ~/.gitconfig, or a proxy variable in /etc/environment).
• Setting GIT_SSL_CAINFO in a job overrides the certificate git uses for the cache, and fetches from GitHub then fail. Unset it, or drop the extra for that job.
• curl -s http://127.0.0.1:6871/_runs-on/git-cache on the runner prints the cache’s counters: hits, misses, bytes served, and entries stored.