Sharing files across workflow jobs with EFS

Since v2.8.2, you can configure RunsOn to automatically create an Elastic File System (EFS), which can be used for storing artefacts across workflow jobs. Compared to the classic actions/cache behaviour, EFS doesn’t have any size limit, and skips the compression step, which makes it faster for storing large numbers of files.

Use cases

EFS is a good fit for e.g.:

1. Maintaining git mirrors of very large git repositories, so that your jobs don’t need to go to GitHub to fetch the code.
2. Storing frequently-accessed data files close to your runners, so that your jobs don’t need to download them from scratch.
3. Programs that need to manipulate caches but can’t use an object storage bucket. In that case, EFS is just seen as a normal filesystem by your runners.

Accessing EFS

Runners automatically get permission to access the EFS file system, using the EC2 instance profile assigned to the runners.

The RUNS_ON_EFS_ID environment variable is available in your steps, and contains the EFS file system ID (e.g. fs-12345678901234567).

If you include efs in the extras parameter of your workflow job, the EFS file system will be mounted in the /mnt/efs directory.

Note: EFS is only available if the RunsOn stack has the EFS feature enabled. And it is currently only available when using the embedded networking stack.

Using EFS in your workflow

The EFS file system is mounted in the /mnt/efs directory. You can access it as you would any other filesystem.

jobs:
  efs:
    runs-on: runs-on=${{ github.run_id }}/runner=2cpu-linux-x64/extras=efs
    steps:
      - run: echo "Hello, world!" > /mnt/efs/hello.txt
      - run: cat /mnt/efs/hello.txt
      - run: df -ah /mnt/efs
      # 127.0.0.1:/      8.0E   35G  8.0E   1% /mnt/efs

Cleanup

There is currently no mechanism to automatically cleanup the EFS file system.

Considerations

Security-wise, the EFS file system is shared across all runners launched by the same RunsOn stack. So make sure you only store files that can be shared across all runners. You can create multiple RunsOn environments if you need better isolation.

Limitations

• EFS is only available on Linux runners.