How to install GitHub Actions Runner Controller on Kubernetes with MicroK8s

In this article we will go over how to install GitHub Actions Runner Controller (ARC) on Kubernetes. For the purpose of this article we will use MicroK8s ↗ which is a lightweight Kubernetes distribution that is easy to install and manage. If you have a standard Kubernetes cluster, you can use it instead and simply remove the microk8s prefix from the commands.

The variant that we will install is the version ~officially maintained by GitHub, which uses Runner Scale Sets (gha-runner-scale-set-controller) provided by the GitHub Actions service instead of relying on webhooks emitted by the GitHub API. The advantage of this version is that you do not need to expose any public service to the internet.

This article assumes that you have created a brand new server somewhere, with Ubuntu 24.04 and at least 8GB of RAM. Hetzner Cloud servers are perfect for that.

Install MicroK8s and enable helm

Log into your server, switch to the root user, and run the following commands:

snap install microk8s --classic
usermod -a -G microk8s $USER

Log out (exit) and log back in for the changes to take effect, and run the following command to verify that MicroK8s is running:

microk8s status --wait-ready # might take a while

You should see something like:

microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
...

Inspect the nodes in your cluster:

microk8s kubectl get nodes

You should see something like:

NAME   STATUS   ROLES    AGE   VERSION
arc    Ready    <none>   99s   v1.30.5

Enable helm so that you can deploy the Charts for ARC:

microk8s enable helm

Install the runner controller

microk8s helm install arc \
  --namespace "arc-systems" \
  --create-namespace \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

You should see something like:

NAME: arc
LAST DEPLOYED: Wed Oct 16 11:53:39 2024
NAMESPACE: arc-systems
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing gha-runner-scale-set-controller.

Your release is named arc.

At this point, the runner controller is installed and running in your cluster in the arc-systems namespace.

Create the arc-runners namespace

This namespace will host the runners that will be created by the runner scale sets:

microk8s kubectl create namespace arc-runners

Now, before you can create the runner scale sets, you need to create a private GitHub App that will be used to authenticate API calls with GitHub.

Create private GitHub App and register it with ARC

To create a private GitHub App for your ARC deployment, go to your organization settings and click on:

"Developer settings" -> "GitHub Apps" -> "New GitHub App"

Fill in the form as follows:

Once created:

• Take a note of the App ID.
• Generate a Private Key.
• Click on the “Install App” link in the sidebar, select your organization, and choose the repositories it has access to.

Finally, take a note of the Installation ID in the browser URL. URL format is:

https://github.com/organizations/ORGANIZATION_NAME/settings/installations/INSTALLATION_ID

At this point, set the following environment variables on your server:

• ORGANIZATION_NAME: The name of your organization (e.g. runs-on-demo).
• APP_ID: The App ID you got from the GitHub App you created.
• INSTALLATION_ID: The Installation ID you got after installing the GitHub App in your organization.
• PRIVATE_KEY_PATH: Path to the private key file on your server (upload it first if needed).

You can now create a secret with the private key and application details in your Kubernetes cluster:

microk8s kubectl create secret generic pre-defined-secret \
   --namespace=arc-runners \
   --from-literal=github_app_id="$APP_ID" \
   --from-literal=github_app_installation_id="$INSTALLATION_ID"  \
   --from-file=github_app_private_key="$PRIVATE_KEY_PATH"

Once this secret is created, you can proceed to create the runner scale sets that will be used to run your workflows.

Create one (or multiple) runner scale set(s)

A runner scale set is managed by a single controller, and is used to launch runners of a specific configuration (image, privileges, etc) whenever a new job requests it.

Note that the name you will give to that runner scale set (RUNNER_SCALE_SET_LABEL in the following commands) will be the label that you must use in your runs-on: definition in workflow files. Only a single label can be used.

At this point, set the following environment variables on your server:

• ORGANIZATION_NAME: The name of your organization (e.g. runs-on-demo).
• RUNNER_SCALE_SET_LABEL: The label of the runner scale set you want to create (e.g. arc-custom-runners-type1).

Then create the runner scale set:

microk8s helm install "${RUNNER_SCALE_SET_LABEL}" \
   --namespace "arc-runners" \
   --create-namespace \
   --set githubConfigUrl="https://github.com/${ORGANIZATION_NAME}" \
   --set githubConfigSecret="pre-defined-secret" \
   oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

You should see something like:

NAME: arc-custom-runners-type1
LAST DEPLOYED: Wed Oct 16 12:00:24 2024
NAMESPACE: arc-runners
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing gha-runner-scale-set.

You can verify your Kubernetes resources with the following command:

microk8s helm list -A

Which should return something like:

NAME                    	NAMESPACE  	REVISION	UPDATED                                	STATUS  	CHART                               APP VERSION
arc                     	arc-systems	1       	2024-10-16 11:53:39.488371231 +0000 UTC	deployed	gha-runner-scale-set-controller-0.9.0.9.3      
arc-custom-runners-type1	arc-runners	1       	2024-10-16 12:00:24.77941948 +0000 UTC 	deployed	gha-runner-scale-set-0.9.3          0.9.3     

You can now go to your organization settings, click on “Actions” -> “Runners”, and see your new runner scale set coming online.

At this point, you have now two pods running in the arc-systems namespace, one for the main ARC controller, and one for your runner scale set, which listens for new jobs and launches runners when needed:

microk8s kubectl get pods -n arc-runners

Which should return something like:

NAME                                         READY   STATUS    RESTARTS   AGE
arc-custom-runners-type1-754b578d-listener   1/1     Running   0          8m10s
arc-gha-rs-controller-5866fb9f96-8trgs       1/1     Running   0          14m

You can add another runner scale set by simply running the same command with a different label, for example:

microk8s helm install arc-custom-runners-type2 \
   --namespace "arc-runners" \
   --create-namespace \
   --set githubConfigUrl="https://github.com/${ORGANIZATION_NAME}" \
   --set githubConfigSecret="pre-defined-secret" \
   oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

You now have 3 pods running in the arc-systems namespace:

microk8s kubectl get pods -n arc-runners
NAME                                         READY   STATUS    RESTARTS   AGE
arc-custom-runners-type1-754b578d-listener   1/1     Running   0          9m26s
arc-custom-runners-type2-754b578d-listener   1/1     Running   0          2s
arc-gha-rs-controller-5866fb9f96-8trgs       1/1     Running   0          16m

Verify that the installation is working by launching a job

It’s finally time to try to launch a job to see if everything works.

Create a new workflow YAML file in your repository, for example:

name: Test ARC

on:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: arc-custom-runners-type1
    steps:
      - run: echo "Hello, world!"

Push the file to your repository. You should see your workflow being executed by one of your runners, which will start in the arc-runners namespace:

microk8s kubectl get pods -n arc-runners 

Which should return something like:

NAME                                          READY   STATUS              RESTARTS   AGE
arc-custom-runners-type1-cbn68-runner-nxj9x   0/1     ContainerCreating   0          26s

After a little while, the runner should be up and the job completed:

Removing a runner scale set

If at any point you want to remove a runner scale set, you can do so with the following command:

microk8s helm delete RUNNER_SCALE_SET_LABEL -n arc-runners

This will then remove the runner scale set from the GitHub UI as well. Note that it will not work if you have already removed the main controller (from the arc-systems namespace), so make sure to remove runner scale sets first if you want to uninstall ARC..

Considerations

ARC is quite easy to setup, but there are a few things you should be aware of:

• everything runs in a Docker container, in an image with a very limited set of software installed. You will need to largely adjust your workflows, and create and maintain your own Docker images if you need more software installed.
• a lot of third-party actions will not work.
• it is sometimes hard to comprehend the logs emitted by the controller and the runners, and you don’t get any metrics emitted.
• you must have a kubernetes cluster running, irrespective of whether you have actual jobs running.
• for each variation of runner configuration, you need to create a new runner scale set.

Conclusion

ARC can be a great choice if you are already comfortable with Kubernetes. If you are looking for something as scalable but much easier to install and manage (and with no dangling resources), you can have a look at RunsOn, which provides 1-1 compatible images, real virtual machines, metrics, static IPs, SSH access, custom images, and more.