Skip to content

Blog

Automatically cleanup outdated AMIs in all AWS regions

Here is a script you can use to automatically cleanup AMIs older than 60 days (configurable), while keeping the 2 most recent AMIs in each region. This helps to remove outdated images, as well as reducing storage costs for your AMIs.

Particularly useful in the case of runs-on.com, where we regularly rebuild base images whenever GitHub releases a new version of the image runner.

The bin/cleanup script (simply adjust the filters as needed):

#!/bin/bash
# Deregisters old AMIs and deletes associated snapshots, in all regions
set -e
set -o pipefail
APPLICATION="RunsOn"
REGIONS="$(aws ec2 describe-regions --query "Regions[].RegionName" --output text)"
# Number of days to keep AMIs
DAYS_TO_KEEP=${DAYS_TO_KEEP:=60}
# Define the age threshold in seconds (60 days)
AGE_THRESHOLD=$((DAYS_TO_KEEP*24*3600))
# Get the current timestamp in seconds since epoch
CURRENT_TIMESTAMP=$(date +%s)
for region in ${REGIONS[@]}; do
echo "---- Region: ${region} ---"
# List all your AMIs and extract relevant information using the AWS CLI
image_count=$(aws ec2 describe-images --owner self --filters "Name=tag:application, Values=${APPLICATION}" --query 'length(Images)' --region "$region" --output text)
echo " Total AMIs in this region: ${image_count}"
if [ "$image_count" -lt 2 ]; then
echo " Less than 2 AMIs found, skipping"
continue
fi
aws ec2 describe-images --owner self --region "${region}" --filters "Name=tag:application, Values=${APPLICATION}" --query 'Images[*].[Name,ImageId,CreationDate]' --output text | \
while read -r name image_id creation_date; do
# Parse the creation date into seconds since epoch
image_timestamp=$(date -d "$creation_date" +%s)
# Calculate the age of the AMI in seconds
age=$((CURRENT_TIMESTAMP - image_timestamp))
# Check if the AMI is older than the threshold
if [ $age -gt $AGE_THRESHOLD ]; then
echo " ! Deregistering AMI: ${image_id} (${name}) created on $creation_date"
snapshot_id=$(aws ec2 describe-images --image-ids "$image_id" --query "Images[].BlockDeviceMappings[].Ebs.SnapshotId" --region "${region}" --output text)
if [ "$DRY_RUN" = "true" ]; then
echo " DRY_RUN is set to true, skipping deregistering AMI ${image_id} and deleting snapshot ${snapshot_id}"
continue
fi
aws ec2 deregister-image --image-id "$image_id" --region "${region}"
echo " ! Deleting snapshot ${snapshot_id} for AMI ${image_id}"
aws ec2 delete-snapshot --snapshot-id "${snapshot_id}" --region "${region}"
fi
done
done

Example output:

---- Region: ap-southeast-2 ---
Total AMIs in this region: 0
---- Region: eu-central-1 ---
Total AMIs in this region: 5
Deregistering AMI: ami-0576e83d0a0f89fbe (runner-ubuntu2204-1699888130) created on 2023-11-13T16:53:48.000Z
Deleting snapshot snap-07db95a7f230d3f76 for AMI ami-0576e83d0a0f89fbe
Deregistering AMI: ami-004d4d18e6db2f812 (runner-ubuntu-22-1699873337) created on 2023-11-13T12:40:48.000Z
Deleting snapshot snap-0500b0e3fb95ab36a for AMI ami-004d4d18e6db2f812
Deregistering AMI: ami-0e6239eae649effcd (runner-ubuntu22-20231115.7-1700233930) created on 2023-11-17T17:01:58.000Z
Deleting snapshot snap-05e795e4c6fe9e66f for AMI ami-0e6239eae649effcd
Deregistering AMI: ami-0dd7f6b263a3ce28c (runner-ubuntu22-20231115-1700156105) created on 2023-11-16T19:24:38.000Z
Deleting snapshot snap-02c1aef800c429b76 for AMI ami-0dd7f6b263a3ce28c
---- Region: us-east-1 ---
Total AMIs in this region: 4
Deregistering AMI: ami-0b56f2d6af0d58ce0 (runner-ubuntu2204-1699888130) created on 2023-11-13T15:54:22.000Z
Deleting snapshot snap-0f2e8759bea8f3937 for AMI ami-0b56f2d6af0d58ce0
Deregistering AMI: ami-04266841492472a95 (runner-ubuntu22-20231115.7-1700233930) created on 2023-11-17T16:02:34.000Z
Deleting snapshot snap-0f0fcf9c6406c3ad9 for AMI ami-04266841492472a95
Deregistering AMI: ami-0738c7108915044fe (runner-ubuntu22-20231115-1700156105) created on 2023-11-16T18:21:40.000Z
Deleting snapshot snap-03f16588f59ed7cea for AMI ami-0738c7108915044fe
...

Example GitHub Action workflow file to schedule a cleanup every night:

name: Cleanup
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
on:
workflow_dispatch:
schedule:
- cron: '0 2 * * *'
jobs:
check:
timeout-minutes: 30
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- run: bin/cleanup

How to setup GitHub hosted runner with a simple cloud-init script

Edit: RunsOn is now available for installation!

While waiting for the RunsOn ephemeral self-hosted runners service, here is a short bash script, that can also be used as cloud-init script for launching GitHub hosted runners non-interactively:

#!/bin/bash -ex
set -o pipefail
RUNNER_ORG=YOUR_ORG
RUNNER_TOKEN=YOUR_TOKEN
RUNNER_LABELS=self-hosted,x64,docker
RUNNER_VERSION=2.311.0
# Install docker, then additional packages and stuff:
apt update -qq
apt install -y ruby awscli
curl https://nodejs.org/dist/v20.9.0/node-v20.9.0-linux-x64.tar.xz | tar -xJf - --strip=1 -C /usr/local/
curl https://get.docker.com | sh
cat > /etc/cron.daily/docker-prune <<EOF
docker image prune -a --filter="until=96h" --force
docker volume prune --force
EOF
chmod a+x /etc/cron.daily/docker-prune
# Create dedicated user
useradd -m -d /home/runner -s /bin/bash runner
usermod -G docker runner
# Download and install runner script
cd /home/runner
mkdir -p actions-runner
cd actions-runner
curl -o actions-runner-linux-x64-$RUNNER_VERSION.tar.gz -L https://github.com/actions/runner/releases/download/v$RUNNER_VERSION/actions-runner-linux-x64-$RUNNER_VERSION.tar.gz
tar xzf ./actions-runner-linux-x64-$RUNNER_VERSION.tar.gz
# Configure runner
su - runner -c "
/home/runner/actions-runner/config.sh --url https://github.com/$RUNNER_ORG --token $RUNNER_TOKEN --labels $RUNNER_LABELS --unattended
"
# Setup systemd scripts
cd /home/runner/actions-runner/
./svc.sh install runner
./svc.sh start
./svc.sh status

You should be good to go!

Note that those runners won’t be ephemeral, so usual caveats apply regarding security of those runners.