Stack configuration
The CloudFormation stack that RunsOn creates when installing the service has many parameters. This page lists them all and describes what each one does.
To update a parameter, simply follow the instructions in the Upgrade guide and update the parameter in the CloudFormation interface.
Main configuration [required]
Section titled âMain configuration [required]âGithubOrganization
Section titled âGithubOrganizationâ- Type: String
- MinLength:
1
- Description: If your GitHub organization lives at github.com/my-org, then enter: my-org. IMPORTANT: this field is case-sensitive.
GithubEnterpriseUrl
Section titled âGithubEnterpriseUrlâ- Type: String
- Default:
Â
- Description: Leave blank, unless you use a self-hosted GitHub Enterprise Server (GHES) instance. In that case, if your GHES instance lives at https://github.mycompany.com â, then the value of this parameter should be: https://github.mycompany.com â
LicenseKey
Section titled âLicenseKeyâ- Type: String
- MinLength:
1
- Description: License key for RunsOn, either as a string or SSM parameter ARN (with the format
arn:aws:ssm:REGION:ACCOUNT:parameter/STACK_NAME/license-key
). Get one at https://runs-on.com/pricing â.
EmailAddress
Section titled âEmailAddressâ- Type: String
- MinLength:
1
- Description: Email address for cost and alert reports. You will receive a confirmation email from AWS at this address during the installation (check your spam folder if needed) so make sure to use a valid address. You must confirm it if you want to receive cost and alert reports.
NetworkingStack
Section titled âNetworkingStackâ- Type: String
- Default:
embedded
- AllowedValues:
embedded
,external
- Description: Let RunsOn manage your networking stack (
embedded
), or use a stack under your control (external
). If you selectexternal
, you will need to provide the VPC ID, the subnet IDs, and the security group ID, and make sure your whole networking setup is compatible with RunsOn (see https://runs-on.com/networking/embedded-vs-external/ â for more details). To get started quickly, we recommend using the âembeddedâ option.
Networking: use external stack [optional]
Section titled âNetworking: use external stack [optional]âExternalVpcId
Section titled âExternalVpcIdâ- Type: String
- Default:
Â
- Description: ID of the external VPC to reuse.
ExternalVpcSubnetIds
Section titled âExternalVpcSubnetIdsâ- Type: CommaDelimitedList
- Default:
Â
- Description: Subnet IDs (public, private, or both) of the external VPC to reuse, comma-separated. The more subnets you have, the more RunsOn will be able to find cheap spot instances, as well as balancing the load across many AZs.
ExternalVpcSecurityGroupId
Section titled âExternalVpcSecurityGroupIdâ- Type: String
- Default:
Â
- Description: Security group ID of the external VPC to assign to the runners. Must allow outbound access to the GitHub IP ranges, EFS (if enabled), and any other service your jobs will call.
Networking: use embedded stack [optional]
Section titled âNetworking: use embedded stack [optional]âVpcCidrBlock
Section titled âVpcCidrBlockâ- Type: String
- Default:
10.1.0.0/16
- Description: CIDR block for the VPC. Updating this value after creation will require deleting the stack and recreating it.
VpcCidrSubnetBits
Section titled âVpcCidrSubnetBitsâ- Type: Number
- Default:
12
- MinValue:
9
- MaxValue:
16
- Description: Number of bits to allocate for the subnet. For example, if you specify VpcCidrBlock to a /16 (the default), and this value to 12 (the default), you will have 4 bits, i.e. max 16 subnets in the VPC. Updating this value after creation will require deleting the stack and recreating it.
VpcEndpoints
Section titled âVpcEndpointsâ- Type: String
- Default:
none
- AllowedValues:
EC2
,ECR
,EC2+ECR
,none
- Description: Enable or disable VPC endpoints among: EC2 (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/interface-vpc-endpoints.html â), ECR (API + Docker - https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html â). Each endpoint incurs ~$21.6/month ($7.2/month per AZ x3).
NatGatewayAvailability
Section titled âNatGatewayAvailabilityâ- Type: String
- Default:
SingleAZ
- AllowedValues:
SingleAZ
,MultiAZ
- Description: Create either a single NAT Gateway (SingleAZ) for all 3 availability zones, or a NAT Gateway per Availability Zone (MultiAZ). Only applies if Private mode is enabled. Each gateway incurs ~$32.4/month, plus bandwidth costs.
NatGatewayElasticIPCount
Section titled âNatGatewayElasticIPCountâ- Type: Number
- Default:
1
- MinValue:
1
- MaxValue:
8
- Description: Number of Elastic IPv4s to attach to each NAT Gateway. Only applies if Private mode is enabled. Defaults to 1. Maximum of 8. Increasing this number is useful if you launch a large number of runners and are rate-limited by some external services due to the limited number of NAT Gateway IP addresses. Incurs ~$3.6/month additional cost for each Elastic IP. Note that by default you are limited to 2 EIPs per NAT gateway, but can request a quota increase from AWS.
VpcFlowLogFormat
Section titled âVpcFlowLogFormatâ- Type: String
- Default:
Â
- Description: VPC Flow Log format. If blank, no VPC Flow Logs will be created. If set to
default
, the default format will be used. More details at https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html#flow-logs-fields â
VpcFlowLogS3BucketArn
Section titled âVpcFlowLogS3BucketArnâ- Type: String
- Default:
Â
- Description: ARN of an S3 bucket to send VPC Flow Logs to in addition to the default CloudWatch logs group. If blank, the logs will only be sent to CloudWatch Logs.
VpcFlowLogRetentionInDays
Section titled âVpcFlowLogRetentionInDaysâ- Type: Number
- Default:
7
- MinValue:
1
- Description: Retention period in days for VPC Flow Logs.
Network and Security settings [optional]
Section titled âNetwork and Security settings [optional]âPrivate
Section titled âPrivateâ- Type: String
- Default:
false
- AllowedValues:
only
,always
,true
,false
- Description: Enable (âtrueâ, âalwaysâ, âonlyâ) or disable (âfalseâ) private networking. If âtrueâ, your jobs can opt-in to launch in private subnets with the
private=true
label, and they will get a static egress IP. If âalwaysâ, all jobs will run in private subnets, and they will all get a static egress IP, unless you manually opt-out in your job definition with theprivate=false
label. If âonlyâ, then jobs can only launch in private subnets. Note that enabling it will create 1 managed NAT gateway, with the corresponding costs. More details at https://runs-on.com/networking/static-ips/ â.
Ipv6Enabled
Section titled âIpv6Enabledâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: Enable or disable IPv6 for runners. Disabling ipv6 might help with Docker Hub rate limiting issues. If enabled and reusing a VPC, ensure the VPC is IPv6 enabled.
SSHAllowed
Section titled âSSHAllowedâ- Type: String
- Default:
true
- AllowedValues:
true
,false
- Description: Allow inbound SSH connections from the specified CIDR range to the runners.
SSHCidrRange
Section titled âSSHCidrRangeâ- Type: String
- Default:
0.0.0.0/0
- MinLength:
1
- Description: CIDR range for inbound SSH access. By default, only repository collaborators with admin permission will be able to SSH into the runner instances.
EC2InstanceCustomPolicy
Section titled âEC2InstanceCustomPolicyâ- Type: String
- Default:
Â
- Description: Optional managed IAM Policy ARN to assign to the EC2 runner instances. Can be used to give runners access to AWS resources that you manage.
AppCustomPolicy
Section titled âAppCustomPolicyâ- Type: String
- Default:
Â
- Description: Optional managed IAM Policy ARN to assign to the App runner service role. Can be used to e.g. allow access to KMS decryption keys for AMIs.
DefaultPermissionBoundaryArn
Section titled âDefaultPermissionBoundaryArnâ- Type: String
- Default:
Â
- Description: ARN of an IAM policy that will be assigned as permission boundary to roles created by the stack. If blank, the roles will not have a permission boundary.
ServerPassword
Section titled âServerPasswordâ- Type: String
- Default:
Â
- Description: Password for the /metrics Prometheus endpoint (https://runs-on.com/monitoring/metrics/#prometheus-metrics â). If blank, the endpoint will be disabled.
DefaultAdmins
Section titled âDefaultAdminsâ- Type: String
- Default:
Â
- Description: Comma-separated list of GitHub usernames that will always be granted SSH access to all the runner instances (if SSH access is enabled), in addition to any repository admins. If blank, only repository collaborators with admin permission will be able to SSH into the runner instances.
EncryptEbs
Section titled âEncryptEbsâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: Encrypt the EBS volumes of the runner instances. Encryption will slow the boot time of runner instances by ~10s.
EnableEfs
Section titled âEnableEfsâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: Enable EFS shared filesystem for runners (only available with âembeddedâ networking). If enabled, an EFS volume will be created and made available to runners. Runners will need appropriate permissions and tools (e.g., nfs-utils) to mount it.
EnableEphemeralRegistry
Section titled âEnableEphemeralRegistryâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: If enabled, create an ECR ephemeral registry that can be used as a type=registry cache backend for Docker BuildKit. The registry will be created with a 10-day lifecycle policy, and runners will automatically be granted push/pull access.
Alert and monitoring settings [optional]
Section titled âAlert and monitoring settings [optional]âAppAlarmDailyMinutes
Section titled âAppAlarmDailyMinutesâ- Type: Number
- Default:
4000
- Description: Trigger an alarm if the cumulative number of minutes consumed during a day is over that number.
CostReportsEnabled
Section titled âCostReportsEnabledâ- Type: String
- Default:
true
- AllowedValues:
true
,false
- Description: Enable or disable cost reports sent by email.
AlertTopicSubscriptionHttpsEndpoint
Section titled âAlertTopicSubscriptionHttpsEndpointâ- Type: String
- Default:
Â
- Description: HTTPS endpoint for cost and alert reports.
ECInstanceDetailedMonitoring
Section titled âECInstanceDetailedMonitoringâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: Enable or disable detailed monitoring for EC2 instances (can incur additional costs).
Ec2LogRetentionInDays
Section titled âEc2LogRetentionInDaysâ- Type: Number
- Default:
7
- MinValue:
1
- Description: Retention period in days for EC2 instance logs.
SqsQueueOldestMessageThresholdSeconds
Section titled âSqsQueueOldestMessageThresholdSecondsâ- Type: Number
- Default:
0
- Description: Trigger an alarm per sqs queue if the oldest message in that queue is older than this time in seconds. Leave as 0 to disable SQS alarms.
Integrations [optional]
Section titled âIntegrations [optional]âIntegrationStepSecurityApiKey
Section titled âIntegrationStepSecurityApiKeyâ- Type: String
- Default:
Â
- Description: API key for Step Security (https://stepsecurity.io/ â).
Advanced app configuration [optional]
Section titled âAdvanced app configuration [optional]âEnvironment
Section titled âEnvironmentâ- Type: String
- Default:
production
- MinLength:
1
- Description: Make RunsOn only listen to jobs with an
env
job label matching this value. Only change this if you know what you are doing. See https://runs-on.com/configuration/environments/ â for more details.
CostAllocationTag
Section titled âCostAllocationTagâ- Type: String
- Default:
stack
- Description: Cost allocation tag key to use for all the deployed resources. The value for this tag will be set to the stack name. Updating this after creation will require deleting the stack and recreating it.
SpotCircuitBreaker
Section titled âSpotCircuitBreakerâ- Type: String
- Default:
2/15/30
- AllowedPattern: ^(false|[0-9]+/[0-9]+/[0-9]+)$
- Description: Enable (
COUNT/WINDOW_MINUTES/RECOVERY_MINUTES
) or disable (false
) the spot instance circuit breaker. If enabled, the circuit breaker will prevent the stack from launching new spot instances during RECOVERY_MINUTES if there was at least COUNT spot interruptions during the last WINDOW_MINUTES. For instance 2/15/30 will prevent the stack from launching new spot instances during 30 minutes if there was at least 2 spot interruptions during the last 15 minutes.
RunnerCustomTags
Section titled âRunnerCustomTagsâ- Type: CommaDelimitedList
- Default:
Â
- Description: Optional custom tags for the runner instances (e.g. âkey1=value1,key2=value2â). Tag keys cannot start with
runs-on
, and can only use letters (a-z, A-Z), numbers (0-9), and the following characters: + - = . , _ : @ (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions â)
RunnerDefaultDiskSize
Section titled âRunnerDefaultDiskSizeâ- Type: Number
- Default:
40
- MinValue:
40
- Description: Disk size in GB for
disk=default
runners.
RunnerDefaultDiskDeviceName
Section titled âRunnerDefaultDiskDeviceNameâ- Type: String
- Default:
/dev/sda1
- Description: Name of the root volume device name for the
disk=default
runner. Only change this if you plan on using custom AMIs that have a different root volume name.
RunnerDefaultVolumeThroughput
Section titled âRunnerDefaultVolumeThroughputâ- Type: Number
- Default:
400
- MinValue:
125
- MaxValue:
1000
- Description: Volume throughput in MiB/s for
disk=default
runners (helps with faster boot times, but costs more).
RunnerLargeDiskSize
Section titled âRunnerLargeDiskSizeâ- Type: Number
- Default:
80
- MinValue:
40
- Description: Disk size in GB for
disk=large
runners.
RunnerLargeDiskDeviceName
Section titled âRunnerLargeDiskDeviceNameâ- Type: String
- Default:
/dev/sda1
- Description: Name of the root volume device name for the
disk=large
runner. Only change this if you plan on using custom AMIs that have a different root volume name.
RunnerLargeVolumeThroughput
Section titled âRunnerLargeVolumeThroughputâ- Type: Number
- Default:
750
- MinValue:
125
- MaxValue:
1000
- Description: Volume throughput in MiB/s for
disk=large
runners (helps with faster boot times, but costs more).
RunnerMaxRuntime
Section titled âRunnerMaxRuntimeâ- Type: Number
- Default:
720
- MinValue:
30
- Description: Maximum runtime in minutes for runners. After this time, the runner will be forcefully terminated.
AppEc2QueueSize
Section titled âAppEc2QueueSizeâ- Type: Number
- Default:
2
- MinValue:
1
- Description: Number of queued jobs that can be processed concurrently by RunsOn. Each job takes about 2-3s to process. Increasing this value should be done with caution, as it will increase the number of EC2 instances that will be launched at once, eating into your AWS and GitHub rate limits. New AWS accounts come with a rate-limit of 2 RunInstances call/s, although you can request a higher limit from AWS (https://docs.aws.amazon.com/AWSEC2/latest/APIReference/throttling.html â).
AppGithubApiStrategy
Section titled âAppGithubApiStrategyâ- Type: String
- Default:
normal
- AllowedValues:
normal
,conservative
- Description: Use normal or conservative GitHub API strategy. The conservative strategy will attempt to save GitHub API tokens by not automatically de-registering runners from the GitHub API after they are no longer needed. Itâs more elegant to clean them up (recommended), but in any case GitHub will eventually clean them up on their side after 24h. Especially useful if you launch tens of thousands of jobs every day.
- Type: Number
- Default:
256
- AllowedValues:
256
,512
,1024
,2048
,4096
- Description: CPU units for RunsOn service (256 or higher). You might only need to change this if you have 10k+ jobs per day.
AppMemory
Section titled âAppMemoryâ- Type: Number
- Default:
512
- AllowedValues:
512
,1024
,2048
,3072
,4096
,6144
,8192
- Description: Memory in MB for RunsOn service (512 or higher). You might only need to change this if you have 10k+ jobs per day.
AppRegistry
Section titled âAppRegistryâ- Type: String
- Default:
public.ecr.aws/c5h5o9k1/runs-on/runs-on
- Description: Docker image (public) registry for the RunsOn service.
AppDebug
Section titled âAppDebugâ- Type: String
- Default:
false
- AllowedValues:
true
,false
- Description: Enable or disable debug mode for the RunsOn stack. Enabling this will switch off the auto-shutdown of the runner instances when they are unable to bootstrap.
S3CacheExpirationInDays
Section titled âS3CacheExpirationInDaysâ- Type: Number
- Default:
10
- Description: Number of days to keep cache entries in the S3 cache bucket.