Rate limits and scale

At tens of thousands of jobs per day, two sets of limits come into play: your AWS account’s EC2 API throttling and service quotas, and GitHub’s API rate limits. RunsOn paces itself to stay within both, so a busy stack does not exhaust quotas shared with the rest of your account. This page covers what RunsOn does automatically, the few knobs you can turn, and the quotas only you can raise.

What RunsOn does automatically#

You do not configure any of this — it is built in.

• Paces every AWS API call. EC2 launch, terminate, start/stop, and CreateTags calls each go through a token-bucket limiter before hitting AWS. Defaults are intentionally low (for example, ~2 RunInstances/CreateFleet calls per second) because RunsOn cannot know how many other stacks or tools share the account’s EC2 API quota. The launch rate scales up automatically with the AppSize preset.
• Meters GitHub API usage against the secondary rate limit. Calls are budgeted as points — 1 for reads (GET), 5 for writes (POST/PATCH/PUT/DELETE) — and held just under GitHub’s 15-points/second ceiling. Each GitHub App installation gets its own budget, and critical-path calls (like generating a runner’s registration token) take priority over background cleanup.
• Retries transient failures. GitHub 5xx responses are retried with backoff, and GitHub secondary-rate-limit responses honor the Retry-After header within the runner’s boot window. EC2 throttling (RequestLimitExceeded) is retried by the AWS SDK on top of the pacing above.
• Falls back when capacity runs out. If a spot launch hits your account’s spot quota (MaxSpotInstanceCountExceeded), RunsOn raises an alert, briefly snoozes spot, and launches on-demand instead so jobs still run. See Spot pricing.

What you can tune#

AppSize — worker and finalization throughput#

AppSize (CloudFormation) / app_size (Terraform) sets the control-plane worker concurrency, the ECS task CPU/memory, and the EC2 launch rate-limit assumptions together. It is the main stack-level throughput control. Fleet also has a per-runner-fleet batch control, described below. Current presets:

As a rough EC2 API quota target, set both RunInstances and CreateFleet request-rate quotas to about half the Provisioning workers count:

The AWS account default for each of those request-rate quotas is usually 2, which is enough for small but too low for larger presets.

For Fleet, registration workers generate GitHub JIT configs and finish instance configuration before a launch batch returns. high provides 10 registration workers and xhigh provides 20. Raising app_size can shorten this finalization step. EC2 request capacity is only one part of the total batch time.

Read the queue-time metrics#

RunsOn exports two queue-time histograms through OpenTelemetry. Both start at the job’s queue time. They stop at different points:

The internal duration is recorded when the runner is launched. The overall duration is recorded when the job completes, so it appears only after the job finishes.

If the overall duration rises while the internal duration stays flat, the extra time is in boot, registration, or GitHub. More workers will not help. Look at runner start-up and the troubleshooting guide instead.

When to raise AppSize#

Raise AppSize only when the control-plane workers are the bottleneck. Check this first, because a larger preset does nothing for an EC2 quota, Spot capacity, or GitHub limit, and can make EC2 throttling worse.

For Flex, the workers are the bottleneck when all of these hold during your bursts:

• runs_on_provisioning_backlog stays above zero for the launch or registration stage, instead of draining within a few seconds.
• runs_on_provisioning_inflight for that stage sits at the preset’s worker count (for example, 4 launch and 2 registration workers on small).
• The internal queue p90 rises with the burst and falls back afterwards. runs_on_jobs_awaiting_launch usually climbs at the same time.
• The rate limiters are not the cause: runs_on_rate_limiter_wait_duration_seconds stays low and there is no RequestLimitExceeded in the logs.

When they do, step up one preset at a time (small → medium → high → xhigh), raise the matching EC2 request-rate quotas first, and rerun a representative burst. If only one stage is saturated, the concurrency overrides can raise just that stage.

Rule out these causes before changing the preset:

For Fleet, the backlog gauges above stay empty. Use the capacity-plan log fields instead: raise app_size when JIT and config finalization extends each launch batch, and max_launch_batch_size when full batches repeat back to back.

Fleet burst launches#

Each Fleet target has one serialized reconciliation loop. While one reconciliation is running, Fleet coalesces newer GitHub assigned-job counts. It processes the current launch batch through EC2 placement, claim persistence, and JIT/config finalization, then reevaluates the latest desired count. Launch batches for one target do not overlap; different Fleet targets run independently.

fleets.<name>.max_launch_batch_size controls how much new capacity one reconciliation can process. It defaults to 50. The effective batch is the smallest of:

• the current capacity deficit
• max_launch_batch_size
• max_runners

Ready hot and stopped instances are attached first within that batch. Fleet sends any remaining claims to a cold CreateFleet request. A 100-job cold burst can therefore arrive as several waves when the batch size is below the demand.

Use a configuration like this when one runner fleet must absorb larger cold bursts:

app_size = "xhigh"

fleets = {
  linux-small = {
    runner                = "small-x64"
    max_runners           = 200
    max_launch_batch_size = 100
  }
}

Tune in this order:

1. Set max_runners to the intended peak concurrency.
2. Check EC2 vCPU and Spot quotas, subnet IP capacity, and runner-family availability at that peak.
3. Raise app_size when JIT/config finalization is extending each batch.
4. Raise max_launch_batch_size when the capacity plan repeatedly caps launches below the cold deficit.
5. Keep hot or stopped capacity for predictable, latency-sensitive peaks. Use GitHub strategy.max-parallel when the workflow should limit demand itself.

A larger batch creates more simultaneous instance and registration work. Increase it gradually and repeat a representative burst test after each change. Setting it above max_runners has no effect.

Diagnose a slow Fleet burst#

Filter Fleet worker logs for Fleet capacity plan evaluated and compare these fields:

The Completed fleet launch batch event reports requested, fulfilled, and unfulfilled counts plus registration_workers. Repeated full batches followed immediately by another capacity plan indicate a batch-limited burst. Unfulfilled counts or launch_backoff=true point to EC2 capacity or quota pressure instead.

For Fleet backlog, use runs_on_fleet_desired_runners, runs_on_fleet_claims, and the corresponding fleet_desired_runners and fleet_claims operator-snapshot fields. The generic reconciler, jobs-awaiting-launch, and provisioning backlog gauges describe Flex queues; they do not measure Fleet’s serialized target reconciliation.

Flex: AppGithubApiStrategy#

Set AppGithubApiStrategy to conservative to spend fewer GitHub API tokens — RunsOn stops eagerly de-registering finished runners (GitHub reaps them within 24h anyway). Recommended once you launch tens of thousands of jobs per day.

Advanced concurrency overrides#

The per-stage worker counts can be overridden independently of AppSize via the RUNS_ON_APP_WEBHOOK_CONCURRENCY, RUNS_ON_APP_PROVISIONING_CONCURRENCY, and RUNS_ON_APP_REGISTRATION_CONCURRENCY environment variables (passed through extra_env_vars on the Terraform module). Prefer AppSize unless you have a specific bottleneck to target.

What you must do in your AWS account#

RunsOn cannot raise your account’s quotas — these are on you, and they are the usual cause of “launches are slow” or “jobs fall back to on-demand”:

Symptoms and where to look#