Pydantic: a CI bill two-thirds lower, with five times the pull requests
AI agents pushed Pydantic's Logfire repository from 365 to 2,032 merged PRs a month. On RunsOn, CI cost $5,700 in September, down from $18,200 in August.
- $5.7k CI cost in September, from $18.2k in August
- 365 → 2,032 PRs merged in 30 days, a year apart
- +31% runner minutes, August to September
- 20,659 main workflow runs in August
- Company
- Pydantic · Python data validation, observability and AI agents
- Before
- GitHub-hosted, Depot and Ubicloud runners
- After
- RunsOn in Pydantic's AWS account, mostly Graviton on spot
- Source
- Pydantic runs on RunsOn, by Bill Easton(Head of Product, Pydantic Logfire), October 16, 2026
About Pydantic#
Pydantic is the company behind Pydantic, the most widely used data validation library for Python. Samuel Colvin started it as an open-source project in 2017, and it passed 10 billion downloads in early 2026. The company also builds Logfire, an observability platform, and Pydantic AI, a framework for building agents.
Logfire is built from one repository, pydantic/platform, and that repository is where this story happens.
What hurt#
AI changed how much the team ships. In the 30 days to August 30, 2026, pydantic/platform merged 2,032 pull requests. Over the same 30 days of 2025, a team of the same size merged 365.
That is real work, not noise: new products, more tests, wider browser coverage and many more automated reviews, with bots opening and reacting to a lot of the activity. Every one of those changes runs CI, and the main workflow’s run count followed:
| Month | Main workflow runs |
|---|---|
| May 2026 | 4,194 |
| June 2026 | 6,669 |
| July 2026 | 11,919 |
| August 2026 | 20,659 (to the 30th) |
The bill followed too. CI ran on GitHub-hosted, Depot and Ubicloud runners:
| Month | CI spend | Runner minutes |
|---|---|---|
| April 2026 | $3,813 | |
| May 2026 | $3,665 | |
| June 2026 | $3,990 | |
| July 2026 | $10,778 | |
| August 2026 | $18,200 | 1,230,264 |
July cost nearly three times April, and August almost five times. GitHub Actions and managed runner services had carried the team this far without running a runner fleet of its own. But above a million runner minutes a month, it wanted more control over which machines it paid for.
Why RunsOn#
The team did not want to replace GitHub Actions, and did not want to build and run its own runner platform either. RunsOn sits between the two: GitHub still orchestrates every workflow, while each job gets a fresh EC2 instance in Pydantic’s own AWS account that disappears when the job ends. RunsOn handles that lifecycle, and Pydantic keeps control of the machines, networking, storage and cost underneath.
The volume made the decision easy. RunsOn pays for itself from about 22 jobs a day, and the main workflow of pydantic/platform alone ran about 700 times a day in August.
Moving over was mostly a label change. Once RunsOn was installed and connected to GitHub, ordinary jobs needed a different runs-on: label and the RunsOn setup action; container builds needed some extra cache and registry changes. Pydantic moved to RunsOn on September 1.
How they run it#
- A catalog from 1 to 16 cores. One-core ARM runners at the small end, 16-core image builders at the top.
- Graviton first. Most Python, TypeScript, agent and control-plane jobs run on arm64. x64 stays for container builds and the tests that need it.
- Several instance families per runner. Each preset lists more than one eligible EC2 family, so the spot allocator has more capacity pools to pick from.
- Spot where a retry is cheap. Merge-queue validation, releases, deployments and jobs that are expensive to restart run on on-demand instances. Long jobs that dislike interruptions use capacity-optimized spot, shorter ones price-capacity-optimized spot. The stack spans three availability zones, and the spot circuit breaker moves new jobs to on-demand for a while after a burst of interruptions.
- Cache next to the runners. Magic Cache moves the existing
actions/cachesteps onto an S3 bucket in their own AWS account, with no change to the steps, and sticky disks keep BuildKit state between Docker builds.
Results#
September was the first month on RunsOn. Total CI cost was $5,700, down from $18,200 in August, and that includes the RunsOn license and every AWS cost: compute, EBS, S3 cache storage and data transfer.
The workload kept growing through the move. Runner minutes went from 1,230,264 in August to 1,617,654 in September: 31% more CI for 69% less money.
To see what each job uses, the team sends RunsOn’s telemetry to Logfire, and built a RunsOn integration for it along the way. Any RunsOn stack can now use it.
What comes next#
The team came out of the move with three rules that matter more than any instance type:
- Use spot where retries are cheap. Keep the merge queue and release paths on on-demand capacity.
- Right-size against the critical path. The cheapest runner is the one that meets the job’s latency target at the lowest total cost.
- Keep cache near the compute. Retain it only while it is likely to be reused.
The agents are not slowing down, and they don’t need to. With the runners in its own account, Pydantic now decides what each job costs.
AI increased the amount we could deliver, and our CI bill grew with it. We did not ask the agents to stop. We changed where their builds and tests ran, then kept tuning the machines, caches, and capacity as the workload grew.