Logfire
Send RunsOn job, capacity, cost, and runner telemetry to Pydantic Logfire, then install the RunsOn dashboards and alerts.
Pydantic Logfire ↗ receives RunsOn telemetry over OTLP, straight from your RunsOn stack. No SDK runs in your workflows. Its RunsOn integration adds two dashboards: one for jobs, queueing, capacity, and estimated cost, and one for the CPU, memory, disk, and network use of your runners.
This guide requires RunsOn v3.4.0 or later, on Flex or Fleet.
Create a send-only API key#
In your Logfire project, open Settings → API Keys → New API Key and select only Send telemetry. Keep the key in your secret manager.
Use the ingestion URL of the region holding your project ↗:
| Project region | Ingestion URL |
|---|---|
| US | https://logfire-us.pydantic.dev |
| EU | https://logfire-eu.pydantic.dev |
These are ingestion URLs, not the api-us or api-eu management API.
Connect the stack#
Once connected, the control plane exports job outcomes, queue times, provisioning, capacity, Spot interruptions, and cost estimates, with no change to your workflows.
Flex
On Terraform, add these inputs to your Flex module, with the key in a sensitive variable:
otel_exporter_endpoint = "https://logfire-us.pydantic.dev"otel_exporter_headers = "authorization=Bearer ${var.logfire_write_token}"otel_exporter_temporality = "delta"otel_logs_enabled = trueotel_traces_enabled = trueOn CloudFormation, update the stack with these parameters. Logs and traces are always exported once an endpoint is set.
| Parameter | Value |
|---|---|
OtelExporterEndpoint | Your ingestion URL |
OtelExporterHeaders | authorization=Bearer <your-api-key> |
OtelExporterTemporality | delta |
Fleet
Add these inputs to your Fleet module, with the key in a sensitive variable:
otel_exporter_endpoint = "https://logfire-us.pydantic.dev"otel_exporter_headers = "authorization=Bearer ${var.logfire_write_token}"otel_exporter_temporality = "delta"otel_logs_enabled = trueotel_traces_enabled = true- The endpoint is the base URL. RunsOn appends
/v1/metrics,/v1/logs, and/v1/tracesitself. - Terraform state holds the header value. Protect the state as you protect the key.
Add runner metrics#
Runner CPU, memory, disk, and network metrics need the otel extra on the runner.
Flex
To turn it on for every runner of a Terraform stack, whatever the workflow label says:
mandatory_extras = ["otel"]Otherwise, add it to a job label or to a pool’s runner spec:
runs-on: runs-on=${{ github.run_id }}/runner=2cpu-linux-x64/extras=otelFleet
Add otel to the runner definition. Every fleet using that runner gets it:
runners = { linux-small = { # ... extras = ["otel"] }}Runners with otel also forward what your workflow tools send to the local OpenTelemetry collector on 127.0.0.1:4318.
Check that data arrives#
Run a job on a runner with otel and let it finish. In Logfire’s Metrics Explorer ↗:
runs_on_jobs_totaland the otherruns_on_*metrics come from the control plane, withservice.nameset toruns-on-flexorruns-on-fleet;system.cpu.utilizationandsystem.memory.utilizationcome from the runner, withservice.nameset toruns-on-agent.
Both sides carry the stack’s environment. Runner metrics also carry stack_name, repo_full_name, and the instance ID as service.instance.id; stack_name matches the control plane’s service.namespace.
The Live view ↗ shows control-plane events and traces. On Flex, each completed job also gets one span per workflow step, sent by the control plane, with or without otel. Runner logs are RunsOn’s bootstrap log, not the GitHub job log.
The OpenTelemetry reference lists every metric and attribute.
Install the dashboards#
Once control-plane metrics arrive, open Integrations → RunsOn in your Logfire project and choose Install.
| Dashboard | Answers |
|---|---|
| RunsOn Operations | Are jobs waiting, failing to launch, or hitting Spot interruptions? How are outcomes, capacity, and cost moving? |
| RunsOn Runner Resources | Which workflows use the CPU and memory they get? Where are disk or network the bottleneck? Which sizes to revisit? |
Pick a time range that covers your runs, then filter by stack, repository, or workflow. Fleet panels stay empty on Flex, and pool panels stay empty without pools.
The install also adds alerts:
- Missing control-plane telemetry is on by default.
- Spot circuit breaker, provisioning failures, pool errors, and low cost-estimate coverage are off. Review their thresholds and turn on the ones you need on the Alerts page.
Alerts notify nobody until you set a notification channel.
Read the cost charts#
Job cost is RunsOn’s estimate of EC2 compute plus root and sticky EBS volumes, not an AWS invoice. A job RunsOn could not price has no estimate rather than a zero, so check the cost-estimate coverage panel before comparing workflows. Retained sticky disk snapshots are a separate hourly upper bound: AWS bills only their changed blocks. See cost estimates.
Troubleshooting#
| Symptom | Check |
|---|---|
| No control-plane data | The regional base URL, a key with Send telemetry on this project, and outbound HTTPS. The control-plane logs show export errors. |
| Control-plane data, but no runner metrics | otel on the runner (mandatory_extras, job label, pool spec, or Fleet runner definition). Run a new job after the change. |
| A dashboard is empty | Clear the filters and widen the time range. The panel may need Fleet, pools, priced jobs, or an incident that has not happened. |
| No alert notifications | The alert is enabled and has a notification channel on the Alerts page. |