Create Alarm

Create a metric alarm with a threshold over evaluation periods.

Set denominator_metric_name for a rate rather than a count: the datapoint becomes metric_name / denominator_metric_name and threshold reads as a fraction (0.05 = 5%). Prefer this for anything that scales with traffic — a fixed error count pages at 3am once the org grows. Pair it with min_sample_count so a single failure in a quiet minute isn’t a 100% error rate.

Both series must be binned on the same clock, since they are divided period by period. For outcome rates that means app.run.terminal_count — runs that finished in the period — and not app.run.count, which counts runs that started and includes ones still running. Mismatched pairs are rejected.

Ratio alarms also require stat sum or count. The stat is applied to each series and the two results divided, so only stats that add up across the minutes in a period come out as a rate — avg would divide two means and min/max would take their extremes from different minutes. Other stats are rejected rather than silently producing a number that isn’t a rate.

A sustained incident only ever notifies once, so repeat messages mean the alarm is flapping. Three levers fix that, and which one depends on the metric: clear_threshold (fire high, clear low) for continuous series like rates and latencies; datapoints_to_alarm for spiky ones that cross the line briefly; and missing_data: "ignore" for sparse ones, where a minute with no traffic is absence of evidence rather than a recovery.

Use GET /orgs/{org_id}/metrics/catalog to see which metric names and labels this org is actually emitting. Two metrics won’t be listed until you alarm on them: org.credits.usage_pct (percent of the billing period’s prepaid budget consumed) and org.credits.remaining (credits left in it) are only sampled for orgs that have an enabled alarm on one, since the underlying read is the billing gate’s and too heavy to run speculatively. Both are levels sampled once a minute, so use max, avg or min — sum would add one reading per minute in the period and is rejected. org.credits.remaining goes negative under overage.

Percent and headroom are worth having separately: 95% of a large budget can still be thousands of credits, while 50% of a small one is one bad afternoon. For burn volume rather than budget position, alarm on org.credits.spend with sum — that one carries project_id and workforce_id dims, so it can scope to a single project or workforce the way a budget can’t.

For latency, stat accepts any percentile from p0.1 to p99.9, in tenths. Equivalent spellings are fine (p95.0 is accepted and comes back as p95), but anything finer than a tenth is rejected rather than rounded, so an alarm never silently watches a different point than the one you asked for. These are exact, computed over the individual run durations rather than interpolated from bins, which is also why they are limited to app.run.duration_ms, cannot be a ratio, and cannot carry label, project_env_id or dims.deployment_id — a run row has no such column, so those filters would silently widen the alarm to everything. Prefer a high percentile over avg for latency: an average hides the tail that users actually notice, and hides it harder as traffic grows.

Two different project fields, gated separately. Top-level project_id is ownership — who may see and edit the alarm afterwards — and setting it requires projects.alarms.manage on that project, while leaving it out makes an org-level alarm and requires alarms.manage. dims.project_id only narrows which metric bins are read, but it still exposes that project’s numbers through the alarm’s notifications, so it requires projects.alarms.read on the project it names.