curl --request POST \
  --url https://api.timbal.ai/orgs/{org_id}/metrics/definitions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Tool errors",
  "metric_name": "tool_errors",
  "has_error": true,
  "value_source": "count",
  "label_source": "span_name"
}
'

{
  "created_at": "2023-11-07T05:31:56Z",
  "enabled": true,
  "id": 123,
  "label_source": "<string>",
  "metric_name": "<string>",
  "name": "<string>",
  "source": "trace_span",
  "value_source": "<string>",
  "has_error": true,
  "min_duration_ms": 123,
  "project_id": 123,
  "span_name": "<string>",
  "span_name_prefix": "<string>",
  "status_code": "<string>"
}

{
  "code": "<string>",
  "message": "<string>"
}

{
  "code": "<string>",
  "message": "<string>"
}

Definitions

Create Metric Definition

Define a metric of your own, emitted as runs land. Once it exists it behaves like any platform metric: readable through GET /orgs/{org_id}/metrics, listed in the catalog with origin: custom, and alarmable with no extra setup.

The definition is a span predicate over each completed run’s trace. Span paths are {workforce}.{step} and matching is on the leaf: use llm or get_address, not the full path. Predicate fields AND together and all are optional — omitting every one matches every non-root span.

metric_name is forced under the reserved trace. prefix so a definition cannot shadow a platform series. Set label_source to span_name to break the metric out per step, then point an alarm at a single step with dims.label.

Definitions are not retroactive: matching happens as runs arrive, so a new definition only sees traces written after it. Dry-run it against stored traces first with POST /orgs/{org_id}/metrics/definitions/preview. Note also that only sum/count/avg/min/max are available on the result — percentiles are not derivable from metric bins.

Set project_id to scope the definition to one project, which also makes it editable by holders of projects.alarms.manage there. Leaving it out defines an org-wide metric that runs against every project’s traces, and needs alarms.manage.

POST

orgs

{org_id}

metrics

definitions

curl --request POST \
  --url https://api.timbal.ai/orgs/{org_id}/metrics/definitions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Tool errors",
  "metric_name": "tool_errors",
  "has_error": true,
  "value_source": "count",
  "label_source": "span_name"
}
'

{
  "created_at": "2023-11-07T05:31:56Z",
  "enabled": true,
  "id": 123,
  "label_source": "<string>",
  "metric_name": "<string>",
  "name": "<string>",
  "source": "trace_span",
  "value_source": "<string>",
  "has_error": true,
  "min_duration_ms": 123,
  "project_id": 123,
  "span_name": "<string>",
  "span_name_prefix": "<string>",
  "status_code": "<string>"
}

{
  "code": "<string>",
  "message": "<string>"
}

{
  "code": "<string>",
  "message": "<string>"
}

Authorizations

Authorization

string

header

required

Timbal API key. Obtain your API key from the Timbal platform settings. See Authentication for more information.

Path Parameters

org_id

string

required

Body

application/json

Body for POST /orgs/{org_id}/metrics/definitions.

Every predicate field is optional and they AND together. All omitted means "every non-root span", which paired with label_source: span_name gives a per-step call count — a reasonable first definition.

metric_name

string

required

Destination metric. trace. is prepended when missing.

name

string

required

Human label for the definition itself, unique within the org.

has_error

boolean | null

true = only errored spans, false = only clean ones, omitted = both.

label_source

string

none (default) or span_name to break the metric out per step.

min_duration_ms

integer<int32> | null

Only spans at least this slow. Spans still running never match.

project_id

string | null

Scope to one project. Omit to apply across the org.

source

enum<string>

Defaults to trace_span, the only source today.

Available options:

trace_span

span_name

string | null

Exact span name — the leaf of the span path (llm, get_address), not the full {workforce}.{step}.

span_name_prefix

string | null

Span-name prefix, for matching a family of tools (get_).

status_code

string | null

value_source

string

count (default), duration_ms, or usage:<key>.

Response

Definition created

created_at

string<date-time>

required

enabled

boolean

required

integer<int64>

required

label_source

string

required

none or span_name.

metric_name

string

required

name

string

required

source

enum<string>

required

Always trace_span today; see [MetricDefinitionSource].

Available options:

trace_span

value_source

string

required

count, duration_ms, or usage:<key>.

has_error

boolean | null

min_duration_ms

integer<int32> | null

project_id

integer<int64> | null

Project scope; omitted when the filter applies to the whole org.

span_name

string | null

span_name_prefix

string | null

status_code

string | null

List Metric Definitions Preview Metric Definition

API Documentation

Me

Organizations

Projects

Knowledge Bases

Evals

ACE

Alarms

Connectors

Create Metric Definition

Authorizations

Path Parameters

Body

Response