# Background Tasks
Source: https://docs.timbal.ai/agents/background-tasks
Execute long-running operations asynchronously
Background tasks let runnables execute asynchronously while the agent continues. When a tool runs in the background, it immediately returns a `task_id` and `status: "running"` instead of blocking until completion.
## Configuring Background Mode
The `background_mode` parameter controls when a runnable executes asynchronously:
* `"never"` (default) — always runs synchronously
* `"always"` — always runs in the background
* `"auto"` — the LLM decides per call by setting `run_in_background=True` on the tool input
```python theme={"dark"}
from timbal import Tool
long_running_tool = Tool(
handler=slow_operation,
background_mode="always",
)
```
The built-in `Bash` tool uses `background_mode="auto"` by default. The model can run a shell command in the background by passing `run_in_background=True`.
```python theme={"dark"}
from timbal import Agent
from timbal.tools import Bash
agent = Agent(
name="task_manager",
model="openai/gpt-4o-mini",
tools=[Bash("*")],
)
# Agent starts the command in the background and returns immediately
result = await agent(prompt="Install project dependencies in the background").collect()
```
## Checking Task Status
Background tools return immediately with a task handle:
```python theme={"dark"}
{"task_id": "a3k9f2", "status": "running"}
```
Once at least one background task is running, the agent automatically exposes a `get_background_task` tool. Call it with the `task_id` to poll status and drain queued events:
```python theme={"dark"}
from timbal.state import get_run_context
# After starting a background task...
span = get_run_context().current_span()
status = span.runnable.get_background_task("a3k9f2")
# {"status": "running" | "completed" | "error" | "cancelled" | "not_found", "events": [...], ...}
```
Or let the agent poll for you on a follow-up turn — it will see `get_background_task` in its tool list and can call it with the `task_id` from the earlier tool result.
# Commands
Source: https://docs.timbal.ai/agents/commands
Invoke tools, agents, and workflows directly with slash commands — no LLM round-trip
Commands let users trigger a runnable directly from a message that starts with `/`. The agent parses the message, maps arguments to the handler's parameters, runs the runnable immediately, and **skips the LLM call** for that turn.
This is useful for deterministic shortcuts in chat UIs, Slack bots, or internal tools where you already know which action to run.
## Basic usage
Set `command` on any `Tool`, `Agent`, or `Workflow` in the agent's `tools` list:
```python theme={"dark"}
from timbal import Agent, Tool
def greet(name: str) -> str:
return f"Hello, {name}!"
agent = Agent(
name="support_agent",
model="openai/gpt-4o-mini",
tools=[Tool(handler=greet, command="greet")],
)
result = await agent(prompt="/greet Alice").collect()
# result.output → "Hello, Alice!"
```
The command name is matched with or without a leading slash — both `/greet` and registering `command="greet"` work. Unknown commands fall through to the LLM as a normal message.
## Argument parsing
Arguments after the command name are split with `shlex` (shell-style quoting) and mapped **positionally** to the handler's parameters in signature order:
```python theme={"dark"}
def add(a: int, b: int) -> int:
return a + b
Tool(handler=add, command="add")
# /add 5 3 → add(a=5, b=3)
# /echo "hello world" → echo(message="hello world")
```
* Extra arguments are ignored
* Missing required arguments surface as normal validation errors on the tool
* Only messages with a **single text block** starting with `/` are treated as commands
## Agents and workflows as commands
Any runnable in `tools` can expose a command — including nested agents and workflows:
```python theme={"dark"}
search_agent = Agent(
name="search_agent",
model="openai/gpt-4o-mini",
command="search",
description="Search and summarize",
)
main_agent = Agent(
name="main_agent",
model="openai/gpt-4o-mini",
tools=[search_agent],
)
# Quote multi-word prompts so they map to a single argument
await main_agent(prompt='/search "python tutorials"').collect()
```
Workflows work the same way — arguments map to the workflow's combined parameter model (all step inputs in signature order):
```python theme={"dark"}
from timbal import Workflow
workflow = Workflow(name="pipeline", command="pipeline")
workflow.step(fetch_data)
workflow.step(process_data, depends_on=["fetch_data"])
agent = Agent(name="agent", model="openai/gpt-4o-mini", tools=[workflow])
await agent(prompt='/pipeline "quarterly report"').collect()
```
## Behavior
| Behavior | Detail |
| ------------------ | ---------------------------------------------------------------------------------------- |
| **No LLM call** | Matching commands execute the runnable directly — no tokens spent on tool selection |
| **Memory** | Command invocations are recorded in conversation memory as a synthetic tool use + result |
| **Unknown `/foo`** | Falls through to the LLM if `foo` is not a registered command |
| **Collisions** | Two tools with the same `command` name log a warning; the first registered wins |
## When to use commands
* **Chat shortcuts** — `/help`, `/status`, `/reset` that should always hit the same handler
* **Power users** — skip LLM routing for actions they already know by name
* **Nested specialists** — `/search`, `/billing` to invoke sub-agents without the parent model choosing tools
For LLM-driven tool selection (the default), see [Adding Tools](/agents/tools).
# Dynamic Agents
Source: https://docs.timbal.ai/agents/dynamic
Learn how to create agents with dynamic system prompts that update automatically using real time data
## Dynamic System Prompts
Agents support dynamic system prompts through callable functions that are executed each time the agent runs, providing fresh context.
### Using Callable Functions
The preferred way to create dynamic system prompts is to pass a callable directly to `system_prompt`. This gives you full control over the system prompt construction and access to all runtime inputs:
```python theme={"dark"}
from datetime import datetime
from timbal import Agent
from timbal.state import get_run_context
def get_system_prompt() -> str:
run_context = get_run_context()
current_span = run_context.current_span()
now = datetime.now()
date_str = now.strftime("%A, %B %d, %Y")
time_str = now.strftime("%H:%M")
system_prompt = f"You're a helpful assistant. Current date: {date_str}. Current time: {time_str}."
# Access runtime inputs
instructions = current_span.input.get("instructions", None)
if instructions:
system_prompt += f"\n\n## Instructions\n{instructions}"
user = current_span.input.get("user", None)
if isinstance(user, dict):
system_prompt += "\n\n## About the User\n"
for k, v in user.items():
if isinstance(v, list):
system_prompt += f"\n- {k}:"
for item in v:
system_prompt += f"\n - {item}"
else:
system_prompt += f"\n- {k}: {v}"
return system_prompt
agent = Agent(
name="dynamic_agent",
model="openai/gpt-4o-mini",
system_prompt=get_system_prompt, # Pass the function directly
)
```
Then call the agent with runtime data:
```python theme={"dark"}
response = await agent(
prompt="Who am I?",
instructions="Be concise and friendly.",
user={
"name": "Alice",
"role": "Developer",
"memories": [
"Prefers Python over JavaScript",
"Working on a new project",
],
},
).collect()
```
### Using Template Syntax
Template syntax will be deprecated in a future release. We recommend using callable functions instead.
For simpler cases, you can use `{module::function}` syntax to embed dynamic values:
```python theme={"dark"}
agent = Agent(
name="dynamic_agent",
model="openai/gpt-4o-mini",
system_prompt="""You are a time-aware assistant.
Current time: {datetime::datetime.now}."""
)
```
The previous example used a built-in function (datetime). You can also create your own custom functions:
```python title="my_functions.py" theme={"dark"}
def get_server_status():
"""Get server status."""
status = check_server() # Calls external function
return f"Server: {status}"
agent = Agent(
name="custom_agent",
model="openai/gpt-4o-mini",
system_prompt="""You are a helpful assistant.
Status: {my_functions::get_server_status}."""
)
```
You can also pass dynamic parameters to these functions using `RunContext` data that you previously set in the context.
```python title="my_functions.py" theme={"dark"}
from timbal import Agent, Tool
from timbal.state import get_run_context
def get_user_language():
span = get_run_context().current_span()
return span.input["language"]
def set_user_language(l):
span = get_run_context().current_span()
span.input["language"] = "catalan"
agent = Agent(
name="multilang_agent",
model="openai/gpt-4o-mini",
pre_hook=set_user_language,
system_prompt="Answer in {my_functions::get_user_language}."
)
await agent(prompt="Which is the capital of Germany?").collect()
```
The response will be in Catalan.
**Benefits:**
* Real-time context: System prompts reflect current state
* Dynamic behavior: Agent adapts to changing conditions
* Automatic execution: Functions run on each conversation
* Performance: Template resolution is fast and cached
* Sync/Async: Handles both sync and async functions automatically
## Dynamic Tools
Timbal provides the `ToolSet` class for dynamic tool resolution. ToolSets resolve tools at runtime before each LLM call, enabling dynamic tool availability based on execution context. Use ToolSets instead of static tool lists when:
* **Context-dependent availability**: Tools should only appear under certain conditions (user permissions, environment state, iteration count)
* **Lazy loading**: Defer tool initialization until actually needed
* **Dynamic configuration**: Tools need runtime parameters or state that isn't known at agent creation
* **Conditional behavior**: Tool availability changes during execution
* **Token efficiency**: Reduce token consumption by exposing only relevant tools instead of all available tools
* **Improved clarity**: When many tools exist but only a few are available per context, the agent sees fewer options and is less likely to get confused
Implement the `resolve()` method to return a list of tools. Access runtime data through `get_run_context()` to inspect the current execution state.
### Example: Role-based tool access
This example shows accessing input parameters to conditionally provide tools. The role can be set via prehook or when calling the agent:
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.core.tool_set import ToolSet
from timbal.state import get_run_context
class RoleBasedToolSet(ToolSet):
async def resolve(self) -> list[Tool]:
span = get_run_context().current_span()
role = span.input.get("role", "user")
if role == "admin":
return [
Tool(handler=view_profile),
Tool(handler=delete_user),
Tool(handler=modify_permissions)
]
else:
return [Tool(handler=view_profile)]
admin_agent = Agent(
name="admin_agent",
model="openai/gpt-4o-mini",
tools=[RoleBasedToolSet()]
)
# Role can be set via prehook or as a parameter
await admin_agent(prompt="Delete user 123", role="admin").collect()
```
The `resolve()` method is called before each LLM call. It reads the `role` from the input parameters and returns different tools:
* `role == "admin"`: returns `view_profile`, `delete_user`, `modify_permissions`
* Otherwise: returns only `view_profile`
The agent only sees the tools returned by `resolve()`, preventing unauthorized actions when the role is not "admin".
# Guardrails
Source: https://docs.timbal.ai/agents/guardrails
Content policy for the four edges of an agent run: input, model output, tool args, and tool results
Guardrails intercept content at the four edges of an agent run and decide what happens to it — block it, redact it, ask the model to try again, escalate to a human, or just record it.
```python theme={"dark"}
from timbal import Agent
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tools=[...],
guardrails="default", # PII redaction + secret redaction + prompt-injection blocking
)
```
The edges:
1. **`input`** — the user's message, checked **before the first LLM call**. A blocked input spends zero tokens and executes zero tools.
2. **`model_output`** — the final assistant response, checked before it reaches the user (stream-safe — see [Streaming](#streaming)).
3. **`model_step`** — opt-in: **every** assistant message, including intermediate tool-calling steps, not just the final response. Use for policies that must hold mid-plan (leaked codenames in reasoning prose, PII in intermediate text). LLM-backed rails here multiply classifier calls per turn — prefer deterministic rails.
4. **`tool_args`** — a tool call's validated arguments, checked before the tool runs. An `escalate` verdict converts into a [human approval gate](/human-in-the-loop/approval-gates).
5. **`tool_result`** — a tool's output, checked before it enters memory (and before [tool result offloading](/agents/memory-compaction#tool-result-offloading), so rails always see the full text).
## From one string to full control
### One string
```python theme={"dark"}
agent = Agent(name="a", model="openai/gpt-4o-mini", guardrails="default")
```
`"default"` is deterministic and free — no classifier calls: `DetectPII(action="redact")` + `RedactSecrets()` + `PromptInjection(action="block")`.
### Shorthands
```python theme={"dark"}
agent = Agent(..., guardrails=["pii:redact", "injection:block", "secrets", "moderation:warn"])
```
Each shorthand is `name` or `name:action`. Valid names: `pii`, `secrets`, `injection`, `keywords`, `moderation`, `length`, `topic`, `judge`. A typo raises immediately with the valid options.
### Configured built-ins
```python theme={"dark"}
from timbal.guardrails import DetectPII, Moderate, PromptInjection, TopicGuard
agent = Agent(
...,
guardrails=[
DetectPII(on_input="redact", on_output="block", types=["email", "credit_card", "ssn"]),
PromptInjection(action="block"),
Moderate(provider="openai", action="warn"),
TopicGuard(allow=["billing", "shipping"],
blocked_message="I can only help with billing and shipping."),
],
)
```
Per-stage actions live on the rail: `on_input=`, `on_output=`, `on_tool_args=`, `on_tool_result=` override the rail's default `action` (and implicitly opt the rail into that stage).
### Plain callables
```python theme={"dark"}
from timbal.guardrails import Verdict, guardrail
def no_competitors(text: str):
if "acme corp" in text.lower():
return Verdict.block("competitor mention")
return True
agent = Agent(..., guardrails=[guardrail(no_competitors, stages=["model_output"])])
```
Return-value coercion: `True`/`None` allow, `False` blocks, a `str` replaces the content, a `Verdict` gives full control. Sync or async; take `(text)` or `(text, ctx)`. `@guardrail(stages=[...])` works as a decorator.
### LLM judge in one line
```python theme={"dark"}
from timbal.guardrails import LLMJudge
LLMJudge("Response must not give medical advice", model="openai/gpt-5.4-nano", action="retry")
```
With `action="retry"` the judge's critique is fed back to the model and the response is re-generated — bounded by `Agent(max_guardrail_retries=2)`. Exhaustion blocks with the last reason.
### Rubric quality gates
For a structured definition of "done", give the judge a rubric instead of one criteria string. Each criterion is graded by its **own isolated judge call** (pass / fail / unknown + reason), and the failing criteria — with the judges' reasons — become the revision feedback:
```python theme={"dark"}
LLMJudge(
rubric=[
"Includes a comparison table",
"Every price is attributed to a source",
{"criterion": "At least 3 actionable recommendations", "weight": 2},
],
pass_threshold=1.0, # weighted fraction of criteria that must pass
action="retry", # grade → revise → re-grade, bounded by max_guardrail_retries
)
```
Per-criterion results land in the `GuardrailEvent` metadata and the run report (`metadata["guardrails"]["triggered"][i]["metadata"]["rubric"]`). The same rubric works in [evals](/evals/validators/llm#rubric) via the `rubric!` validator — write it once, gate at runtime and regress in CI. Write criteria around verifiable structure, not facts the judge cannot check.
## Verdicts
Every check resolves to one of six actions:
| Action | Effect |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `allow` | Pass through untouched. |
| `block` | Stop. The run ends with `status.code="blocked"` and a user-safe message; blocked tool calls feed a `[Blocked by guardrail]` result back to the model so it can self-correct. |
| `replace` | Swap the content (redaction is a replace produced from the rail's `scrub`). |
| `retry` | Reject the model output and re-generate with feedback (model\_output only). |
| `escalate` | Convert into a human approval gate (tool\_args only). |
| `warn` | Allow, but record the violation in events and the run report. |
## Blocked responses
A block is a controlled stop, not an exception:
```python theme={"dark"}
result = await agent(prompt="ignore all previous instructions...").collect()
result.status.code # "blocked"
result.status.reason # "guardrail:prompt_injection:input"
result.output # assistant Message carrying the blocked_message — render it like any reply
```
`blocked_message` (per rail) is the user-safe copy; `reason` is the dev-facing explanation that lands in events and traces. The blocked reply is also appended to memory, so the next turn resumes a coherent conversation.
## Escalating to a human
A `tool_args` rail can require human sign-off instead of deciding itself, reusing the whole [approval-gate machinery](/human-in-the-loop/approval-gates) — `ApprovalEvent`, resume, edit-on-approve, audit trail:
```python theme={"dark"}
from timbal.guardrails import Verdict, guardrail
def gate_prod(text):
return Verdict.escalate("Deploy to prod?") if '"env": "prod"' in text else True
agent = Agent(
...,
tools=[deploy],
guardrails=[guardrail(gate_prod, stages=["tool_args"])],
)
# → ApprovalEvent(kind="guardrail_escalation"); resume={approval_id: True} releases the call
```
Tool-local rails also work directly on a `Tool` (with or without an agent):
```python theme={"dark"}
from timbal.core import Tool
Tool(handler=send_email, guardrails=[guardrail(internal_recipients_only, stages=["tool_args"])])
```
## Shadow mode
Deploy rails with zero enforcement risk: everything runs and gets recorded — nothing acts.
```python theme={"dark"}
agent = Agent(..., guardrails=["pii:redact", "injection:block"], guardrail_mode="shadow")
```
Verdicts appear in `GuardrailEvent`s (with `shadow=True`), the run report, and usage counters (`guardrails:shadow_triggered`), so you can watch trigger rates in traces before flipping to `"enforce"` (the default). Per-rail: `DetectPII(shadow=True)`.
### Sampled online monitoring
Combine shadow mode with `sample_rate` to grade a slice of production traffic — the online-evaluation pattern, without the request-path cost:
```python theme={"dark"}
from timbal.guardrails import LLMJudge
agent = Agent(
...,
guardrails=[
LLMJudge(
rubric=["Answers the question directly", "Cites a source"],
shadow=True,
sample_rate=0.05, # grade ~5% of responses; verdicts land in traces
),
],
)
```
Sampled-out checks record nothing. `sample_rate` exists for shadow/`warn` monitoring: sampling an *enforcing* rail creates nondeterministic enforcement gaps (and buffer-until-verdict still engages on every run, because the streaming decision precedes the sampling roll) — configuring that logs a warning.
## Streaming
* Rails that only **redact** (deterministic detectors) transform text **and thinking** deltas **in flight**, with a per-content-block holdback window so patterns spanning chunk boundaries are still caught.
* Any rail that can **block / retry / escalate** forces **buffer-until-verdict**: deltas are withheld and replayed once the rails allow the message — a blocked response never leaks a single chunk. This trades streaming latency for enforcement; use `warn`/shadow rails if you need live streaming with observation only.
* Thinking blocks on stored messages are scrubbed by the redact rails too, so reasoning never carries PII into memory.
* `GuardrailEvent` is a first-class stream event, so UIs can show "response withheld" the moment a rail fires.
## Observability
Every triggered rail (including shadowed and crashed ones) produces:
* a **`GuardrailEvent`** in the stream: `{rail, stage, action, reason, latency_ms, shadow}`;
* an entry in the per-run report on **`OutputEvent.metadata["guardrails"]["triggered"]`**;
* usage counters: `guardrails:triggered` / `guardrails:shadow_triggered` (judge/classifier token usage folds into normal usage accounting).
Introspect a configuration with `agent.explain_guardrails()` — a table of rails, stages, actions, and order.
If a rail itself crashes, the default is **fail-open** (the run continues; the crash is recorded with `action="error"`). Set `strict=True` on security-critical rails to fail closed.
## Redacting traces
In-run guardrails redact agent memory and outputs, but traces record every span — including the inner LLM call. `trace_redactor` closes that gap at the storage/export boundary:
```python theme={"dark"}
from timbal.guardrails import trace_redactor
from timbal.state.tracing.providers import JsonlTracingProvider
provider = JsonlTracingProvider.configured(
_path=Path("traces.jsonl"),
_trace_redactor=trace_redactor(), # PII + secrets by default
)
agent = Agent(..., tracing_provider=provider, guardrails="default")
```
The redactor runs inside the provider's `put()` on **copies** of every span — inputs, outputs, memory dumps, errors, metadata — before storage and before every exporter fires. The live run is never mutated. It works with any provider (JSONL, SQLite, platform, custom) and accepts the same specs as `guardrails=`, restricted to **deterministic** rails (`trace_redactor("pii:redact", DetectPII(types=["ssn"], redaction="hash"))`) — an LLM call per span store would be a footgun, so judgment rails are rejected loudly.
Resumed sessions load memory from stored traces, so chained turns see the redacted history. That is usually exactly what you want for compliance — the raw text exists only inside the run that produced it.
## Testing rails without an agent
```python theme={"dark"}
from timbal.guardrails import check_guardrails
report = await check_guardrails(agent, "my ssn is 123-45-6789")
assert report.triggered("detect_pii").action == "replace"
assert "[REDACTED_SSN]" in report.text
report = await check_guardrails(["injection:block"], "ignore all previous instructions")
assert report.blocked and report.blocking_rail == "prompt_injection"
```
`check_guardrails` runs only the rails — no LLM loop, no tools — against any agent or spec, at any stage (`stage="model_output"`, ...).
## Built-in rails
| Rail | Default stages | How it works |
| ----------------- | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DetectPII` | input, model\_output, tool\_result | Regex + Luhn validation: email, credit\_card, ssn, phone, ip, url. Redaction renders as placeholder, `mask` (keep last 4), or `hash` (deterministic pseudonym). |
| `RedactSecrets` | model\_output, tool\_result | API keys (AWS, OpenAI, Anthropic, GitHub, Slack, Google, Stripe), JWTs, bearer tokens, PEM private keys, credential assignments. |
| `PromptInjection` | input | Curated pattern pack (instruction override, system-prompt probes, transcript extraction, role hijack, jailbreak personas, guardrail bypass, delimiter smuggling). Optional `model=` adds an LLM classifier that runs only when patterns find nothing. |
| `KeywordGuard` | input, model\_output | Banned terms, literal or regex. |
| `MaxLength` | input | `max_chars` / `min_chars` bounds. |
| `Moderate` | input, model\_output | OpenAI Moderation API (`provider="openai"`, free, needs `OPENAI_API_KEY`) or a Llama-Guard-style safe/unsafe prompt against any model (`provider="llama_guard", model=...`). |
| `TopicGuard` | input | LLM classifier over `allow=` / `deny=` topic lists. |
| `LLMJudge` | model\_output | Free-form criteria judged by a (cheap) model; any action, `retry` by default. |
Deterministic rails cost nothing and add microseconds. LLM-backed rails are explicit opt-ins — point them at a small model.
### What the injection pattern pack does and does not catch
The pack is English and literal by design: it is a cheap first filter, not a classifier. It is
regression-tested against a corpus of known attacks and benign lookalikes, and it deliberately
does **not** fire on ordinary phrasing that shares its vocabulary ("print the instructions for
the desk", "remove the safety guard from my lawnmower").
It does not catch non-English attacks, base64 or character-separated obfuscation, or
paraphrases that avoid the keywords entirely. Those need the classifier:
```python theme={"dark"}
PromptInjection(model="openai/gpt-5.4-nano") # patterns first; classifier only when they find nothing
```
Treat either as defence in depth, not a boundary: the durable mitigations are least-privilege
tools, `tool_args` rails, and approval gates on anything destructive.
Order matters. Every rail is checked against the text as of **its position in the list**, so a rail placed after a redactor sees the redacted text — put normalizing/redacting rails before judging rails and the judges never receive the raw content. The first non-allow verdict in list order wins. Adjacent rails that cannot rewrite the text are checked concurrently for latency; a `redact`/`retry` rail is a barrier. A custom rail that returns replacement text must declare `action="redact"`, otherwise it raises (batching is decided from the action, so an undeclared rewrite would be an invisible ordering bug).
## Known limitations
* `model_output` rails judge the **final** assistant message. For intermediate tool-calling text, opt into the `model_step` stage (every step) or rely on the `tool_args` / `tool_result` edges and in-flight scrubbing.
* `retry` verdicts regenerate final responses only; a `retry` fired mid-plan (on a tool-calling step) coerces to block rather than corrupting the tool loop.
* `trace_redactor` accepts deterministic rails only, and redacted traces are what resumed sessions load as history.
# Overview
Source: https://docs.timbal.ai/agents/index
Master proven strategies for designing advanced, specialized AI agents using an architecture that work together seamlessly to tackle complex challenges
## What are Agents?
Agents are autonomous execution units that **orchestrate LLM interactions with tool calling**.
Without tools, an agent functions as a basic LLM. The simplest agent requires just a **name** and **model**:
```python theme={"dark"}
from timbal import Agent
agent = Agent(
name="my_agent",
model="openai/gpt-5"
) # That's it! You've created your first agent!
```
For a full step-by-step guide, check the [quickstart section](/quickstart) or see practical examples in the [examples section](/examples).
## Reading guide
The sidebar follows a **basics → platform → production** path. You can jump anywhere, but this order matches how most agents are built:
1. **[Tools](/agents/tools)** — give the agent capabilities (the core primitive)
2. **[Structured output](/agents/structured-output)** — constrain what comes back
3. **[Memory](/agents/memory)** → **[Memory compaction](/agents/memory-compaction)** — multi-turn context, then keeping oversized tool results and history within the window
4. **[Skills](/agents/skills)** — domain packages (knowledge + tools) once the agent loop makes sense
5. **[Dynamic agents](/agents/dynamic)** — runtime prompts and tool sets
6. **[Background tasks](/agents/background-tasks)** · **[Commands](/agents/commands)** — utilities (async tools, slash shortcuts)
After **[Workflows](/workflows)**, see **[Human in the loop](/human-in-the-loop)** for approvals, `suspend()`, and durable resume (agents, workflow steps, and tools).
This page covers running agents, models, and I/O. The sections below are the reference for that; the linked pages go deeper on each topic.
## Model Providers
You can specify any model using the "provider/model" format. See all supported models in the [Model Reference](/models/overview).
Some models require specific parameters (like `max_tokens` for Claude). Pass it as a top-level field:
```python highlight={4} theme={"dark"}
agent = Agent(
name="claude_agent",
model="anthropic/claude-sonnet-4-6",
max_tokens=1024
)
```
**Note:** Make sure to define all required environment variables—such as the API key model that you need—in your `.env` file.
```bash theme={"dark"}
OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_claude_api_key
```
### Fallback Models
Use `FallbackModel` when you want an agent to try another model if the primary provider is rate limited, temporarily unavailable, times out, or returns a retryable server error.
```python theme={"dark"}
from timbal import Agent, FallbackModel
agent = Agent(
name="support_agent",
model=FallbackModel(
"anthropic/claude-sonnet-4-6",
"openai/gpt-5.5",
"google/gemini-2.5-pro",
),
max_tokens=1024, # Required for Anthropic models
)
```
Models are tried in order. By default, each model gets two retries before Timbal moves to the next model in the chain. If every model fails with a retryable provider error, Timbal raises `FallbackExhausted` with the per-model errors.
For per-model settings, use `ModelEntry`:
```python theme={"dark"}
from timbal import Agent, FallbackModel, ModelEntry
agent = Agent(
name="support_agent",
model=FallbackModel(
ModelEntry(
"anthropic/claude-sonnet-4-6",
max_retries=3,
retry_delay=0.5,
),
ModelEntry(
"openai/gpt-5.5",
max_retries=1,
),
),
max_tokens=1024,
)
```
Fallback only switches models before the first streamed chunk is emitted. If a stream fails after output has started, Timbal raises the error instead of silently switching models and risking duplicated or inconsistent output.
### Thinking and reasoning
Some models support extended thinking before responding. Check per-model capabilities on the [Model Reference](/models/overview) pages.
Configure at construction time via `model_params` (per-request overrides use `provider_params` — see [Overriding Model Configuration](#overriding-model-configuration) below):
```python theme={"dark"}
# Anthropic — max_tokens required; budget_tokens must be < max_tokens
agent = Agent(
name="reasoning_agent",
model="anthropic/claude-sonnet-4-6",
max_tokens=20000,
model_params={"thinking": {"type": "enabled", "budget_tokens": 10000}},
)
# OpenAI
agent = Agent(
name="reasoning_agent",
model="openai/gpt-5",
model_params={"reasoning": {"effort": "high", "summary": "auto"}},
)
```
Define tools as Python functions - the framework handles schema generation, parameter validation, and execution orchestration.
## Running Agents
Execute agents by calling them with a `prompt` parameter and using `.collect()` to get the result:
```python theme={"dark"}
response = await agent(
prompt="What is the capital of Germany?"
).collect()
```
### Streaming Events
For real-time processing, you can stream events as they happen:
```python theme={"dark"}
async for event in agent(prompt="Hello"):
print(event)
```
### Approval-Required Tools
Any runnable — Tool, Agent, or Workflow step — can pause for human approval before it runs. Mark it with `requires_approval`, listen for `ApprovalEvent`, and resume by calling the runnable again with `resume`:
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.types.events import ApprovalEvent
refund = Tool(
handler=lambda amount: f"refunded ${amount}",
requires_approval=lambda amount: amount > 100,
approval_prompt=lambda amount: f"Approve refunding ${amount}?",
)
agent = Agent(name="support_agent", model="openai/gpt-5", tools=[refund])
approval_id = None
async for event in agent(prompt="Refund $250"):
if isinstance(event, ApprovalEvent):
approval_id = event.approval_id
result = await agent(
prompt="Refund $250",
resume={approval_id: True},
).collect()
```
The full reference — durable cross-process resume, audit fields, redaction, parallel gates, `pending_approvals()`, status reasons, usage counters, plus `suspend()` for asking the user mid-run — lives on the dedicated [Human in the Loop](/human-in-the-loop) section.
## Input
Agents communicate through `Message` objects - Timbal's data structure that standardizes both input and output.
```python highlight={4} theme={"dark"}
from timbal.types.message import Message
response = await agent(
prompt=Message.validate("What is the capital of Germany?")
).collect()
```
Agents accept multiple input formats, automatically converting them to `Message` objects:
````python highlight={3, 9, 14} theme={"dark"}
from timbal.types.file import File
# String
response = await agent(
prompt="What's the weather?"
).collect()
# File - Timbal type
response = await agent(
prompt=File.validate("image.png")
).collect()
# List
response = await agent(
prompt=["Describe this image", File.validate("image.png")]
).collect()
### Passing Input Parameters
You can pass custom input parameters when calling an agent. Access these values anywhere in your agent (tools, system prompts, hooks) using `get_run_context().current_span().input`:
```python
from timbal import Agent, Tool
from timbal.state import get_run_context
def get_user_info():
span = get_run_context().current_span()
user_id = span.input.get("user_id")
role = span.input.get("role")
return f"User {user_id} with role {role}"
agent = Agent(
name="user_agent",
model="openai/gpt-4o-mini",
tools=[Tool(handler=get_user_info)]
)
# Pass custom inputs
response = await agent(
prompt="Who am I?",
user_id="123",
role="admin"
).collect()
````
The tool can access `user_id` and `role` from the input parameters. Input parameters work with both `.collect()` and streaming.
For more information about accessing input parameters and using the run context, see the [Context & State Management](/core-concepts/context) page.
### Overriding Model Configuration
What if you want to change the model, `max_tokens`, or thinking config for each run? Instead of creating multiple agents, you can pass these as input parameters. This is useful for A/B testing different models, adjusting token limits per request, or dynamically selecting models based on task complexity.
```python theme={"dark"}
# Agent with default values
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini", # Default model
max_tokens=1024 # Default max_tokens
)
# Override model: was "openai/gpt-4o-mini", now "anthropic/claude-sonnet-4-6"
response = await agent(
prompt="What is the capital of Germany?",
model="anthropic/claude-sonnet-4-6"
).collect()
# Override max_tokens: was 1024, now 2048
async for event in agent(
prompt="What is the capital of Germany?",
max_tokens=2048
):
print(event)
# Override thinking via provider_params at call time (provider-specific format).
# At construction time, use model_params instead — same keys, same shape.
# For OpenAI: pass "reasoning" key
response = await agent(
prompt="Solve this complex problem",
model="openai/gpt-5.5",
provider_params={"reasoning": {"effort": "high", "summary": "auto"}}
).collect()
# For Anthropic: pass "thinking" key
response = await agent(
prompt="Solve this complex problem",
model="anthropic/claude-sonnet-4-6",
provider_params={"thinking": {"type": "enabled", "budget_tokens": 10000}}
).collect()
```
The parameter names `model`, `max_tokens`, and `provider_params` are reserved and will affect model configuration when passed as input. These parameters will not be available as regular input to your agent. If you need to pass custom data without changing the actual model configuration, use different parameter names (e.g., `data_model` instead of `model` if you want to pass a data model name).
## Output
Calling `.collect()` returns an `OutputEvent` containing the agent's response. Access the `Message` via the `.output` property:
Learn more about events in [Events & Streaming](/core-concepts/events).
```python theme={"dark"}
result = await agent(prompt="What's 2+2?").collect()
# result is an OutputEvent
print(result.output)
# Message(role=assistant, content=[TextContent(type='text', text='2 + 2 = 4.')])
# You can access content directly
print(result.output.content[0].text)
# "2 + 2 = 4."
# However, the best way to get text is using collect_text()
print(result.output.collect_text())
# "2 + 2 = 4."
```
**Important:** When using models with thinking enabled, the content array structure changes:
* `content[0]` will contain the thinking/reasoning content
* `content[1]` will contain the actual text response
Directly accessing `result.output.content[0].text` may return thinking content instead of the response text. Always use `collect_text()` to reliably extract the text response, regardless of whether thinking is enabled.
## Messages
Messages are the structured data format that agents use to communicate. They contain a role and content, with automatic handling of different content types and provider compatibility.
```python theme={"dark"}
from timbal.types.message import Message
```
Messages contain a **role** and **content**:
**Role Types:**
* **user** - Messages from the user
* **assistant** - Messages from the AI agent
* **system** - System instructions and context
* **tool** - Tool execution results
**Content Types:**
* **TextContent** - Plain text messages
* **FileContent** - Files like PDFs, images, documents
* **ToolUseContent** - Function calls to tools
* **ToolResultContent** - Results from tool executions
Messages can contain different types of content - text, files, tool calls, and tool results. The framework automatically handles complex content structures:
```python theme={"dark"}
from timbal.types.content import FileContent, TextContent
from timbal.types.file import File
# Message with text and file
mixed_message = Message(
role="user",
content=[
TextContent(text="Analyze this document:"),
FileContent(file=File.validate("report.pdf"))
]
)
```
The same message above can be created easily using `Message.validate()`:
```python theme={"dark"}
from timbal.types.file import File
message = Message.validate([
"Summarize this document:",
File.validate("quarterly_report.pdf")
])
```
## Files
Agents can process files directly through the message content system. The framework automatically handles file reading, content extraction, and formatting for the AI model.
```python theme={"dark"}
from timbal.types.file import File
```
The framework supports common document and media formats:
* **Text files** (.txt, .md) - Direct content inclusion
* **PDFs** (.pdf) - Text extraction with structure preservation
* **Images** (.png, .jpg, .gif) - Visual analysis through vision-capable models
* **Spreadsheets** (.xlsx, .csv) - Structured data representation
* **Documents** (.docx) - Text and formatting extraction
Files are automatically converted to Timbal `File` objects using `File.validate()`:
```python theme={"dark"}
file = File.validate("quarterly_report.pdf")
```
# MCP Servers
Source: https://docs.timbal.ai/agents/mcp
Connect any Model Context Protocol server to your agents and use its tools like native Timbal tools
## What is MCP?
[Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) is an open standard that enables communication between AI models and external tools and services. Thousands of MCP servers exist for filesystems, databases, browsers, SaaS APIs, and more.
Timbal agents can connect to any MCP server with `MCPServer`. The server's tools are discovered at runtime and exposed to the LLM exactly like native tools — schemas, streaming, tracing, and error handling included.
This page covers **consuming MCP servers from your agents**. For the reverse direction — connecting your editor to the Timbal platform's MCP server — see [MCP Integration](/mcp-integration).
## Basic Usage
Add an `MCPServer` to the agent's tools list. Two transports are supported:
```python theme={"dark"}
from timbal import Agent
from timbal.core import MCPServer
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tools=[
# stdio: spawn a local server process
MCPServer(
name="filesystem",
transport="stdio",
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem", "."],
),
# http: connect to a remote server (streamable HTTP)
MCPServer(
name="timbal",
transport="http",
url="https://api.timbal.ai/mcp",
),
],
)
```
That's it — before each LLM call, the agent lists the server's tools and offers them to the model alongside any other tools. When the model calls one, the arguments are forwarded to the server and the result comes back as a tool result.
With `name=` set, a tool declared as `list_files` on the `filesystem` server appears to the LLM as `filesystem__list_files`. The bare name is still used on the wire.
## Configuration
| Field | Transport | Description |
| ----------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name` | both | Optional identifier. When set, tools are exposed as `{name}__{tool}` so multiple servers don't collide in the agent's flat tool registry. Recommended whenever you attach more than one server (and required by the [codegen CLI](#codegen-cli)). |
| `transport` | — | `"stdio"` or `"http"` (required) |
| `command` | stdio | Executable to spawn (e.g. `"npx"`, `"uvx"`, `"python"`) |
| `args` | stdio | Command arguments |
| `env` | stdio | Environment variables for the spawned process |
| `url` | http | Server URL |
| `headers` | http | HTTP headers sent with every request |
### Authentication
For servers that require auth, pass headers (http) or environment variables (stdio). Read secrets from the environment instead of hardcoding them:
```python theme={"dark"}
import os
timbal_mcp = MCPServer(
name="timbal",
transport="http",
url="https://api.timbal.ai/mcp",
headers={"Authorization": f"Bearer {os.environ['TIMBAL_API_KEY']}"},
)
github_mcp = MCPServer(
name="github",
transport="stdio",
command="npx",
args=["-y", "@modelcontextprotocol/server-github"],
env={"GITHUB_PERSONAL_ACCESS_TOKEN": os.environ["GITHUB_TOKEN"]},
)
```
## How Results Are Handled
MCP tool results are converted so the LLM always receives something useful:
* **Text content** becomes a plain string (or a list of strings for multiple blocks)
* **Images, audio, and binary resources** become Timbal [`File`](/core-concepts/runnables#files)s, forwarded to the model as file content — vision models can see returned images directly
* **Structured content** is returned as-is when the server sends no text representation
* **Errors** (`isError`) raise inside the tool call, so the model receives a proper error tool result and can self-correct
## Connection Lifecycle
Connections are **lazy**: nothing is spawned or contacted until the agent first needs the server's tools. The tool list is fetched once and cached for the life of the connection.
Close the connection when you're done:
```python theme={"dark"}
server = MCPServer(transport="stdio", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", "."])
agent = Agent(name="my_agent", model="openai/gpt-4o-mini", tools=[server])
result = await agent(prompt="List the files in the current directory").collect()
await server.close()
```
For long-running processes (servers, bots), create the `MCPServer` once and reuse it across runs — reconnecting per request adds latency, especially for stdio servers that spawn a subprocess.
## Dynamic Resolution
`MCPServer` is a [`ToolSet`](/agents/dynamic#dynamic-tools): tools are resolved at runtime, not import time. This means the available tools always reflect what the server currently offers, and you can mix MCP servers freely with functions, built-in tools, and other agents in the same `tools` list.
## Codegen CLI
The [codegen CLI](https://github.com/timbal-ai/timbal/tree/main/python/timbal/codegen) can add MCP servers to an agent's source file:
```bash theme={"dark"}
# stdio server
python -m timbal.codegen add-mcp --name fs \
--command npx --args '["-y", "@modelcontextprotocol/server-filesystem", "."]'
# http server — $VARS are emitted as os.environ lookups, never hardcoded
python -m timbal.codegen add-mcp --name timbal \
--url https://api.timbal.ai/mcp \
--headers '{"Authorization": "Bearer $TIMBAL_API_KEY"}'
# import a standard mcpServers JSON config (claude-desktop / cursor style)
python -m timbal.codegen add-mcp --from-json @mcp.json
```
Re-running `add-mcp` with the same `--name` replaces the server spec. Remove a server with `remove-tool --name ` — the assignment and import are cleaned up automatically.
# Memory
Source: https://docs.timbal.ai/agents/memory
Understand and manage how Agent memory works
Memory in Timbal agents enables multi-turn conversations and context persistence across agent interactions. **Agents automatically maintain conversation history without requiring additional configuration**.
## How Memory Works
Timbal implements memory through its [tracing system](/core-concepts/tracing), which captures conversation history during agent execution. When an agent runs, it automatically resolves memory from previous interactions to maintain conversational context.
### Automatic Memory Resolution
During each agent execution:
1. The agent checks for previous conversation context
2. If found, it retrieves conversation history from the tracing data
3. Previous messages are automatically included in the current conversation
4. The agent processes the new input with full conversation context
This happens transparently - agents receive conversation memory without any code changes.
### Storage Options
Memory storage depends on your deployment environment:
* **Timbal Platform**: Conversation history is automatically persisted with high availability and cross-instance sharing
* **Local Development**: Uses in-memory storage that's fast but cleared on restart
## Nested Agent Memory
When an agent is used as a tool of another agent, the **child gets an isolated context** — it does not inherit the parent's conversation history. The parent only sees what the child returns as its tool result. That keeps specialist agents focused and prevents parent history from bloating every nested call.
```python theme={"dark"}
from timbal import Agent
billing_agent = Agent(
name="billing_agent",
model="anthropic/claude-haiku-4-5",
# Fresh context per call — only the tool prompt from the parent, not the full chat
)
support_agent = Agent(
name="support_agent",
model="anthropic/claude-sonnet-4-6",
tools=[billing_agent], # add more specialist agents as needed
)
```
To hand the child specific context, put it in the tool call (the parent's prompt / args). To share state across turns of the *parent*, use the parent's own memory via `parent_id` session chaining — see [Rewind](#rewind) and [Tracing](/core-concepts/tracing).
## Configuration
Memory is automatically configured based on your deployment:
* **Platform Deployment**: No configuration needed
* **Self-Hosted**: Uses in-memory storage by default, with platform integration available
Memory works automatically. For deeper understanding of the underlying mechanisms, see [Tracing](/core-concepts/tracing) and [Context](/core-concepts/context).
## Rewind
Timbal supports conversation branching, allowing you to rewind to any previous point and explore alternative conversation paths.
### How Branching Works
Each agent interaction creates a unique run with its own context. You can branch from any previous run by referencing its `run_id`, creating independent conversation paths that diverge from that point.
```python theme={"dark"}
from timbal import Agent
from timbal.state import RunContext, set_run_context
agent = Agent(name="example", model="openai/gpt-4o-mini")
# Main conversation
step1 = await agent(prompt="Hello").collect()
step2 = await agent(prompt="My name is David").collect()
step3 = await agent(prompt="What's my name?").collect()
# Output: "Your name is David"
# Branch from step 1 (before name was shared)
context = RunContext(parent_id=step1.run_id)
set_run_context(context)
branch = await agent(prompt="What's my name?").collect()
# Output: "I don't know your name"
```
This creates branching conversations:
```
"Hello"
├── "My name is David" → "What's my name?" → "David"
└── "What's my name?" → "I don't know"
```
### Common Use Cases
* **A/B Testing**: Compare different conversation strategies
* **Error Recovery**: Return to a state before an error occurred
* **Debugging**: Isolate specific conversation states for testing
* **Exploration**: Test "what if" scenarios without affecting the main conversation
Each branch maintains independent memory from the branching point. Learn more about the underlying mechanisms in [Context](/core-concepts/context).
## Keeping memory within the context window
For long-running conversations, memory can grow large enough to exceed the model's context window. Timbal has two complementary layers:
1. **[Tool result offloading](/agents/memory-compaction#tool-result-offloading)** — oversized tool results are spilled to a store *when produced*, so they never dominate every later LLM call. Lossless; the model pages content back via `read_tool_result`.
2. **[Memory compaction](/agents/memory-compaction)** — strategies (keep last N turns, shrink old tool results, LLM summarize) that fire when context utilization crosses `memory_compaction_ratio` (default 75%).
Most long-running agents want both: offloading prevents the bloat at the source; compaction handles everything else. See [Memory Compaction](/agents/memory-compaction) for the full reference.
# Memory Compaction
Source: https://docs.timbal.ai/agents/memory-compaction
Keep agent memory within context window limits with tool result offloading and built-in compaction strategies
As conversations grow longer, the accumulated message history can exceed a model's context window. Timbal keeps memory bounded with two complementary layers, both configured on the agent and applied without changing how you call it:
1. **[Tool result offloading](#tool-result-offloading)** — oversized tool results are moved out of the window *the moment they are produced*, losslessly: the payload goes to a store, and the model keeps a preview plus a handle it can page back on demand.
2. **[Compaction strategies](#how-compaction-works)** — when context utilization crosses a threshold, the history already inside the window is trimmed or condensed (drop old tool results, keep the last N turns, summarize with an LLM).
Most long-running agents want both: offloading prevents the bloat at the source; compaction handles everything else.
## Tool Result Offloading
A single tool call can return hundreds of kilobytes — a big file read, a verbose log, a large API response. Because tool results persist in conversation memory, an oversized result is re-sent on **every subsequent LLM call**, paying its token cost for the rest of the run.
Tool result offloading intercepts a result when it is produced. If it exceeds a size threshold, the full payload is persisted to an offload store and the model sees a short placeholder instead:
```python theme={"dark"}
from timbal import Agent
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tools=[search_docs, run_query],
tool_result_limit=20_000, # offload results over 20,000 chars
)
```
Results under the threshold pass through untouched. Oversized ones are replaced with a placeholder like:
```text theme={"dark"}
[Tool result offloaded: 184,203 chars from 'search_docs'. The full content was saved and
can be read with read_tool_result(handle="06a7.../t1") — page with offset/limit or filter
with pattern.]
Shape: {"results": list[92], "total": int}
Preview (first 1,000 of 184,203 chars):
...
```
The reduction happens **once**, before the result ever enters memory, the serialized trace, or a provider request:
* **Lossless** — unlike truncation or summarization, the full payload stays retrievable. The model decides whether it needs line 500 of that output.
* **Prompt-cache friendly** — history stays append-only. The oversized payload never occupies a cached prefix that would later be rewritten (rewriting old messages invalidates provider prompt caches from that point on).
* **Persistent** — the placeholder is what gets traced and what seeds the next turn's memory. The reduction is never recomputed per request.
### Configuration
`tool_result_limit` accepts an int (threshold shorthand) or a `ToolResultLimit`:
```python theme={"dark"}
from timbal.core import LocalOffloadStore, Spill, ToolResultLimit, Truncate
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tool_result_limit=ToolResultLimit(
threshold=20_000, # chars of text content (default 20,000)
action=Spill(preview_chars=1_000), # what to do when the threshold is reached
store=LocalOffloadStore(), # where spilled payloads live
),
)
```
**`Spill`** (default) — persist the full payload, keep a preview + handle inline. Lossless. If the store write fails, `fallback` (default `Truncate()`) applies so an oversized result never slips through:
```python theme={"dark"}
Spill(preview_chars=1_000, fallback=Truncate(max_chars=2_000))
```
**`Truncate`** — clamp to a character budget. Lossy, zero-cost, no store needed:
```python theme={"dark"}
Truncate(strategy="head") # keep the start — headers, schemas
Truncate(strategy="tail") # keep the end — build/test output where errors land last
Truncate(strategy="head_tail") # keep both ends, elide the middle (default)
```
### Per-tool overrides
`Tool(result_limit=...)` overrides the agent default. `None` exempts a tool entirely:
```python theme={"dark"}
from timbal.core import Tool, ToolResultLimit, Truncate
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tool_result_limit=20_000, # agent-wide default: spill over 20k
tools=[
Tool(name="run_tests", handler=run_tests,
result_limit=ToolResultLimit(threshold=8_000, action=Truncate(strategy="tail"))),
Tool(name="load_policy", handler=load_policy, result_limit=None), # never reduced
],
)
```
Some results are never reduced, regardless of configuration:
* **Error results** — the model needs the full error to recover.
* **Pinned tools** (`pin_result=True`) — durable context like [skill](/agents/skills) documentation. See [Pinned results](#pinned-results).
* **`read_tool_result`'s own output** — it is bounded by construction.
### Reading content back
When offloading is active, the agent auto-registers a `read_tool_result` tool:
```python theme={"dark"}
read_tool_result(handle, offset=0, limit=200, pattern=None)
```
* Line-numbered paging via `offset`/`limit` (hard-capped at 500 lines / 50,000 chars per call, so a read can never blow the window back up).
* `pattern` filters to lines containing a **literal substring** (not a regex).
* Unknown or expired handles return a clean tool error the model can react to.
The model uses it on its own — the placeholder tells it how.
### The offload store
Spilled payloads go through a narrow protocol, so the backend is swappable:
```python theme={"dark"}
class OffloadStore(Protocol):
async def write(self, key: str, data: bytes) -> str: # returns a handle
async def read(self, handle: str) -> bytes:
```
The default `LocalOffloadStore` writes one file per result under `~/.timbal/offload/` (owner-only permissions; reads are hardened against path traversal and symlink escapes). Handles are store-relative keys, not absolute paths, so a different backend can resolve the same handle in another process.
Payloads are kept **forever by default** — deleting on run end would break a resumed or chained session that still holds handles. Bound disk usage with opt-in age-based pruning:
```python theme={"dark"}
from datetime import timedelta
from timbal.core import LocalOffloadStore, ToolResultLimit
store = LocalOffloadStore(cleanup_after=timedelta(days=7))
agent = Agent(..., tool_result_limit=ToolResultLimit(store=store))
```
Pruning runs off the hot path and never fails a run.
## How Compaction Works
Compaction is triggered automatically when context-window utilization exceeds `memory_compaction_ratio` (default `0.75`, i.e. 75%), at two points:
* **Turn start** — before the first LLM call of a turn, utilization is estimated from the **previous run's** token usage. The configured compactors are applied to the resolved memory before it reaches the LLM.
* **Mid-loop** — between iterations *within* a turn (after each tool round), utilization is read from the **previous LLM call's** reported usage. This bounds a single turn that makes many or large tool calls instead of letting it grow until it overflows. Mid-loop compaction always protects the most recent, still-unconsumed assistant tool batch (the results the next model call must read), so it never sends the agent back to re-plan the same step.
Both use the same strategies and the same ratio; you don't configure them separately.
```python theme={"dark"}
from timbal import Agent
from timbal.core.memory_compaction import keep_last_n_turns
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=keep_last_n_turns(10),
memory_compaction_ratio=0.75, # trigger when previous run used >75% of context
)
```
Set `memory_compaction_ratio=0.0` to always compact, or `1.0` to effectively disable auto-triggering.
Offloading and compaction compose cleanly: offloaded placeholders are small, so they push utilization down and compaction triggers later or never; `compact_tool_results` keeps offloaded placeholders intact by default so their handles stay dereferenceable; and `summarize` writes its canonical record to the same store, readable back through the same `read_tool_result` tool.
## Built-in Strategies
Import strategies from `timbal.core.memory_compaction`.
### `keep_last_n_turns(n)`
Keeps only the last `n` user/assistant turn pairs. Structure-aware: never leaves orphaned tool calls or results.
```python theme={"dark"}
from timbal.core.memory_compaction import keep_last_n_turns
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=keep_last_n_turns(5),
)
```
### `keep_last_n_messages(n)`
Keeps only the last `n` messages regardless of role. Also structure-aware.
```python theme={"dark"}
from timbal.core.memory_compaction import keep_last_n_messages
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=keep_last_n_messages(20),
)
```
### `compact_tool_results(...)`
Reduces the size of tool call history. Useful when tools return large payloads that are no longer needed verbatim.
```python theme={"dark"}
from timbal.core.memory_compaction import compact_tool_results
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=compact_tool_results(
keep_last_n=2, # keep last 2 tool call pairs intact
threshold=10, # only apply when memory exceeds 10 messages
replacement="[Tool '{tool_name}' result truncated ({result_length} chars)]",
),
)
```
**`replacement` controls what happens to compacted tool results:**
* `None` (default) — drop tool results and their corresponding `tool_use` entries entirely. Assistant messages that become empty are also dropped.
* `str` — replace each result with a template string. Supported placeholders: `{tool_name}`, `{call_id}`, `{result_length}`, and `{handle}` (the offload handle when the result was [offloaded](#tool-result-offloading), empty otherwise).
* `callable(tool_name, call_id, result_text) -> str` — call a function per result and use the return value as the replacement.
**`keep_offloaded`** (default `True`) — results already [offloaded at production time](#tool-result-offloading) are kept intact: they are small placeholders whose handle keeps the full payload reachable. Set `False` to compact them like any other result.
```python theme={"dark"}
# Drop all tool results
compact_tool_results()
# Replace with a short summary string
compact_tool_results(replacement="[{tool_name}: {result_length} chars]")
# Custom replacement logic
def shorten(tool_name, call_id, result_text):
return f"[{tool_name}]: {result_text[:100]}..."
compact_tool_results(replacement=shorten)
```
### `summarize(...)`
Summarizes old messages into a single context message using an LLM call. Uses **incremental summarization**: on subsequent runs, only the new overflow messages are sent to the summarizer, which updates the existing summary rather than regenerating it from scratch.
System messages are always preserved and never included in summarization.
```python theme={"dark"}
from timbal.core.memory_compaction import summarize
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=summarize(
threshold=20, # summarize when non-system messages exceed 20
model="openai/gpt-4o-mini", # model used for summarization (defaults to agent's model)
keep_last_n=4, # keep last 4 messages unsummarized
max_summary_tokens=500,
),
)
```
Use a smaller, cheaper model for summarization to reduce cost. For example, `model="openai/gpt-5.4-nano"` works well for summarization tasks.
The summary message is **sectioned**, and everything except the LLM summary itself is assembled mechanically — never paraphrased by the summarizer:
* **Verbatim User Messages** — the user's own words from the summarized region, carried forward verbatim across passes (`preserve_user_messages=True`, default). Summaries that drop or mischaracterize user instructions are the most damaging compaction failure, so the user's words are never trusted to the LLM. Budgeted by `max_verbatim_chars` (default 10,000; oldest entries dropped first).
* **Compacted Transcripts** — when an offload store is available, the full untruncated text of every summarized region is persisted and its handle listed, readable via `read_tool_result` (`canonical_record=True`, default). Summarization becomes recoverable instead of destructive. The store is shared automatically from the agent's [`tool_result_limit`](#tool-result-offloading), or passed explicitly via `store=`.
* **Note** — conservative continuation guidance: the model is told to verify intent against the user's verbatim words rather than acting on the summary alone.
* **Rehydrated Context** — output of an optional `rehydrate=` callable (sync or async, returning `str`, `list[str]`, or `None`), re-run on **every** compaction pass. Use it to re-inject working state that must survive summarization — the active plan, recently edited files:
```python theme={"dark"}
from pathlib import Path
from timbal.core.memory_compaction import summarize
memory_compaction=summarize(
threshold=20,
rehydrate=lambda: Path("PLAN.md").read_text(),
)
```
## Pinned results
Some tool results are durable context the model must keep referencing — for example loaded
[skill](/agents/skills) documentation. Those results are **pinned**: every compaction strategy
preserves them verbatim (and never orphans their paired tool call), regardless of `keep_last_n`,
turn windows, or drop/replacement mode. For `keep_last_n_messages` / `keep_last_n_turns` the
effective behavior is "last N **plus** pinned"; for `summarize`, pinned results are kept verbatim
and never fed to the summarizer (like system messages). Offloading skips pinned results too.
You opt a tool in declaratively with `pin_result=True`:
```python theme={"dark"}
from timbal.core.tool import Tool
tool = Tool(
name="load_policy",
handler=load_policy,
pin_result=True, # results survive compaction for the life of the conversation
)
```
The built-in `read_skill` tool sets this automatically, so skill guidance never gets compacted
away. Pinning is durable across pause/resume — the flag is persisted with the trace.
## Composing Strategies
Pass a list of compactors to apply them in order. Each compactor receives the output of the previous one.
```python theme={"dark"}
from timbal.core.memory_compaction import compact_tool_results, keep_last_n_turns
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
memory_compaction=[
compact_tool_results(keep_last_n=2), # first: shrink tool results
keep_last_n_turns(10), # then: trim to last 10 turns
],
)
```
## Observability
When compaction fires, the agent span records a `compaction` key in its metadata:
```json theme={"dark"}
{
"triggered": true,
"utilization": 0.91,
"steps": [
{ "compactor": "compact_tool_results", "before": 42, "after": 28 },
{ "compactor": "keep_last_n_turns", "before": 28, "after": 12 }
],
"passes": 1
}
```
* `utilization` — the context window fraction that triggered the most recent pass (`null` when `memory_compaction_ratio=0.0`).
* `steps` — one entry per compactor with message counts before and after (for the most recent pass).
* `passes` — how many times compaction fired on this span (turn start plus each mid-loop pass).
Every offload is recorded separately, in an `offload` key:
```json theme={"dark"}
{
"offload": [
{ "tool": "search_docs", "call_id": "t1", "original_chars": 184203,
"action": "spill", "handle": "06a7.../t1" }
]
}
```
`action` is `"spill"`, `"truncate"`, or `"truncate_fallback"` (a spill that degraded because the store was unavailable). The handle also persists on the tool result itself (`ToolResultContent.offload_handle`), surviving trace serialization and reload.
This data is visible in the Timbal platform trace viewer alongside the other span metadata.
# Skills
Source: https://docs.timbal.ai/agents/skills
Extend Agent capabilities through dynamic discovery of knowledge and tools
## What are Skills?
Skills provide **domain-specific knowledge and tools** that agents can selectively activate based on user requests.
Each skill is a module with its own documentation and tools that become accessible only when the skill is activated.
This is an extension of [Anthropic's Agent Skills](https://claude.com/blog/skills)
## How Skills Work
### Structure
Each skill is a directory containing:
* `SKILL.md` (required): Knowledge provided by the Skill
* `tools/` (optional): Folder for python files defining instances of skill-specific tools
* Supporting Files (optional): Additional documentation (must be referenced in `SKILL.md`)
Example structure:
```
skills/
└── payment_processing/
├── SKILL.md
├── fraud_detection.md
├── ...
└── tools/
├── process_refund.py
└── check_status.py
└── ...
```
### Loading Behavior
1. **Discovery**: The agent receives a list of available skills with their names and descriptions
2. **Activation**: When relevant to a user query, the agent loads the `SKILL.md` file and its associated tools
3. **Persistence**: Once loaded, skills remain available throughout the conversation
This lazy-loading approach keeps the agent's context efficient while ensuring specialized knowledge is available when needed.
**Skills, compaction, and offloading.** A skill's documentation reaches the model as the result of the `read_skill` tool call. Those results are **pinned**, so [memory compaction](/agents/memory-compaction) never drops or truncates them, and [tool result offloading](/agents/memory-compaction#tool-result-offloading) never spills them — the guidance stays available for as long as the skill is active, even on long runs where the rest of the history is compacted. This keeps an agent's tools and the instructions for using them from drifting apart.
## Using Skills
### Step 1: Enable Skills in Your Agent
Create a skills directory and configure your agent to use it:
```python highlight=6 theme={"dark"}
from timbal import Agent
agent = Agent(
name="my_agent",
model="anthropic/claude-sonnet-4-6",
skills_path="./skills"
)
```
The agent can now discover skills from this directory.
### Step 2: Create the SKILL.md File
Every skill requires a `SKILL.md` file with YAML frontmatter defining its name and description:
```markdown skills/payment_processing/SKILL.md theme={"dark"}
---
name: payment_processing
description: Complete payment processing including payments, refunds, and fraud detection
---
## Overview
There are different operations for e-commerce orders.
### Payment Policy
- Payments must be made within 30 days of order placement
- See `fraud_detection.md` for security guidelines
### Usage Guidelines
Always verify the order exists before processing payments or refunds.
### Escalation Process
If you cannot resolve an issue, escalate to the customer support team.
```
**Important**:\
The `name` and `description` fields are **required** in the YAML frontmatter. The agent uses these fields to decide when to activate the skill\
The `name` field must match the skill's directory name
### Step 3: Add Tools (Optional)
Skills can provide specialized tools that only become available when the skill is used.
```python skills/payment_processing/tools/process_refund.py theme={"dark"}
from timbal import Tool
async def process_refund(order_id: str, amount: float, reason: str) -> str:
"""Process a customer refund."""
# Your refund logic here
return f"Refund of ${amount} processed for order {order_id}"
# Create the tool instance
process_refund_tool = Tool(
name="process_refund",
description="Process a refund for a customer order",
handler=process_refund
)
```
### Step 4: Add Supporting Documentation (Optional)
For complex skills, include additional reference files and link to them in `SKILL.md`.
In the previous example, the `fraud_detection.md` file will only be read when the agent determines it is needed.
## Selecting Which Skills to Load
By default, every skill under `skills_path` is loaded into the agent. When a single directory is shared across multiple agents — each needing a different subset — use `skills_include` (whitelist) or `skills_exclude` (blacklist) to filter by directory name.
### Whitelist
Load only the listed skills:
```python theme={"dark"}
agent = Agent(
name="payments_agent",
model="anthropic/claude-sonnet-4-6",
skills_path="./skills",
skills_include=["payment_processing", "inventory_lookup"],
)
```
Unknown names raise `ValueError` — typos fail loudly instead of silently loading nothing.
### Blacklist
Load every skill except the listed ones:
```python theme={"dark"}
agent = Agent(
name="support_agent",
model="anthropic/claude-sonnet-4-6",
skills_path="./skills",
skills_exclude=["experimental_billing"],
)
```
Unknown names in `skills_exclude` are silently ignored.
`skills_include` and `skills_exclude` are mutually exclusive. Both require `skills_path` to be set.
### Explicit Selection Without `skills_path`
For full control, pass `Skill` instances directly via `tools=[...]`. This bypasses `skills_path` entirely and is useful when skills live in unrelated locations or when an agent only needs one or two:
```python theme={"dark"}
from timbal import Agent
from timbal.core.skill import Skill
agent = Agent(
name="refunds_agent",
model="anthropic/claude-sonnet-4-6",
tools=[Skill(path="./skills/payment_processing")],
)
```
## Best Practices
* Each skill should have a single, well-defined purpose.
* Write clear descriptions. Agents use it to decide when to activate a Skill.
* Use descriptive names for the skill.
## Key Takeaways
* **Skills are modular packages** that combine knowledge and tools for specific domains
* **Dynamic loading** keeps agents efficient by loading only relevant capabilities
* **SKILL.md YAML frontmatter** defines the skill's identity and purpose
* **Tools and supporting documentation** are loaded on demand when the skill is activated
# Structured Output
Source: https://docs.timbal.ai/agents/structured-output
Use Pydantic models to structure and validate your agent outputs
Instead of answering in natural language, the agent can return a specific output structure using Pydantic models.
## Why use structured output?
Using structured output ensures your agents return predictable, type-safe responses:
* Automatically validates and parses responses
* Guarantees required fields are present and correctly typed
* Reduces ambiguity compared to natural-language answers
By defining schemas upfront, `result.output` is always a validated Pydantic instance.
## Usage
1. Define the desired output schema using a [Pydantic model](https://docs.pydantic.dev/latest/api/base_model/).
2. Pass that model to the agent's `output_model` parameter.
3. Call the agent — `result.output` is the validated model, not a `Message`.
Using `tools` and `output_model` together may produce unexpected behavior — the agent can call tools instead of returning structured output. Prefer one or the other unless you have a deliberate multi-step flow.
```python theme={"dark"}
from pydantic import BaseModel, Field
from timbal import Agent
class Ingredient(BaseModel):
name: str = Field(..., description="Name of the ingredient")
amount: float = Field(..., description="Amount of the ingredient")
unit: str = Field(..., description="Unit of the ingredient")
class Recipe(BaseModel):
ingredients: list[Ingredient] = Field(..., description="List of ingredients")
total_time: float = Field(..., description="Total time in minutes")
steps: list[str] = Field(..., description="Steps to follow")
agent = Agent(
name="chef_agent",
model="openai/gpt-4o-mini",
output_model=Recipe,
)
result = await agent(prompt="I want to make lasagna").collect()
recipe: Recipe = result.output
print(recipe.total_time) # float
print(recipe.ingredients[0].name)
```
`agent.return_model` reflects the expected output type — `Message` by default, your Pydantic class when `output_model` is set.
## Validation and retries
If the model returns JSON that fails validation, the agent feeds the error back and retries (up to `max_iter` times) before raising. Field descriptions in your Pydantic model are sent to the LLM and strongly influence output quality — use them.
Pass extra validation context with `output_model_context` when your model's validators need runtime data:
```python theme={"dark"}
agent = Agent(
name="chef_agent",
model="openai/gpt-4o-mini",
output_model=Recipe,
output_model_context={"locale": "en-US"},
)
```
## Output example
`result.output` is a `Recipe` instance. Serialized:
```json theme={"dark"}
{
"ingredients": [
{ "name": "Lasagna sheets", "amount": 12, "unit": "pieces" },
{ "name": "Ground beef", "amount": 500, "unit": "grams" },
{ "name": "Ricotta cheese", "amount": 400, "unit": "grams" },
{ "name": "Mozzarella", "amount": 200, "unit": "grams" },
{ "name": "Marinara sauce", "amount": 600, "unit": "ml" },
{ "name": "Parmesan", "amount": 50, "unit": "grams" }
],
"total_time": 90,
"steps": [
"Brown the ground beef and season with salt and pepper.",
"Spread a thin layer of marinara in a baking dish.",
"Layer lasagna sheets, ricotta, beef, mozzarella, and sauce. Repeat.",
"Finish with sauce and parmesan on top.",
"Bake at 180°C for 45 minutes until bubbling.",
"Rest 10 minutes before serving."
]
}
```
# Adding Tools
Source: https://docs.timbal.ai/agents/tools
Give your agents the ability to interact with the outside world through automatic schema generation, parameter validation, and concurrent execution
## Basic Usage
Add functions directly as tools:
```python theme={"dark"}
def get_weather(location: str) -> str:
"""Get weather for a location."""
return f"Weather in {location}: 22°C"
agent = Agent(
name="weather_agent",
model="openai/gpt-4o-mini",
tools=[get_weather] # Function automatically becomes a tool
)
```
## Tool Configuration
Use the `Tool` type for custom descriptions and parameter control. The `handler` parameter is required and must be the function:
```python theme={"dark"}
from timbal import Tool
agent = Agent(
name="weather_agent",
model="openai/gpt-4o-mini",
tools=[Tool(handler=get_weather)]
)
```
### Custom Description
If the function name or docstring doesn't clearly describe what the tool does, add a custom `description`. This is the description the LLM sees about the tool:
```python theme={"dark"}
Tool(
handler=get_weather,
description="Get current weather information for any location"
)
```
### Parameter Visibility
When a tool has many parameters, reduce tokens and avoid overwhelming the LLM by showing only required parameters:
```python theme={"dark"}
Tool(
handler=search,
schema_params_mode="required" # Only show required params (default: "all")
)
```
You can also include or exclude specific parameters to fine-tune what the LLM sees:
```python theme={"dark"}
# Include extra parameters even in "required" mode
Tool(
handler=search_function,
schema_params_mode="required",
schema_include_params=["model"] # Include this optional param
)
# Exclude sensitive or internal parameters
Tool(
handler=api_function,
schema_exclude_params=["api_key", "debug_mode"] # Hide from LLM
)
```
### Default Parameters
Set default values that are automatically applied when the tool is called. These values are used when not specified by the LLM:
```python theme={"dark"}
Tool(
handler=database_query,
default_params={
"timeout": 30
}
)
```
You can also use functions or environment variables as default parameters:
```python theme={"dark"}
def get_current_timestamp():
return datetime.now().isoformat()
def process_data(data: str, timestamp: str | None = None, user_id: str = "default"):
return f"Processed {data} at {timestamp} by {user_id}"
# Tool with dynamic default parameters
tool = Tool(
name="data_processor",
handler=process_data,
default_params={
"timestamp": get_current_timestamp,
"user_id": "system_user"
}
)
```
## Built-in Tools
Timbal provides built-in tools for common use cases. These tools are ready to use and don't require implementing handlers.
### WebSearch
`WebSearch` enables agents to search the web.
`WebSearch` only works with **OpenAI** and **Anthropic** models. It's a specification-only tool that defines the tool schema for the LLM but doesn't contain executable logic. The actual web search execution is handled by the model provider. Other model providers are not supported.
```python theme={"dark"}
from timbal import Agent
from timbal.tools import WebSearch
agent = Agent(
name="research_agent",
model="openai/gpt-4o-mini",
tools=[
WebSearch(
allowed_domains=["wikipedia.org", "github.com"], # Restrict to specific domains
user_location={
"type": "approximate",
"country": "US",
"city": "New York"
} # Localize results
)
]
)
```
Available options:
* `allowed_domains`: List of domains to restrict searches to
* `blocked_domains`: List of domains to exclude (Anthropic only)
* `user_location`: Dictionary with location info to localize search results. Must include `type` field. Common values are `"approximate"` or `"exact"`, but valid values depend on the provider.
#### Multiple Instances of the Same Tool
If you need multiple instances of the same built-in tool with different configurations, they will have the same name by default, which causes conflicts. Set unique names and descriptions for each instance:
```python theme={"dark"}
general_search = WebSearch()
general_search.name = "general_search"
general_search.description = "Search the web for general information."
domain_search = WebSearch(allowed_domains=["example.com"])
domain_search.name = "domain_search"
domain_search.description = "Search only within example.com domain."
agent = Agent(
name="research_agent",
model="anthropic/claude-opus-4-6",
tools=[general_search, domain_search]
)
```
## Agent as a Tool
You can use an Agent as a tool within another Agent, enabling hierarchical agent compositions where specialized agents handle specific tasks.
Since `Agent` instances can be treated as `Tool` objects, they inherit the same parameter control configurations available to regular tools.
```python theme={"dark"}
from timbal import Agent
def calculate_cost(days: int, hotel_rate: float, flights: float = 0) -> float:
"""Calculate total trip cost."""
return (days * hotel_rate) + flights
# Pricing specialist agent
pricing_agent = Agent(
name="pricing_calculator",
description="Calculate travel costs", # Tool description for other agents
model="openai/gpt-4o-mini",
system_prompt="Answer only with the price. Any text",
tools=[calculate_cost]
)
# Main travel agent
travel_agent = Agent(
name="travel_assistant",
model="openai/gpt-4o",
system_prompt="Help plan trips and provide travel advice.",
tools=[pricing_agent]
)
```
**Automatic Nesting:** When an agent is used as a tool within another agent (as shown above), it's automatically nested for proper tracing and context management. However, if you use an agent inside a `pre_hook` or `post_hook`, you must manually call `.nest()`. See [Using Agents in Hooks](/core-concepts/context#using-agents-in-hooks) for details.
### Pinning results against compaction
Some tool results are durable context the model must keep referencing (loaded policy docs, skill guidance). Opt in with `pin_result=True` — [memory compaction](/agents/memory-compaction) never drops or truncates those results, and [tool result offloading](/agents/memory-compaction#tool-result-offloading) skips them too:
```python theme={"dark"}
Tool(
handler=load_policy,
pin_result=True, # survive compaction for the life of the conversation
)
```
The built-in `read_skill` tool sets this automatically. See [Pinned results](/agents/memory-compaction#pinned-results).
### Limiting oversized results
When a tool can return huge payloads, set a per-tool size limit. Results over the threshold are spilled to a store (or truncated) **once, when produced**, before they enter memory:
```python theme={"dark"}
from timbal.core import ToolResultLimit, Truncate
Tool(
handler=run_tests,
result_limit=ToolResultLimit(threshold=8_000, action=Truncate(strategy="tail")),
)
Tool(
handler=load_policy,
result_limit=None, # exempt — never reduced
)
```
An agent-wide default is `Agent(tool_result_limit=...)`. Precedence: `Tool.result_limit` > agent default. See [Tool Result Offloading](/agents/memory-compaction#tool-result-offloading).
## Commands
Register a `command` on any tool, agent, or workflow to let users invoke it directly with a `/` prefix — bypassing the LLM for that turn:
```python theme={"dark"}
Tool(handler=get_weather, command="weather")
# User sends: /weather Barcelona
# → runs get_weather(location="Barcelona") directly
```
See [Commands](/agents/commands) for argument parsing, nested agents, and workflows.
## MCP Servers
Connect any [Model Context Protocol](https://modelcontextprotocol.io/docs/getting-started/intro) server and use its tools like native ones:
```python theme={"dark"}
from timbal import Agent
from timbal.core import MCPServer
agent = Agent(
name="my_agent",
model="openai/gpt-4o-mini",
tools=[
MCPServer(
name="filesystem",
transport="stdio",
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem", "."],
),
],
)
```
The server's tools are discovered at runtime and offered to the LLM alongside your other tools. See [MCP Servers](/agents/mcp) for transports, authentication, result handling, and connection lifecycle.
## Dynamic Tools
When tool availability depends on runtime conditions (user role, permissions, input parameters), you need dynamic tool resolution.
Instead of exposing all tools to the agent, use Timbal's `ToolSet` class to resolve which tools are available at runtime.
For example, with a role-based ToolSet:
* **Role: admin** → Available tools: `delete_user`, `modify_permissions`, `view_profile`
* **Role: user** → Available tools: `view_profile`
The ToolSet checks the role at runtime and returns only the relevant tools. The agent only has access to the tools returned by the ToolSet for that role. Users without admin role won't see admin tools, preventing the agent from attempting unauthorized actions.
See [Dynamic Agents](/agents/dynamic#dynamic-tools) for implementation details and examples.
## Summary
* **Automatic Introspection**: Function signatures become tool schemas automatically
* **Enhanced Validation**: Pydantic-based parameter validation
* **Execution Flexibility**: Support for all Python callable types
* **Better Configuration**: Fine-grained parameter control
* **Performance**: Concurrent execution and optimized patterns
* **Robustness**: Improved error handling and tracing
* **Tool Sets**: Dynamic tool resolution
* **MCP Servers**: Any Model Context Protocol server as a tool source — see [MCP Servers](/agents/mcp)
* **Commands**: Slash-command shortcuts for direct tool invocation — see [Commands](/agents/commands)
* **Result limits & pinning**: Keep oversized or durable tool output under control — see [Memory Compaction](/agents/memory-compaction)
For more advanced patterns, see [Dynamic Agents](/agents/dynamic) and explore the built-in tools in the Timbal library.
# Chat Completions
Source: https://docs.timbal.ai/api-reference/ace/chat-completions
POST /ace/{ace_uid}/v1/chat/completions
OpenAI-compatible chat completions endpoint for an ACE.
# Create from Agent
Source: https://docs.timbal.ai/api-reference/ace/create-from-workforce
POST /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/ace
Create an empty ACE (ready, empty playbook) and link it to a workforce component. Does not trigger playbook generation.
# Cancel Job
Source: https://docs.timbal.ai/api-reference/ace/jobs/cancel
POST /ace/{ace_uid}/jobs/{job_uid}/cancel
Cancel an active ACE job.
# Enqueue Job
Source: https://docs.timbal.ai/api-reference/ace/jobs/create
POST /ace/{ace_uid}/jobs
Enqueue a job against an ACE. Today: `regenerate` (playbook [re-]generation from the linked workforce's agent code file; the current playbook stays usable until the job commits a new one). The kind vocabulary grows server-side — future kinds include usage-driven adaptation.
# Get Job
Source: https://docs.timbal.ai/api-reference/ace/jobs/get
GET /ace/{ace_uid}/jobs/{job_uid}
Get a single ACE job by id.
# List Jobs
Source: https://docs.timbal.ai/api-reference/ace/jobs/list
GET /ace/{ace_uid}/jobs
List jobs for an ACE, most recent first.
# Retry Job
Source: https://docs.timbal.ai/api-reference/ace/jobs/retry
POST /ace/{ace_uid}/jobs/{job_uid}/retry
Retry a failed, cancelled, or errored ACE job.
# Link to Agent
Source: https://docs.timbal.ai/api-reference/ace/link
PUT /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/ace
Link an existing ACE to a workforce component.
# List Context Variables
Source: https://docs.timbal.ai/api-reference/ace/list-context-vars
GET /ace/{ace_uid}/vars
List the variables declared on an ACE.
# List Policies
Source: https://docs.timbal.ai/api-reference/ace/list-policies
GET /ace/{ace_uid}/policies
List the policies declared on an ACE.
# Unlink from Agent
Source: https://docs.timbal.ai/api-reference/ace/unlink
DELETE /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/ace
Detach the ACE linked to a workforce component.
# Attach Channel to Alarm
Source: https://docs.timbal.ai/api-reference/alarms/attach-channel
POST /orgs/{org_id}/alarms/{alarm_id}/channels
Attach another notification channel to an alarm, so one condition reaches several destinations (page Slack and email the on-call, say).
Accepts the same channel shape as alarm creation: `{ "id": }` to reuse an existing channel, or inline `type` + `config` to create one. Inline channels identical to an existing row are deduped rather than duplicated.
Idempotent — re-attaching a channel that is already linked returns `already_linked: true` and changes nothing.
Gated by the alarm, not the channel: `alarms.manage` for an org-level alarm, `projects.alarms.manage` on the owner for a project alarm. Channels themselves are org-wide, so a project-scoped caller can point their alarm at any existing channel — attaching one is a routing decision about their alarm, whereas creating or deleting the channel row needs `notification_channels.manage`.
# Create Channel
Source: https://docs.timbal.ai/api-reference/alarms/channels/create
POST /orgs/{org_id}/channels
Create a delivery channel for this org, or reuse an existing one.
Channels dedupe on `(type, config)`: posting the same Slack channel twice returns the original row with `channel_action: "reused"` rather than creating a near-duplicate. Attach the returned `id` to an alarm via `POST /orgs/{org_id}/alarms/{alarm_id}/channels`.
# Delete Channel
Source: https://docs.timbal.ai/api-reference/alarms/channels/delete
DELETE /orgs/{org_id}/channels/{channel_id}
Delete a delivery channel for the organization.
Refused with `409` while alarms still reference the channel; `force: true` deletes anyway and unlinks them. The one case `force` will not do is strip the last channel from an *enabled* alarm — that alarm would go on evaluating and recording state transitions with nowhere to send them. Attach a replacement channel or disable the alarm first.
# List Channels
Source: https://docs.timbal.ai/api-reference/alarms/channels/list
GET /orgs/{org_id}/channels
List the organization's delivery channels — the addresses alarms can notify (email inboxes, Slack channels).
Channels carry no conditions of their own. What to watch and when to fire lives on the alarm; this is only where the message goes.
# Create Alarm
Source: https://docs.timbal.ai/api-reference/alarms/create
POST /orgs/{org_id}/alarms
Create a metric alarm with a threshold over evaluation periods.
Set `denominator_metric_name` for a **rate** rather than a count: the datapoint becomes `metric_name / denominator_metric_name` and `threshold` reads as a fraction (`0.05` = 5%). Prefer this for anything that scales with traffic — a fixed error count pages at 3am once the org grows. Pair it with `min_sample_count` so a single failure in a quiet minute isn't a 100% error rate.
Both series must be binned on the same clock, since they are divided period by period. For outcome rates that means `app.run.terminal_count` — runs that *finished* in the period — and not `app.run.count`, which counts runs that *started* and includes ones still running. Mismatched pairs are rejected.
Ratio alarms also require `stat` `sum` or `count`. The stat is applied to each series and the two results divided, so only stats that add up across the minutes in a period come out as a rate — `avg` would divide two means and `min`/`max` would take their extremes from different minutes. Other stats are rejected rather than silently producing a number that isn't a rate.
A sustained incident only ever notifies once, so repeat messages mean the alarm is flapping. Three levers fix that, and which one depends on the metric: `clear_threshold` (fire high, clear low) for continuous series like rates and latencies; `datapoints_to_alarm` for spiky ones that cross the line briefly; and `missing_data: "ignore"` for sparse ones, where a minute with no traffic is absence of evidence rather than a recovery.
Use `GET /orgs/{org_id}/metrics/catalog` to see which metric names and labels this org is actually emitting. Two metrics won't be listed until you alarm on them: `org.credits.usage_pct` (percent of the billing period's prepaid budget consumed) and `org.credits.remaining` (credits left in it) are only sampled for orgs that have an enabled alarm on one, since the underlying read is the billing gate's and too heavy to run speculatively. Both are levels sampled once a minute, so use `max`, `avg` or `min` — `sum` would add one reading per minute in the period and is rejected. `org.credits.remaining` goes negative under overage.
Percent and headroom are worth having separately: 95% of a large budget can still be thousands of credits, while 50% of a small one is one bad afternoon. For burn *volume* rather than budget position, alarm on `org.credits.spend` with `sum` — that one carries `project_id` and `workforce_id` dims, so it can scope to a single project or workforce the way a budget can't.
For latency, `stat` accepts any percentile from `p0.1` to `p99.9`, in tenths. Equivalent spellings are fine (`p95.0` is accepted and comes back as `p95`), but anything finer than a tenth is rejected rather than rounded, so an alarm never silently watches a different point than the one you asked for. These are exact, computed over the individual run durations rather than interpolated from bins, which is also why they are limited to `app.run.duration_ms`, cannot be a ratio, and cannot carry `label`, `project_env_id` or `dims.deployment_id` — a run row has no such column, so those filters would silently widen the alarm to everything. Prefer a high percentile over `avg` for latency: an average hides the tail that users actually notice, and hides it harder as traffic grows.
Two different project fields, gated separately. Top-level `project_id` is ownership — who may see and edit the alarm afterwards — and setting it requires `projects.alarms.manage` on that project, while leaving it out makes an org-level alarm and requires `alarms.manage`. `dims.project_id` only narrows which metric bins are read, but it still exposes that project's numbers through the alarm's notifications, so it requires `projects.alarms.read` on the project it names.
# Delete Alarm
Source: https://docs.timbal.ai/api-reference/alarms/delete
DELETE /orgs/{org_id}/alarms/{alarm_id}
Delete a metric alarm and its channel links.
Needs `alarms.manage` for an org-level alarm, or `projects.alarms.manage` on the owning project for a project alarm.
# Detach Channel from Alarm
Source: https://docs.timbal.ai/api-reference/alarms/detach-channel
DELETE /orgs/{org_id}/alarms/{alarm_id}/channels/{channel_id}
Detach a channel from an alarm. The `NotificationChannels` row itself is left alone — other alarms and notification rules may still be using it.
Removing the **last** channel from an enabled alarm is rejected: it would keep evaluating and recording transitions while notifying nobody, which looks healthy from the outside. To swap a sole channel, attach the replacement first; to silence an alarm, `PATCH` it with `enabled: false`.
# Get Alarm
Source: https://docs.timbal.ai/api-reference/alarms/get
GET /orgs/{org_id}/alarms/{alarm_id}
One alarm's configuration, current state and linked channels.
Same shape as an entry in `GET /orgs/{org_id}/alarms`, for refreshing a detail view without refetching the whole list.
Needs `alarms.read` for an org-level alarm, or `projects.alarms.read` on the owning project for a project alarm.
# Alarms Health
Source: https://docs.timbal.ai/api-reference/alarms/health
GET /orgs/{org_id}/alarms/health
For every enabled alarm, when the metric series behind it last produced a datapoint.
Answers the one question an alarm's own `state` cannot: is it still watching anything? Under the default `not_breaching` policy an alarm whose metric stopped being emitted — a deleted workforce, a disabled trace filter, a renamed metric — sits at `ok` indefinitely and looks identical to one that is genuinely healthy.
`stale` marks silence lasting ten evaluation windows (minimum one hour), which is long enough that a quiet-but-live series won't trip it. A brand-new alarm reads as stale until its first datapoint, which is accurate: it cannot fire yet.
For a ratio alarm, `last_datapoint_at` is the **older** of the two series, since a period only yields a datapoint where both sides have one. A live numerator over a dead denominator is therefore reported as stale — every period resolves to a missing ratio, so the alarm is blind despite one of its series still flowing. `denominator_last_datapoint_at` tells you which side went quiet.
Covers the same alarms `GET /orgs/{org_id}/alarms` returns for you, so `healthy` and `stale_count` describe your visible subset — a project-scoped caller gets their project's blind spots, not the org's.
# Alarm History
Source: https://docs.timbal.ai/api-reference/alarms/history
GET /orgs/{org_id}/alarms/{alarm_id}/history
State transition history for an alarm, newest first.
Needs `alarms.read` for an org-level alarm, or `projects.alarms.read` on the owning project for a project alarm.
# List Alarms
Source: https://docs.timbal.ai/api-reference/alarms/list
GET /orgs/{org_id}/alarms
The metric alarms you can see in this org, with current state and channels.
Filtered per alarm, not per request: org-level alarms (credits, org-wide spend) need `alarms.read`, and an alarm tagged to a project needs `projects.alarms.read` on that project. Holding one and not the other returns the matching subset rather than a `403`, so a role scoped to a single project sees exactly that project's alarms.
# Metrics Catalog
Source: https://docs.timbal.ai/api-reference/alarms/metrics/catalog
GET /orgs/{org_id}/metrics/catalog
Metric names and label values actually recorded for this org in the recent past. Use it to populate a metric picker: `GET /orgs/{org_id}/metrics` requires a name, and alarms reference names that only exist once something has emitted them.
`origin` separates the two kinds: `system` metrics (`app.*`, `deployment.*`, `org.*`) are emitted by the platform and have no configuration, while `custom` ones come from a metric definition — see `definition_ids` for the config behind them.
This reports what is *flowing*, not what is configured, so the two can disagree in both directions: a definition that has never matched a span will not appear, and a deleted definition's series stays until its bins age out.
Needs `analytics.read` for the org-wide catalog. Passing `project_id` narrows it to series carrying that project's dimension and accepts `projects.alarms.read` there instead, so a project-scoped role gets a working metric picker. That view omits series with no project dimension, `org.credits.*` among them — which are org-level metrics that caller cannot alarm on anyway.
# Create Metric Definition
Source: https://docs.timbal.ai/api-reference/alarms/metrics/definitions/create
POST /orgs/{org_id}/metrics/definitions
Define a metric of your own, emitted as runs land. Once it exists it behaves like any platform metric: readable through `GET /orgs/{org_id}/metrics`, listed in the catalog with `origin: custom`, and alarmable with no extra setup.
The definition is a span predicate over each completed run's trace. Span paths are `{workforce}.{step}` and matching is on the **leaf**: use `llm` or `get_address`, not the full path. Predicate fields AND together and all are optional — omitting every one matches every non-root span.
`metric_name` is forced under the reserved `trace.` prefix so a definition cannot shadow a platform series. Set `label_source` to `span_name` to break the metric out per step, then point an alarm at a single step with `dims.label`.
Definitions are **not retroactive**: matching happens as runs arrive, so a new definition only sees traces written after it. Dry-run it against stored traces first with `POST /orgs/{org_id}/metrics/definitions/preview`. Note also that only sum/count/avg/min/max are available on the result — percentiles are not derivable from metric bins.
Set `project_id` to scope the definition to one project, which also makes it editable by holders of `projects.alarms.manage` there. Leaving it out defines an org-wide metric that runs against every project's traces, and needs `alarms.manage`.
# Delete Metric Definition
Source: https://docs.timbal.ai/api-reference/alarms/metrics/definitions/delete
DELETE /orgs/{org_id}/metrics/definitions/{definition_id}
Stop emitting a custom metric. Takes effect within one flush interval (~10s).
Bins the definition already wrote are kept — they are real observations — and age out with normal metric retention, so the series stays readable and keeps appearing in the catalog for a while with no definition behind it. Alarms pointing at the metric are left alone and will fall to `insufficient_data` once the series stops.
Needs `alarms.manage` for an org-wide definition, or `projects.alarms.manage` on the owning project for a project one.
# List Metric Definitions
Source: https://docs.timbal.ai/api-reference/alarms/metrics/definitions/list
GET /orgs/{org_id}/metrics/definitions
The org's custom metric definitions and their current predicates.
`project_id` filters to what is *in effect* for that project, which includes org-wide definitions (those with no `project_id`) since they run against that project's runs too.
This lists configuration, not data. A definition that has never matched a span appears here but not in `GET /orgs/{org_id}/metrics/catalog`.
Filtered per definition the same way `GET /orgs/{org_id}/alarms` is: org-wide definitions need `alarms.read`, project ones need `projects.alarms.read` on the owner. So a project-scoped caller sees their own definitions but not the org-wide ones that also run against their runs, even with `project_id` set.
# Preview Metric Definition
Source: https://docs.timbal.ai/api-reference/alarms/metrics/definitions/preview
POST /orgs/{org_id}/metrics/definitions/preview
Run a candidate predicate against recent stored traces without saving it, and see what it would have recorded.
Definitions are not retroactive, so a saved predicate that matches nothing looks identical to one that matches nothing *yet*. This answers that before you commit: which spans matched, what values they'd contribute, and how many distinct labels the metric would carry.
Samples the newest runs (20 by default, 50 max) — enough to tell a working predicate from a broken one, not a measurement of the true rate. Pass `workforce_id` to make the sample representative of the agent you're actually matching. This is the only endpoint that reads stored traces, which is why it is capped; do not poll it.
Because it returns trace content, `project_id` doubles as the permission target: omit it and the sample spans the whole org, which needs `alarms.manage`; set it and the sample is capped to that project, which needs `projects.alarms.manage` there. A caller scoped to one project must therefore pass it.
# Update Metric Definition
Source: https://docs.timbal.ai/api-reference/alarms/metrics/definitions/update
PATCH /orgs/{org_id}/metrics/definitions/{definition_id}
Only the name and the enabled flag are mutable. The predicate and destination metric are fixed at creation: changing them would splice two different meanings into one series with nothing marking the seam, so a changed definition should be a new definition.
Disabling stops new observations within one flush interval (~10s). Bins already written stay and age out with normal metric retention.
The owning project is not mutable either, so the permission needed here is fixed: `alarms.manage` for an org-wide definition, `projects.alarms.manage` on the owner for a project one.
# Query Metric
Source: https://docs.timbal.ai/api-reference/alarms/metrics/query
GET /orgs/{org_id}/metrics
Time-bucketed count/sum/avg/min/max for one metric, with optional dimension filters.
Needs `analytics.read` for an unrestricted query. Passing `project_id` also accepts `projects.alarms.read` on that project, so a project-scoped role can read the series behind its own alarms.
# Update Alarm
Source: https://docs.timbal.ai/api-reference/alarms/update
PATCH /orgs/{org_id}/alarms/{alarm_id}
Update alarm thresholds, evaluation settings, or enabled state.
`clear_threshold` is the one field where `null` differs from omitting the key: send `null` to drop the hysteresis, omit it to leave it as is. It is validated against the resulting `threshold` and `comparison`, so patching either side of the pair alone can't invert them.
`metric_name` and `denominator_metric_name` are not patchable: changing what an alarm watches would leave its state and history describing a different question, so define a new alarm instead.
`enabled: true` requires at least one attached channel. Channels may be detached freely while an alarm is disabled, so this is what stops that from re-enabling an alarm that evaluates and records transitions with nowhere to notify.
The alarm's owning project is not patchable either, so the permission needed here is fixed for the life of the alarm: `alarms.manage` when it is org-level, `projects.alarms.manage` on the owner when it is not.
# Authentication
Source: https://docs.timbal.ai/api-reference/authentication
How to authenticate to the Timbal API
The Timbal API uses API keys to authenticate requests. You can create and manage your API keys by navigating to **Profile → API Keys** in the dashboard.
API keys are **personal to your user account** and inherit all your permissions across organizations. Each key carries the same privileges as your user account, so be sure to keep them secure! Do not share your API keys in publicly accessible areas such as GitHub, client-side code, and so forth.
**Important:** For security, we do not store your API keys after creation. Make sure to copy and store them in a secure location immediately—you will not be able to recover them later.
When using the Timbal framework, you can set these environment variables for automatic authentication:
```bash theme={"dark"}
export TIMBAL_API_KEY="your_api_key_here"
export TIMBAL_API_HOST="https://api.timbal.ai"
```
The framework will automatically use these values, eliminating the need to pass credentials in your code.
All API requests must be made over HTTPS. Calls made over plain HTTP will fail. API requests without authentication will also fail.
# Browse Columns
Source: https://docs.timbal.ai/api-reference/connectors/browse/columns
GET /orgs/{org_id}/connectors/{connector_id}/browse/columns
List columns for a remote table.
# Browse Connections
Source: https://docs.timbal.ai/api-reference/connectors/browse/connections
GET /orgs/{org_id}/connectors/{connector_id}/browse/connections
List remote connections available to a connector.
# List Filesystem Entries
Source: https://docs.timbal.ai/api-reference/connectors/browse/filesystems/entries
GET /orgs/{org_id}/connectors/{connector_id}/browse/filesystems/entries
List one directory under a connector filesystem root, with size/mtime per entry, optional glob filter, sort, and bounded recursion (depth). Requires fs:browse (0.1.27+).
# List Filesystems
Source: https://docs.timbal.ai/api-reference/connectors/browse/filesystems/list
GET /orgs/{org_id}/connectors/{connector_id}/browse/filesystems
List filesystem roots configured on a connector (names only). Requires a connector advertising fs:browse (0.1.27+).
# Search Filesystem
Source: https://docs.timbal.ai/api-reference/connectors/browse/filesystems/search
GET /orgs/{org_id}/connectors/{connector_id}/browse/filesystems/search
Search file contents under a connector filesystem root with a line regex — bounded on files, matches, bytes, and time; binaries skipped. Runs entirely in-process on the node. Requires ConnectorsManage (reads file contents, same bar as ad-hoc SQL) and fs:browse (0.1.27+).
# Summarize Filesystem
Source: https://docs.timbal.ai/api-reference/connectors/browse/filesystems/summary
GET /orgs/{org_id}/connectors/{connector_id}/browse/filesystems/summary
Summarize a directory tree on a connector filesystem root without listing it: file/dir counts, total bytes, largest files, extension histogram. The probe to run before a recursive copy. Requires fs:browse (0.1.27+).
# Browse Table
Source: https://docs.timbal.ai/api-reference/connectors/browse/table
GET /orgs/{org_id}/connectors/{connector_id}/browse/table
Full introspection of one remote table: rich columns (defaults, identity, comments, precision), indexes, foreign keys, check constraints, table comment and size. Requires a connector advertising browse:describe (0.1.25+).
# Browse Tables
Source: https://docs.timbal.ai/api-reference/connectors/browse/tables
GET /orgs/{org_id}/connectors/{connector_id}/browse/tables
List remote tables available to a connector.
# Create Enrollment Token
Source: https://docs.timbal.ai/api-reference/connectors/enrollment-tokens/create
POST /orgs/{org_id}/connectors/enrollment-tokens
Create a connector enrollment token.
# Copy Filesystem to Knowledge Base
Source: https://docs.timbal.ai/api-reference/connectors/filesystems/copy
POST /orgs/{org_id}/connectors/{connector_id}/filesystems/copy
Copy a file or directory tree from a connector filesystem root into a knowledge base's document store. Store-only by default (parse: true to opt in). Requires fs:copy (0.1.27+).
# Get Connector
Source: https://docs.timbal.ai/api-reference/connectors/get
GET /orgs/{org_id}/connectors/{connector_id}
Get a connector by id.
# Create Connector Job
Source: https://docs.timbal.ai/api-reference/connectors/jobs/create
POST /orgs/{org_id}/connectors/{connector_id}/jobs
Dispatch a job to a connector.
# List Connector Jobs
Source: https://docs.timbal.ai/api-reference/connectors/jobs/list
GET /orgs/{org_id}/connectors/jobs
List connector jobs for the organization.
# Stop Connector Job
Source: https://docs.timbal.ai/api-reference/connectors/jobs/stop
POST /orgs/{org_id}/connectors/{connector_id}/jobs/{job_id}/stop
Cancel a running extract job. The row is terminalized as `stopped` first
(so the run is dead even if the connector is offline or never acks), then a
`StopJob` command aborts the connector-side task best-effort. Idempotent:
stopping an already-terminal job returns its current state.
# List Connectors
Source: https://docs.timbal.ai/api-reference/connectors/list
GET /orgs/{org_id}/connectors
List connectors for the organization.
# Push Local Config
Source: https://docs.timbal.ai/api-reference/connectors/local-config
POST /orgs/{org_id}/connectors/{connector_id}/local-config
Push replacement connections/filesystems maps to a connector box.
# Query Connector
Source: https://docs.timbal.ai/api-reference/connectors/query
POST /orgs/{org_id}/connectors/{connector_id}/query
Run an ad-hoc SQL query against a connector's source and return the bounded result inline. Read-only by default (enforced node-side); write mode must also be allowed by the connection's local policy on the connector box. Requires a connector advertising query:sql (0.1.26+).
# Rebind Connector
Source: https://docs.timbal.ai/api-reference/connectors/rebind
POST /orgs/{org_id}/connectors/{connector_id}/rebind
Rebind a connector to a different knowledge base.
# Revoke Connector
Source: https://docs.timbal.ai/api-reference/connectors/revoke
POST /orgs/{org_id}/connectors/{connector_id}/revoke
Revoke a connector's credential and remove it from the fleet.
# Create Connector Sync
Source: https://docs.timbal.ai/api-reference/connectors/syncs/create
POST /orgs/{org_id}/connectors/{connector_id}/syncs
Create a connector sync configuration.
# Delete Connector Sync
Source: https://docs.timbal.ai/api-reference/connectors/syncs/delete
DELETE /orgs/{org_id}/connectors/{connector_id}/syncs/{sync_id}
Delete a connector sync.
# Get Connector Sync
Source: https://docs.timbal.ai/api-reference/connectors/syncs/get
GET /orgs/{org_id}/connectors/{connector_id}/syncs/{sync_id}
Get a connector sync by id.
# List Connector Syncs
Source: https://docs.timbal.ai/api-reference/connectors/syncs/list
GET /orgs/{org_id}/connectors/syncs
List connector syncs for the organization.
# Run Sync Now
Source: https://docs.timbal.ai/api-reference/connectors/syncs/run
POST /orgs/{org_id}/connectors/{connector_id}/syncs/{sync_id}/run
Run a connector sync immediately.
# Update Connector Sync
Source: https://docs.timbal.ai/api-reference/connectors/syncs/update
PATCH /orgs/{org_id}/connectors/{connector_id}/syncs/{sync_id}
Update a connector sync.
# Update Connector
Source: https://docs.timbal.ai/api-reference/connectors/update
POST /orgs/{org_id}/connectors/{connector_id}/update
Trigger a connector software update.
# Errors
Source: https://docs.timbal.ai/api-reference/errors
How to handle errors
The Timbal API uses conventional HTTP response codes to indicate the success or failure of an API request.
In general: Codes in the `2xx` range indicate success. Codes in the `4xx` range indicate an error that failed given the information provided (*e.g.*, a required parameter was omitted, a charge failed, etc.). Codes in the `5xx` range indicate an error with Timbal servers (these are rare).
## Rate limiting
Some endpoints enforce **per-user** rate limits. When limits apply, successful **`2xx`** responses and **`429 Too Many Requests`** may include:
| Header | Meaning |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| `RateLimit-Limit` | Maximum requests allowed in the current window. |
| `RateLimit-Remaining` | Requests left before the window resets. |
| `RateLimit-Reset` | Seconds until the window resets. |
| `Retry-After` | On **`429`**, seconds to wait before retrying (prefer this over a fixed backoff when present). |
Legacy aliases **`X-RateLimit-Limit`**, **`X-RateLimit-Remaining`**, and **`X-RateLimit-Reset`** mirror the `RateLimit-*` values.
| Code | Status | Description |
| ------------------------------ | ----------------- | --------------------------------------------------------------------------------------------------- |
| 200 | OK | Everything worked as expected. |
| 400
422 | Bad Requests | The request was unacceptable, often due to missing a required parameter. |
| 401 | Unauthorized | No valid API key provided. |
| 402 | Request Failed | The parameters were valid but the request failed. |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request. |
| 404 | Not Found | The requested resource doesn’t exist. |
| 429 | Too Many Requests | Per-user rate limit exceeded. Honor **`Retry-After`** when sent; otherwise use exponential backoff. |
| 500
502
503
504 | Server Errors | Something went wrong on our end. |
# List Eval Definitions
Source: https://docs.timbal.ai/api-reference/evals/definitions
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/definitions
List eval definitions (test cases from evals/*.yaml) per workforce component. Reads the env branch's worktree by default; pass ?rev= to read from git.
# Create Eval Job
Source: https://docs.timbal.ai/api-reference/evals/jobs/create
POST /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/jobs
Enqueue a background eval job (today: generate_initial — Claude-generate an initial eval suite for a workforce component, written to the env branch's worktree).
# Get Eval Job
Source: https://docs.timbal.ai/api-reference/evals/jobs/get
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/jobs/{job_uid}
Get a single eval job by uid.
# List Eval Jobs
Source: https://docs.timbal.ai/api-reference/evals/jobs/list
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/jobs
List eval jobs (background eval generation) for an env, most recent first.
# Create Eval Run
Source: https://docs.timbal.ai/api-reference/evals/runs/create
POST /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/runs
Manually run evals for a workforce component (no deploy). Defaults to the env worktree so uncommitted evals are included; pass source=git for a clean extract.
# Delete Eval Run
Source: https://docs.timbal.ai/api-reference/evals/runs/delete
DELETE /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/runs/{eval_run_id}
Delete an eval run from the history. Running runs cannot be deleted.
# Stream Eval Run Events
Source: https://docs.timbal.ai/api-reference/evals/runs/events
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/runs/{eval_run_id}/events
Stream an in-flight eval run's JSONL events (cursor + long-poll). On `expired: true`, fall back to the single-run GET.
# Get Eval Run
Source: https://docs.timbal.ai/api-reference/evals/runs/get
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/runs/{eval_run_id}
Get a single eval run, including its logs.
# List Eval Runs
Source: https://docs.timbal.ai/api-reference/evals/runs/list
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/runs
List eval runs (deploy-gate and manual executions) for an environment.
# Upload file (temporary)
Source: https://docs.timbal.ai/api-reference/files/upload
POST /files
Upload a short-lived file for temporary staging.
**Utility endpoint** — not org-scoped. Upload requires authentication; the response includes a download URL (\~24h). Anyone with the URL can download until expiry — treat it as a secret. Not for sensitive or regulated data.
Ephemeral staging only (\~24h lifecycle); there is no durable org file record behind this route.
For **durable storage, parsing, embedding, and reuse** across a knowledge base, use [Knowledge Bases → Files](/api-reference/knowledge-bases/files/list) (`POST /orgs/{org_id}/k2/{kb_id}/files`) instead.
# Introduction
Source: https://docs.timbal.ai/api-reference/introduction
Introduction to the Timbal Platform API
The Timbal API is organized around [REST](https://en.wikipedia.org/wiki/REST). Our API has predictable resource-oriented URLs, accepts JSON-encoded request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs.
Unlike traditional APIs that offer a single *test mode* or *sandbox*, the Timbal Platform empowers you with unlimited deployment flexibility through versioning.
Every resource you create automatically supports versioning, similar to Git's branching and commit system. You can create new versions of your agents, workflows, and knowledge bases, then seamlessly navigate between versions, roll back changes, or branch off in different directions.
Each version of your agents, workflows, and knowledge bases can be configured with its own environment variables, permissions, and settings—allowing you to seamlessly manage development, staging, production, or any custom configuration that suits your deployment pipeline.
## Organization scoping
Most resources in Timbal are scoped to an **organization**, meaning their API path is prefixed with `/orgs/{org_id}`. Resources are then nested under their parent in a standard RESTful hierarchy. For example:
```
/orgs/{org_id}/projects
/orgs/{org_id}/projects/{project_id}/envs
/orgs/{org_id}/projects/{project_id}/envs/{env_id}/deploy
/orgs/{org_id}/k2/{kb_id}/query
```
Not all endpoints are org-scoped. Some are user-scoped and live under `/me` (e.g. listing organizations or managing API keys). **`POST /files`** uploads a short-lived binary to object storage (Bearer auth, same as the rest of the API); it does **not** include `org_id` in the path—org context comes from the authenticated key.
HTTP verbs follow REST conventions:
| Verb | Usage |
| -------- | --------------------------- |
| `GET` | Retrieve a resource or list |
| `POST` | Create a resource |
| `PATCH` | Partially update a resource |
| `DELETE` | Delete a resource |
# Create Backup
Source: https://docs.timbal.ai/api-reference/knowledge-bases/backups/create
POST /orgs/{org_id}/k2/{kb_id}/backups
Create an on-demand backup of a knowledge base.
On-demand backup of a knowledge base.
# Get Backup
Source: https://docs.timbal.ai/api-reference/knowledge-bases/backups/get
GET /orgs/{org_id}/k2/{kb_id}/backups/{backup_id}
Get a single backup.
# List Backups
Source: https://docs.timbal.ai/api-reference/knowledge-bases/backups/list
GET /orgs/{org_id}/k2/{kb_id}/backups
List backups for a knowledge base.
# Create Knowledge Base
Source: https://docs.timbal.ai/api-reference/knowledge-bases/create
POST /orgs/{org_id}/k2
Create a new knowledge base
# Delete Knowledge Base
Source: https://docs.timbal.ai/api-reference/knowledge-bases/delete
DELETE /orgs/{org_id}/k2/{kb_id}
Delete a knowledge base
# Add File
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/add
POST /orgs/{org_id}/k2/{kb_id}/files
Add a file to a knowledge base by upload or URL.
File names must be unique within a directory. Uploading `report.pdf` to `docs/` when it already exists returns `409`.
Folders and files share the same namespace: you cannot upload a file whose name matches an existing folder in the same directory.
# Delete Chunk
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/chunks/delete
DELETE /orgs/{org_id}/k2/{kb_id}/files/{file_id}/chunks/{chunk_uid}
Delete a chunk from a knowledge base file.
# Get Chunk
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/chunks/get
GET /orgs/{org_id}/k2/{kb_id}/files/{file_id}/chunks/{chunk_uid}
Get a single chunk by uid.
# Insert Chunk
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/chunks/insert
POST /orgs/{org_id}/k2/{kb_id}/files/{file_id}/chunks
Insert a new chunk into a knowledge base file.
# List Chunks
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/chunks/list
GET /orgs/{org_id}/k2/{kb_id}/files/{file_id}/chunks
List chunks for a knowledge base file in document order.
# Update Chunk
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/chunks/update
PATCH /orgs/{org_id}/k2/{kb_id}/files/{file_id}/chunks/{chunk_uid}
Edit a chunk's content, embedding, or page range.
# Create Directory
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/create-dir
POST /orgs/{org_id}/k2/{kb_id}/directories
Create a virtual directory in the knowledge base
Folders appear in file listings with `content_type` set to `application/vnd.timbal.k2-directory`. Creating a folder that already exists is idempotent and returns `200`.
Folders and files share the same namespace: creating folder `reports` under `docs/` returns `409` if a file named reports already exists there.
# Delete File
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/delete
DELETE /orgs/{org_id}/k2/{kb_id}/files/{file_id}
Delete a file from a knowledge base
Use this endpoint to delete a **regular file** by `file_id`, and also to remove a **directory** from the knowledge base.
For a folder, use the same **`id`** returned for that row in **`GET /orgs/{org_id}/k2/{kb_id}/files`** (or the `placeholder_file_id` from create-directory). Deleting that listing row removes the folder from file listings.
# Get File
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/get
GET /orgs/{org_id}/k2/{kb_id}/files/{file_id}
Get a file with its parsings and embeddings metadata
Listings from `GET /orgs/{org_id}/k2/{kb_id}/files` include **folder rows** (`content_type` `application/vnd.timbal.k2-directory`). This **get file** endpoint is only for **regular uploaded files** (parsings, embeddings, etc.).
If you pass a **directory row’s `id`** from the list endpoint, the API responds with **`404` Not Found** — virtual directories do not have a full file payload here.
# List Files
Source: https://docs.timbal.ai/api-reference/knowledge-bases/files/list
GET /orgs/{org_id}/k2/{kb_id}/files
List files and folders in a knowledge base directory.
Both regular files and folders are returned. Folder entries have `content_type` set to `application/vnd.timbal.k2-directory` and `content_length` of `0`. Use the optional directory query parameter to scope results to a specific folder (e.g. `?directory=docs`).
# Get Knowledge Base
Source: https://docs.timbal.ai/api-reference/knowledge-bases/get
GET /orgs/{org_id}/k2/{kb_id}
Get knowledge base details
# List Knowledge Bases
Source: https://docs.timbal.ai/api-reference/knowledge-bases/list
GET /orgs/{org_id}/k2
List knowledge bases in an organization
# Create Policy Assignment
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/assignments/create
POST /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}/assignments
Bind an access-shaping rule to a role.
# Delete Policy Assignment
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/assignments/delete
DELETE /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}/assignments/{assignment_id}
Revoke a role binding on an access-shaping rule.
# Get Policy Assignment
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/assignments/get
GET /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}/assignments/{assignment_id}
Fetch a single role binding for an access-shaping rule.
# List Policy Assignments
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/assignments/list
GET /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}/assignments
List role bindings for an access-shaping rule.
# Create Policy
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/create
POST /orgs/{org_id}/k2/{kb_id}/policies
Author a new access-shaping rule on a knowledge base.
# Delete Policy
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/delete
DELETE /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}
Soft-delete an access-shaping rule on a knowledge base.
# Get Policy
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/get
GET /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}
Fetch a single access-shaping rule by id.
# Set Governance Mode
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/governance-mode
PUT /orgs/{org_id}/k2/{kb_id}/policies/governance-mode
Set query-time governance mode for a knowledge base.
# List Policies
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/list
GET /orgs/{org_id}/k2/{kb_id}/policies
List access-shaping rules authored on a knowledge base.
# Update Policy
Source: https://docs.timbal.ai/api-reference/knowledge-bases/policies/update
PATCH /orgs/{org_id}/k2/{kb_id}/policies/{policy_id}
Update an access-shaping rule on a knowledge base.
# Query
Source: https://docs.timbal.ai/api-reference/knowledge-bases/query
POST /orgs/{org_id}/k2/{kb_id}/query
Execute a SQL query against the knowledge base
# Restore
Source: https://docs.timbal.ai/api-reference/knowledge-bases/restores/create
POST /orgs/{org_id}/k2/{kb_id}/restores
Restore a knowledge base from a completed backup.
# Get Restore
Source: https://docs.timbal.ai/api-reference/knowledge-bases/restores/get
GET /orgs/{org_id}/k2/{kb_id}/restores/{restore_id}
Get a single restore.
# List Restores
Source: https://docs.timbal.ai/api-reference/knowledge-bases/restores/list
GET /orgs/{org_id}/k2/{kb_id}/restores
List restore attempts for a knowledge base.
# Get Schema
Source: https://docs.timbal.ai/api-reference/knowledge-bases/schema
GET /orgs/{org_id}/k2/{kb_id}/schema
Get the schema for a knowledge base
# Edit Knowledge Base
Source: https://docs.timbal.ai/api-reference/knowledge-bases/update
PATCH /orgs/{org_id}/k2/{kb_id}
Update a knowledge base.
# Upload Data
Source: https://docs.timbal.ai/api-reference/knowledge-bases/upload
POST /orgs/{org_id}/k2/{kb_id}/upload
Upload a data file into a knowledge base.
# List Organizations
Source: https://docs.timbal.ai/api-reference/me/list-orgs
GET /me/orgs
List organizations the authenticated user belongs to
# Delete Profile Photo
Source: https://docs.timbal.ai/api-reference/me/photo/delete
DELETE /me/photo
Clear the authenticated user's profile picture.
# Upload Profile Photo
Source: https://docs.timbal.ai/api-reference/me/photo/upload
PUT /me/photo
Upload and set the authenticated user's profile picture.
# Create API Token
Source: https://docs.timbal.ai/api-reference/me/tokens/create
POST /me/tokens
Create an API credential for the authenticated user.
# List API Tokens
Source: https://docs.timbal.ai/api-reference/me/tokens/list
GET /me/tokens
List the authenticated user's API credentials.
# Revoke API Token
Source: https://docs.timbal.ai/api-reference/me/tokens/revoke
DELETE /me/tokens/{token_id}
Revoke an API credential owned by the authenticated user.
# Update Profile
Source: https://docs.timbal.ai/api-reference/me/update
PATCH /me
Patch the authenticated user's profile.
# Org Analytics — Costs
Source: https://docs.timbal.ai/api-reference/orgs/analytics/costs
GET /orgs/{org_id}/analytics/costs
Time-binned credits and USD spend across an org, grouped by product surface.
# Org Analytics — Usage
Source: https://docs.timbal.ai/api-reference/orgs/analytics/usage
GET /orgs/{org_id}/analytics/usage
Time-binned run counts, user counts, and duration percentiles across an org.
# Org Analytics — Users
Source: https://docs.timbal.ai/api-reference/orgs/analytics/users
GET /orgs/{org_id}/analytics/users
Per-user run counts and full-ledger spend aggregates across an org.
# Get Credit Budgets
Source: https://docs.timbal.ai/api-reference/orgs/billing/budgets/get
GET /orgs/{org_id}/billing/budgets
Read credit budgets for the organization.
# Patch Credit Budgets
Source: https://docs.timbal.ai/api-reference/orgs/billing/budgets/patch
PATCH /orgs/{org_id}/billing/budgets
Merge partial credit budget changes for the organization.
# Replace Credit Budgets
Source: https://docs.timbal.ai/api-reference/orgs/billing/budgets/replace
PUT /orgs/{org_id}/billing/budgets
Replace the organization's full credit budget configuration.
# Get Compute Billing
Source: https://docs.timbal.ai/api-reference/orgs/billing/compute/get
GET /orgs/{org_id}/billing/compute
Get purchased compute machines and the org's compute pool usage.
# Set Compute Billing
Source: https://docs.timbal.ai/api-reference/orgs/billing/compute/put
PUT /orgs/{org_id}/billing/compute
Set the compute machine add-ons purchased by the organization.
# Update Overage Billing
Source: https://docs.timbal.ai/api-reference/orgs/billing/overage
POST /orgs/{org_id}/billing/overage
Update overage billing settings for the organization.
`enabled` turns usage-based overage on or off. `cap_credits` limits how many overage credits can accrue; omit it or set it to `null` for no cap.
To read current-period overage usage (invoiced vs pending credits), see [Get Overage Summary](/api-reference/orgs/billing/overage-summary).
# Get Overage Summary
Source: https://docs.timbal.ai/api-reference/orgs/billing/overage-summary
GET /orgs/{org_id}/billing/overage
Get overage credit usage for the current billing period.
# List Account Contacts
Source: https://docs.timbal.ai/api-reference/orgs/contacts/list
GET /orgs/{org_id}/contacts
Account-side contacts assigned to an organization.
Account-side contacts assigned to an organization.
# Sign Content URL
Source: https://docs.timbal.ai/api-reference/orgs/content/sign
POST /orgs/{org_id}/content/sign
Refresh a signed URL for stored content.
Refresh a signed URL for stored content. Pass a content URL previously returned by the API (signed or unsigned), or a bare object key. The server resolves it back to a known object, re-checks your access, and mints a fresh URL — useful when a cached signed URL has expired.
# Create Organization
Source: https://docs.timbal.ai/api-reference/orgs/create
POST /orgs
Create a new organization.
Self-serve callers may own at most one organization. Attempting to create a second org returns `429`; contact sales for additional orgs.
# List Credit Grants
Source: https://docs.timbal.ai/api-reference/orgs/credit-grants/list
GET /orgs/{org_id}/credit-grants
Credit grants awarded to an organization, most recent first.
Credit grants awarded to an organization, most recent first.
# Create Domain
Source: https://docs.timbal.ai/api-reference/orgs/domains/create
POST /orgs/{org_id}/domains
Add a custom domain to an organization
# Delete Domain
Source: https://docs.timbal.ai/api-reference/orgs/domains/delete
DELETE /orgs/{org_id}/domains/{domain_id}
Delete a domain
# Get Domain
Source: https://docs.timbal.ai/api-reference/orgs/domains/get
GET /orgs/{org_id}/domains/{domain_id}
Get a custom domain with current DNS and certificate status.
# List Domains
Source: https://docs.timbal.ai/api-reference/orgs/domains/list
GET /orgs/{org_id}/domains
List connected domains for an organization
# List Embedding Models
Source: https://docs.timbal.ai/api-reference/orgs/embedding-models/list
GET /orgs/{org_id}/embedding-models
List all available embedding models for the organization
# Get Organization
Source: https://docs.timbal.ai/api-reference/orgs/get
GET /orgs/{org_id}
Get an organization by ID
# List IAM Actions
Source: https://docs.timbal.ai/api-reference/orgs/iam/actions/list
GET /orgs/{org_id}/iam/actions
Catalog of grantable IAM actions for the organization.
# List Users by Action
Source: https://docs.timbal.ai/api-reference/orgs/iam/actions/users
GET /orgs/{org_id}/iam/actions/{action}/users
List members who effectively hold an org-wide IAM action.
# Check IAM Permissions
Source: https://docs.timbal.ai/api-reference/orgs/iam/check
POST /orgs/{org_id}/iam/check
Simulate authorization checks against a user's effective grants.
# List Users by Resource
Source: https://docs.timbal.ai/api-reference/orgs/iam/resources/users
GET /orgs/{org_id}/iam/resources/{resource}/users
List members with effective access to a resource, per action.
# Batch Update User Roles
Source: https://docs.timbal.ai/api-reference/orgs/iam/users/batch-roles
PUT /orgs/{org_id}/iam/users/roles
Replace role attachments for multiple members in one request.
# Get User Effective Grants
Source: https://docs.timbal.ai/api-reference/orgs/iam/users/effective-grants
GET /orgs/{org_id}/iam/users/{user_id}/effective-grants
List a member's effective IAM grants in the organization.
# Create Identity Provider
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/create
POST /orgs/{org_id}/identity-providers
Create an SSO or directory identity connection for the organization.
# Default Role
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/default-role
PUT /orgs/{org_id}/identity-providers/{provider_id}/default-role
Set or clear the SSO connection's default just-in-time role.
# Delete an SSO connection (cascades to its group mappings)
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/delete
DELETE /orgs/{org_id}/identity-providers/{provider_id}
Delete an SSO identity connection.
# Get a single SSO connection (secret masked)
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/get
GET /orgs/{org_id}/identity-providers/{provider_id}
Get an SSO identity connection by id.
# Map an external IdP group to a role for this connection
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/group-mappings/create
POST /orgs/{org_id}/identity-providers/{provider_id}/group-mappings
Map an external IdP group to a role for this connection
# Delete a group-to-role mapping
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/group-mappings/delete
DELETE /orgs/{org_id}/identity-providers/{provider_id}/group-mappings/{mapping_id}
Delete a group-to-role mapping
# List external-group to role mappings for a connection
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/group-mappings/list
GET /orgs/{org_id}/identity-providers/{provider_id}/group-mappings
List external-group to role mappings for a connection
# List Identity Providers
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/list
GET /orgs/{org_id}/identity-providers
List SSO identity connections for the organization.
# Update Identity Provider
Source: https://docs.timbal.ai/api-reference/orgs/identity-providers/update
PATCH /orgs/{org_id}/identity-providers/{provider_id}
Update an SSO identity connection.
# Create Model Policy
Source: https://docs.timbal.ai/api-reference/orgs/models/policies/create
POST /orgs/{org_id}/models/policies
Create a rule restricting which LLM providers or models the organization can use.
# Delete Model Policy
Source: https://docs.timbal.ai/api-reference/orgs/models/policies/delete
DELETE /orgs/{org_id}/models/policies/{policy_id}
Delete an LLM model or provider policy rule from the organization.
# List Model Policies
Source: https://docs.timbal.ai/api-reference/orgs/models/policies/list
GET /orgs/{org_id}/models/policies
List the organization's LLM model and provider usage policies.
# Delete Organization Logo
Source: https://docs.timbal.ai/api-reference/orgs/photo/delete
DELETE /orgs/{org_id}/photo
Clear the organization's logo.
# Upload Organization Logo
Source: https://docs.timbal.ai/api-reference/orgs/photo/upload
PUT /orgs/{org_id}/photo
Upload and set the organization's logo.
# Create Organization Role
Source: https://docs.timbal.ai/api-reference/orgs/roles/create
POST /orgs/{org_id}/iam/roles
Create a custom role for the organization.
# Delete Organization Role
Source: https://docs.timbal.ai/api-reference/orgs/roles/delete
DELETE /orgs/{org_id}/iam/roles/{role_id}
Delete a custom organization role.
# Get Organization Role
Source: https://docs.timbal.ai/api-reference/orgs/roles/get
GET /orgs/{org_id}/iam/roles/{role_id}
Retrieve a role and the full set of grants it carries.
# List Organization Roles
Source: https://docs.timbal.ai/api-reference/orgs/roles/list
GET /orgs/{org_id}/iam/roles
List system and custom roles for the organization.
# Update Organization Role
Source: https://docs.timbal.ai/api-reference/orgs/roles/update
PATCH /orgs/{org_id}/iam/roles/{role_id}
Partially update a custom organization role.
# Create SCIM Token
Source: https://docs.timbal.ai/api-reference/orgs/scim-tokens/create
POST /orgs/{org_id}/scim-tokens
Create a SCIM bearer token for the organization.
# List SCIM Tokens
Source: https://docs.timbal.ai/api-reference/orgs/scim-tokens/list
GET /orgs/{org_id}/scim-tokens
List SCIM tokens for the organization.
# Revoke SCIM Token
Source: https://docs.timbal.ai/api-reference/orgs/scim-tokens/revoke
DELETE /orgs/{org_id}/scim-tokens/{token_id}
Revoke a SCIM token
# List Templates
Source: https://docs.timbal.ai/api-reference/orgs/templates/list
GET /orgs/{org_id}/templates
Public project template catalog available to this organization.
# Update Organization
Source: https://docs.timbal.ai/api-reference/orgs/update
PATCH /orgs/{org_id}
Partially update an organization
Omitted fields are left unchanged. To explicitly clear a field, pass it as `null`.
# Add User
Source: https://docs.timbal.ai/api-reference/orgs/users/add
POST /orgs/{org_id}/iam/users
Invite one or more users to an organization
# Remove User
Source: https://docs.timbal.ai/api-reference/orgs/users/delete
DELETE /orgs/{org_id}/iam/users
Remove one or more members from an organization
# List Users
Source: https://docs.timbal.ai/api-reference/orgs/users/list
GET /orgs/{org_id}/iam/users
List members of an organization.
# Update User Roles
Source: https://docs.timbal.ai/api-reference/orgs/users/update
PUT /orgs/{org_id}/iam/users/{user_id}/roles
Replace a member's role attachments.
# Pagination
Source: https://docs.timbal.ai/api-reference/pagination
How to paginate through resources
The Timbal API uses **cursor-based pagination** with page tokens for efficient navigation through large collections.
## Making Requests
**Initial request:**
```http theme={"dark"}
GET /apps
```
**Subsequent requests:**
When a response includes a `next_page_token`, use it as the `page_token` query parameter:
```http theme={"dark"}
GET /apps?page_token=eyJpZCI6IjEyMyIsInRzIjoxNjk...
```
Page tokens are opaque strings—never parse or modify them.
# Cost Analytics
Source: https://docs.timbal.ai/api-reference/projects/analytics/credits
GET /orgs/{org_id}/projects/{project_id}/analytics/credits
Time-binned credits and USD cost breakdown for a project.
Only **project admins** can call this endpoint. Each response returns at most **1440** time bins; there is no pagination. You may omit `group_by`, `from`, and/or `to` — the server applies defaults — and the JSON body includes `group_by`, `from`, and `to` so you can see the bin size and range that were actually used.
# Usage Analytics
Source: https://docs.timbal.ai/api-reference/projects/analytics/usage
GET /orgs/{org_id}/projects/{project_id}/analytics/usage
Time-binned run counts, user counts, and duration percentiles for a project.
Only **project admins** can call this endpoint. Each response returns at most **1440** time bins; there is no pagination. You may omit `group_by`, `from`, and/or `to` — the server applies defaults — and the JSON body includes `group_by`, `from`, and `to` so you can see the bin size and range that were actually used.
# User Analytics
Source: https://docs.timbal.ai/api-reference/projects/analytics/users
GET /orgs/{org_id}/projects/{project_id}/analytics/users
Per-user run counts and project-attributed spend aggregates for a project.
Only **project admins** can call this endpoint. Results are **paginated**: pass `page_token` from the previous response's `next_page_token` to fetch the next page. The response includes `from` and `to` for the window you requested.
# Get App Auth Settings
Source: https://docs.timbal.ai/api-reference/projects/auth/get
GET /orgs/{org_id}/projects/{project_id}/auth
Get app sign-in settings for a project.
# Update Auth Provider
Source: https://docs.timbal.ai/api-reference/projects/auth/providers/update
PUT /orgs/{org_id}/projects/{project_id}/auth/providers/{provider}
Enable, disable, or configure an app sign-in provider.
# Update App Auth Settings
Source: https://docs.timbal.ai/api-reference/projects/auth/update
PATCH /orgs/{org_id}/projects/{project_id}/auth
Toggle app sign-in for a project.
# Delete Channel
Source: https://docs.timbal.ai/api-reference/projects/channels/delete
DELETE /orgs/{org_id}/projects/{project_id}/channels/{workforce}/{provider}
Disconnect a messaging channel binding, discarding its stored credentials.
# Upsert Channel
Source: https://docs.timbal.ai/api-reference/projects/channels/put
PUT /orgs/{org_id}/projects/{project_id}/channels/{workforce}/{provider}
Connect or update a messaging channel binding. Upserts on (workforce, provider); `credentials` is write-only — omit to keep the stored secrets, `null` to clear them, an object to replace them.
# Create Project
Source: https://docs.timbal.ai/api-reference/projects/create
POST /orgs/{org_id}/projects
Create a new project.
# Delete Project
Source: https://docs.timbal.ai/api-reference/projects/delete
DELETE /orgs/{org_id}/projects/{project_id}
Delete a project.
# Duplicate Project
Source: https://docs.timbal.ai/api-reference/projects/duplicate
POST /orgs/{org_id}/projects/{project_id}/duplicate
Duplicate a project, copying its code and knowledge base structure.
# Create Environment
Source: https://docs.timbal.ai/api-reference/projects/environments/create
POST /orgs/{org_id}/projects/{project_id}/envs
Create a new environment for a project.
# Delete Environment
Source: https://docs.timbal.ai/api-reference/projects/environments/delete
DELETE /orgs/{org_id}/projects/{project_id}/envs/{env_id}
Delete an environment.
# Deploy
Source: https://docs.timbal.ai/api-reference/projects/environments/deploy
POST /orgs/{org_id}/projects/{project_id}/envs/{env_id}/deploy
Deploy an environment to a revision.
# List Environment Deployments
Source: https://docs.timbal.ai/api-reference/projects/environments/deployments/list
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/deployments
List deployments for an environment.
# Get Metrics
Source: https://docs.timbal.ai/api-reference/projects/environments/get-metrics
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/metrics/{metric_type}
Query runtime metrics for an environment.
# List Environments
Source: https://docs.timbal.ai/api-reference/projects/environments/list
GET /orgs/{org_id}/projects/{project_id}/envs
List environments for a project with resolved per-component deploy config.
# Get Environment Logs
Source: https://docs.timbal.ai/api-reference/projects/environments/logs
GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/logs
Fetch runtime logs for a deployed project component.
# Stop
Source: https://docs.timbal.ai/api-reference/projects/environments/stop
POST /orgs/{org_id}/projects/{project_id}/envs/{env_id}/stop
Stop a running environment.
# Update Branch
Source: https://docs.timbal.ai/api-reference/projects/environments/update-branch
PATCH /orgs/{org_id}/projects/{project_id}/envs/{env_id}/branch
Set or clear the source branch for an environment.
# Update Config
Source: https://docs.timbal.ai/api-reference/projects/environments/update-config
PATCH /orgs/{org_id}/projects/{project_id}/envs/{env_id}/config
Update deploy configuration for an environment, including per-component overrides.
# Update Domain
Source: https://docs.timbal.ai/api-reference/projects/environments/update-domain
PATCH /orgs/{org_id}/projects/{project_id}/envs/{env_id}/domain
Attach, replace, or detach a custom domain for an environment.
# Get Project
Source: https://docs.timbal.ai/api-reference/projects/get
GET /orgs/{org_id}/projects/{project_id}
Get a project by ID.
# Link Knowledge Base
Source: https://docs.timbal.ai/api-reference/projects/link-knowledge-base
PUT /orgs/{org_id}/projects/{project_id}/k2
Link a knowledge base to a project.
# List Projects
Source: https://docs.timbal.ai/api-reference/projects/list
GET /orgs/{org_id}/projects
List projects in an organization.
Organization admins will retrieve a list of all projects in the organization. Other users will retrieve a list of projects they have access to.
Use the `view` query parameter to control the response detail level. The default `list` view returns a lightweight `ProjectPreview` per project. Pass `view=full` to get the same rich `ProjectDetail` shape returned by the [Get Project](/api-reference/projects/get) endpoint — including environments, knowledge bases, integrations, and workforce.
# Get Run
Source: https://docs.timbal.ai/api-reference/projects/runs/get
GET /orgs/{org_id}/projects/{project_id}/runs/{run_id}
Get a single project run by id or idempotency key.
# List Runs
Source: https://docs.timbal.ai/api-reference/projects/runs/list
GET /orgs/{org_id}/projects/{project_id}/runs
List runs for a project.
# Add Reaction
Source: https://docs.timbal.ai/api-reference/projects/runs/reactions-add
POST /orgs/{org_id}/projects/{project_id}/runs/{run_id}/reactions
Add a reaction to a project run.
# List Reactions
Source: https://docs.timbal.ai/api-reference/projects/runs/reactions-list
GET /orgs/{org_id}/projects/{project_id}/runs/{run_id}/reactions
List reactions on a project run.
# Unlink Knowledge Base
Source: https://docs.timbal.ai/api-reference/projects/unlink-knowledge-base
DELETE /orgs/{org_id}/projects/{project_id}/k2
Unlink the knowledge base from a project.
# Update Project
Source: https://docs.timbal.ai/api-reference/projects/update
PATCH /orgs/{org_id}/projects/{project_id}
Edit project metadata.
# Batch Create Variables
Source: https://docs.timbal.ai/api-reference/projects/vars/batch-create
POST /orgs/{org_id}/projects/{project_id}/vars/batch
Batch create project variables.
# Create Variable
Source: https://docs.timbal.ai/api-reference/projects/vars/create
POST /orgs/{org_id}/projects/{project_id}/vars
Create a project variable.
# Delete Variable
Source: https://docs.timbal.ai/api-reference/projects/vars/delete
DELETE /orgs/{org_id}/projects/{project_id}/vars/{var_id}
Delete a project variable.
# Get Variable
Source: https://docs.timbal.ai/api-reference/projects/vars/get
GET /orgs/{org_id}/projects/{project_id}/vars/{var_id}
Get a project variable by ID.
# List Variables
Source: https://docs.timbal.ai/api-reference/projects/vars/list
GET /orgs/{org_id}/projects/{project_id}/vars
List variables for a project.
# Update Variable
Source: https://docs.timbal.ai/api-reference/projects/vars/update
PATCH /orgs/{org_id}/projects/{project_id}/vars/{var_id}
Update a project variable.
# Call Workforce
Source: https://docs.timbal.ai/api-reference/projects/workforce/call
POST /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/{*path}
Proxy HTTP requests to a deployed workforce component.
This route is a **gateway proxy** to a **running** workforce deployment (an agent or workflow). The platform forwards your HTTP request to the deployment; responses pass back through unchanged (aside from normal proxy behavior).
The reference below is for **POST** so the API playground and generated fields render. The same path and parameters apply to **GET**, **PUT**, **PATCH**, and **DELETE** — use the verb you need when calling the API directly.
## Supported HTTP methods
| Method | Use case |
| ------------------- | --------------------------------------------------------------- |
| **GET** | Read-only calls (e.g. health, metadata). |
| **POST** | Typical RPC-style calls (e.g. `…/run`). |
| **PUT** / **PATCH** | Idempotent or partial updates when your component exposes them. |
| **DELETE** | Deletes or teardown actions your component exposes. |
Path, query, and auth are the same for every method. The deployment must handle the method you send.
## Path and query
* **`workforce`** — Which component to hit: numeric id, manifest UUID, or name (same identifiers you use elsewhere in the project API).
* **`path`** — Suffix on the deployment server after the component segment (for example `run` or `healthcheck`). Use the path your running app defines; can be empty for the deployment root.
* **`rev`** (query, required) — Git branch the deployment is tied to (for example `main`).
## Limits
* **WebSocket upgrades are not supported** on this proxy route.
* If nothing is running for that component on `rev`, you get **404**. If the deployment returns an error upstream, you may see **502**.
# Call Workforce (DELETE)
Source: https://docs.timbal.ai/api-reference/projects/workforce/call-delete
DELETE /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/{*path}
Proxy HTTP requests to a deployed workforce component.
This route is a **gateway proxy** to a **running** workforce deployment (an agent or workflow). See [Call Workforce (POST)](/api-reference/projects/workforce/call) for path, query, and limit details.
# Call Workforce (GET)
Source: https://docs.timbal.ai/api-reference/projects/workforce/call-get
GET /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/{*path}
Proxy HTTP requests to a deployed workforce component.
This route is a **gateway proxy** to a **running** workforce deployment (an agent or workflow). See [Call Workforce (POST)](/api-reference/projects/workforce/call) for path, query, and limit details.
# Call Workforce (PATCH)
Source: https://docs.timbal.ai/api-reference/projects/workforce/call-patch
PATCH /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/{*path}
Proxy HTTP requests to a deployed workforce component.
This route is a **gateway proxy** to a **running** workforce deployment (an agent or workflow). See [Call Workforce (POST)](/api-reference/projects/workforce/call) for path, query, and limit details.
# Call Workforce (PUT)
Source: https://docs.timbal.ai/api-reference/projects/workforce/call-put
PUT /orgs/{org_id}/projects/{project_id}/workforce/{workforce}/{*path}
Proxy HTTP requests to a deployed workforce component.
This route is a **gateway proxy** to a **running** workforce deployment (an agent or workflow). See [Call Workforce (POST)](/api-reference/projects/workforce/call) for path, query, and limit details.
# Create Workforce
Source: https://docs.timbal.ai/api-reference/projects/workforce/create
POST /orgs/{org_id}/projects/{project_id}/workforce
Create a workforce component on a branch revision.
# Delete Workforce
Source: https://docs.timbal.ai/api-reference/projects/workforce/delete
DELETE /orgs/{org_id}/projects/{project_id}/workforce
Delete a workforce component on a branch revision.
# List Workforce
Source: https://docs.timbal.ai/api-reference/projects/workforce/list
GET /orgs/{org_id}/projects/{project_id}/workforce
List workforce components on a branch revision.
# Context & State Management
Source: https://docs.timbal.ai/core-concepts/context
The shared memory that connects everything together across the entire lifetime of a run
The RunContext is the central storage and state management system for your runs. Beyond storing spans, it enables data sharing between components, input/output manipulation, and hierarchical data access across parent-child relationships.
## Accessing the RunContext
The RunContext is accessible from any callable within a Runnable's execution using `get_run_context()`. This includes:
* **Main handlers**: The main function that does the work (*i.e.* the function you pass to a Tool).
* **Default parameter callables**: Functions used to compute default values at runtime (as shown in [Runnables](./runnables#parameter-handling-and-basic-execution)).
* **Lifecycle hooks**: Functions that run before or after the main handler.
The `RunContext.current_span()` method returns the span object for the currently executing Runnable, containing all execution data - input parameters, output, timing, metadata, and any custom data you store on it. You get direct access to the live span object being built during execution.
Here's a simple example showing context access from a main handler:
```python highlight={7} theme={"dark"}
from datetime import datetime
import httpx
from timbal import Tool
from timbal.state import get_run_context
async def api_call(endpoint: str) -> dict:
span = get_run_context().current_span()
# Store request metadata for observability
span.endpoint = endpoint
span.request_start = datetime.now()
# Perform actual HTTP request
async with httpx.AsyncClient() as client:
response = await client.get(endpoint)
# Store response metadata for debugging/monitoring
span.response_status = response.status_code
span.request_duration = datetime.now() - span.request_start
return response.json()
api_tool = Tool(
name="api_call",
handler=api_call,
)
```
Beyond `current_span()`, the RunContext provides methods like `.parent_span()` and `.step_span()` to access parent or neighbor spans. We'll explore these methods in future sections when working with multi-step workflows and nested executions.
## Lifecycle Hooks
Beyond the main handler, Runnables support lifecycle hooks - functions that run at specific points during execution. These provide structured access points for context interaction and enable powerful data transformation patterns.
Every Runnable supports two optional hooks:
* **`pre_hook`**: A function that runs before the main handler
* **`post_hook`**: A function that runs after the main handler completes
Hooks can modify inputs, store custom data, and transform outputs - all while sharing the same RunContext.
### Pre-hooks: Modifying Input and Adding Context
A `pre_hook` runs before your handler and can both modify input parameters and store additional context data:
```python highlight={19} theme={"dark"}
from datetime import datetime
from timbal.state import get_run_context
def pre_hook():
span = get_run_context().current_span()
# Modify input parameters that will be passed to the handler
span.input["name"] = span.input["name"].capitalize()
# Add a new parameter
span.input["location"] = "Barcelona"
# Store custom data for later use
span.greet_time = datetime.now()
def greet(name: str, location: str) -> str:
return f"Hello {name} from {location}!"
greet_tool = Tool(
name="greet",
pre_hook=pre_hook,
handler=greet,
)
result = await greet_tool(name="alice").collect() # "Hello Alice from Barcelona!"
```
Pre-hooks are perfect for:
* **Data Preparation**: Process raw webhook payloads, parse JSON, or normalize input formats
* **Input Enhancement**: Enrich data with additional context from databases or APIs
* **Request Preprocessing**: Extract headers, validate signatures, or decode authentication tokens
* **State Initialization**: Set up execution context, timestamps, or tracking metadata
### Post-hooks: Processing Output After Completion
A `post_hook` runs after your handler and can access both input and output. You can also **modify or completely replace the output** by assigning a new value to `span.output`:
```python highlight={8} theme={"dark"}
def post_hook():
span = get_run_context().current_span()
# Retrieve custom data stored in pre_hook
greet_time = span.greet_time
print(f"Greeting at {greet_time}")
# Modify the output before it's returned
# You can assign any value to completely replace the handler's output
span.output = "Greeting overridden!"
greet_tool = Tool(
name="greet",
pre_hook=pre_hook,
handler=greet,
post_hook=post_hook,
)
result = await greet_tool(name="alice").collect() # "Greeting overridden!"
```
**Output Modification**: You can completely replace the output in a post-hook by assigning to `span.output`. The assigned value will be returned instead of the handler's original output. This is useful for transforming results, adding metadata, or implementing custom response formatting.
Post-hooks are perfect for:
* **Logging**: Record execution details and results
* **Metadata Storage**: Store processing metrics, timestamps, or analysis data
* **Output Modification**: Transform or enrich the final result
* **Cleanup Tasks**: Handle resource cleanup or state management
These simple examples show the basics. In practice, hooks excel at:
* **Input manipulation**: Processing webhooks where we don't control the shape of the incoming data.
* **Agent adaptation**: Converting between modalities (audio ↔ text) for different models.
More advanced patterns in the [Agents](../agents) section.
## Using Agents in Hooks
When you use an Agent inside a `pre_hook` or `post_hook`, you need to call `.nest()` to establish the proper hierarchical path for tracing and context management. This ensures the agent's execution is correctly nested under the parent agent's path.
Agents used as tools within another agent are automatically nested. However, agents used in hooks require manual nesting.
```python highlight={13} theme={"dark"}
from timbal import Agent
from timbal.state import get_run_context
# Agent that will be used in a hook
agent_is_company = Agent(
name="agent_is_company",
system_prompt="""Determine if the input is a company name.
Return only 'true' or 'false'.""",
model="openai/gpt-4.1-mini"
)
# Nest the agent under the parent agent's path
agent_is_company.nest("agent")
async def pre_hook():
span = get_run_context().current_span()
result = await agent_is_company(
prompt=f"Is '{span.input.get('name')}' a company?"
).collect()
span.is_company = result.output.collect_text().strip().lower() == "true"
agent = Agent(
name="agent",
model="openai/gpt-4.1-mini",
pre_hook=pre_hook
)
```
The `.nest()` method updates the agent's path hierarchy, ensuring proper tracing structure (e.g., `agent.agent_is_company` instead of just `agent_is_company`) and correct span nesting under the parent. Nested agents still run with **isolated memory** — they do not inherit the parent's conversation history (see [Nested Agent Memory](/agents/memory#nested-agent-memory)).
## Early Exit with bail()
Use `bail()` to exit early from any Runnable execution when validation fails or conditions aren't met. The `bail()` function raises an `EarlyExit` error that stops execution of the current runnable.
You can use `bail()` in:
* **Handlers**: Exit early from tool or agent handlers
* **Hooks**: Exit early from pre\_hook or post\_hook functions
* **Default parameter callables**: Exit early when computing default values
```python highlight={6} theme={"dark"}
from timbal.errors import bail
from timbal import Tool
def process_data(role: str) -> str:
if role != "admin":
bail("Admin access required")
return f"Processed: {role}"
tool = Tool(
name="process_data",
handler=process_data
)
```
This is useful for input validation, filtering unwanted requests, or implementing conditional logic in any Runnable.
# Events & Streaming
Source: https://docs.timbal.ai/core-concepts/events
Monitor execution in real-time and handle streaming results
## The Event Stream
In the previous [examples](./runnables#parameter-handling-and-basic-execution), we used `.collect()` to get the final result. When you call a Runnable, it doesn't just return the answer - it returns a stream of events that tell you what's happening step by step (an async generator). The `.collect()` method waits for all events and gives you just the final answer.
You can iterate through the async generator to process events in real-time:
```python theme={"dark"}
async for event in add_tool(a=5, b=3):
print(event)
# Output:
# StartEvent(run_id="068c4458382e79bb80006dc019ac3039", ...)
# OutputEvent(run_id="068c4458382e79bb80006dc019ac3039", output=8, ...)
```
This enables you to:
* Monitor execution progress in real-time
* Handle streaming responses from LLMs
* Debug execution flow
* Build reactive user interfaces
## Event Logging
In reality, you don't need to manually print events. By default, events are logged to standard output by the framework. You can control logging behavior with these environment variables:
* `TIMBAL_LOG_EVENTS`: Which events to log (default: `"START,OUTPUT"`)
* `TIMBAL_LOG_FORMAT`: Log format - `"dev"` for human-readable or `"json"` for structured (default)
* `TIMBAL_LOG_LEVEL`: Standard log level (default: `"INFO"`)
## Event Types
Events are the communication mechanism that Runnables use to stream information throughout their execution lifecycle. Every Runnable execution produces a sequence of events that can be consumed in real-time or collected for later processing.
Events are designed to be lightweight. For comprehensive execution data, see [Traces](./tracing.mdx).
Every execution produces at least a **Start** event (when it begins) and an **Output** event (when it finishes). LLMs and streaming operations also produce **Delta** events for intermediate results. Runs that pause for human input emit **Approval** or **Interaction** events.
The following examples show what these events look like for an LLM interaction:
### Start Event
Signals the beginning of an execution.
```python theme={"dark"}
from timbal.types.events import StartEvent
start_event = StartEvent(
run_id="068c4458382e79bb80006dc019ac3039",
parent_run_id=None,
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
parent_call_id="068c4458382e708f8000cc0f9b19d970",
)
```
These fields are present in all events to identify their source - **the framework handles this automatically**. We'll explore this in greater detail in advanced sections.
### Delta Events
Streaming content is emitted as typed **Delta Events** — structured, semantic updates about text, tool calls, thinking, and other content types.
#### Delta Event Structure
Each `DeltaEvent` contains an `item` property with a typed delta item. All delta items inherit from `DeltaItem` and have an `id` and `type` field:
```python theme={"dark"}
from timbal.types.events import DeltaEvent
from timbal.types.events.delta import Text, TextDelta, ToolUse
# Text block start (complete text content)
text_event = DeltaEvent(
run_id="068c4458382e79bb80006dc019ac3039",
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
item=Text(id="text_0", text="Hello")
)
# Text streaming delta
text_delta_event = DeltaEvent(
run_id="068c4458382e79bb80006dc019ac3039",
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
item=TextDelta(id="text_0", text_delta=" world")
)
# Tool call start
tool_event = DeltaEvent(
run_id="068c4458382e79bb80006dc019ac3039",
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
item=ToolUse(id="call_123", name="search", input="")
)
```
#### Delta Item Types
DeltaEvents use a pattern of paired types: a "start" type for the beginning of a content block, and a "delta" type for streaming incremental updates. All items have an `id` field to correlate deltas with their parent block.
**Text** - Start of a text content block:
```python theme={"dark"}
Text(
id="text_0",
text="Hello" # Initial text content
)
```
**TextDelta** - Streaming text increments:
```python theme={"dark"}
TextDelta(
id="text_0",
text_delta=" world" # Incremental text
)
```
**ToolUse** - Start of a tool call:
```python theme={"dark"}
ToolUse(
id="call_abc123",
name="search_web",
input="", # Initially empty
is_server_tool_use=False
)
```
**ToolUseDelta** - Streaming tool input parameters:
```python theme={"dark"}
ToolUseDelta(
id="call_abc123",
input_delta='{"query' # Partial JSON
)
```
LLMs stream tool input parameters incrementally. Multiple `ToolUseDelta` events will be emitted for a single tool call. You need to accumulate these deltas and parse the complete JSON afterwards.
**Thinking** - Start of LLM reasoning block:
```python theme={"dark"}
Thinking(
id="thinking_0",
thinking="Let me analyze this problem..."
)
```
**ThinkingDelta** - Streaming thinking increments:
```python theme={"dark"}
ThinkingDelta(
id="thinking_0",
thinking_delta="First, I'll consider..."
)
```
**Custom** - Arbitrary custom content:
```python theme={"dark"}
Custom(
id="custom_0",
data={"type": "image_progress", "percent": 45}
)
```
Use `Custom` for streaming content that doesn't fit standard types (e.g., multimodal content, provider-specific features, experimental data). This allows custom collectors and tools to emit arbitrary typed data while participating in the delta event system.
**ContentBlockStop** - Signals end of a content block:
```python theme={"dark"}
ContentBlockStop(id="text_0")
```
This event signals that a content block (text, tool use, thinking) has finished streaming. Use it to finalize processing of accumulated deltas.
#### Using Delta Events
```python theme={"dark"}
from timbal.types.events import DeltaEvent
from timbal.types.events.delta import (
Text, TextDelta, ToolUse, ToolUseDelta,
Thinking, ThinkingDelta, Custom, ContentBlockStop
)
async for event in agent(prompt="Hello"):
if isinstance(event, DeltaEvent):
item = event.item
# Type-safe handling of different content types
if isinstance(item, Text):
print(item.text, end="", flush=True)
elif isinstance(item, TextDelta):
print(item.text_delta, end="", flush=True)
elif isinstance(item, ToolUse):
print(f"\n[Calling {item.name}]")
elif isinstance(item, ToolUseDelta):
# Accumulate input_delta for later JSON parsing
pass
elif isinstance(item, Thinking):
print(f"\n[Thinking: {item.thinking}]")
elif isinstance(item, ThinkingDelta):
print(item.thinking_delta, end="", flush=True)
elif isinstance(item, Custom):
print(f"\n[Custom: {item.data}]")
elif isinstance(item, ContentBlockStop):
print(f"\n[Block {item.id} complete]")
```
#### Benefits of Delta Events
* **Type Safety**: Each delta item has specific fields and types
* **Semantic Information**: Know exactly what type of content is streaming
* **Better Observability**: Track tool calls, thinking, and text separately
* **Block Lifecycle**: Start and stop events for each content block
* **UI Flexibility**: Render different content types with appropriate components
* **Structured Logging**: Monitor different content types independently
### Approval and Interaction Events
When a run pauses for human input, the framework emits structured events before the final `OutputEvent`:
* **`ApprovalEvent`** — a `requires_approval` gate fired before a runnable executed. Resume with `resume={approval_id: True}` (or an `ApprovalResolution`).
* **`InteractionEvent`** — a handler called `suspend()` to ask the user for input. Resume with `resume={suspension_id: value}`.
See [Human in the Loop](/human-in-the-loop) for the full reference.
### Output Event
Contains the final result and signals completion. Contains either the final `output` result (if successful) or `error` information (if something went wrong).
```python theme={"dark"}
from timbal.types.events import OutputEvent
output_event = OutputEvent(
run_id="068c4458382e79bb80006dc019ac3039",
parent_run_id=None,
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
parent_call_id="068c4458382e708f8000cc0f9b19d970",
output={
"role": "assistant",
"content": [{"type": "text", "text": "Hello! How can I assist you today?"}]
},
error=None
)
```
## Handling Errors
When a Runnable encounters an error during execution, it **won't raise an exception**. Instead, it will always return an `OutputEvent` with error information. This ensures the event stream continues and you can handle errors gracefully:
```python theme={"dark"}
try:
result = await runnable(**kwargs).collect()
except Exception as e:
# This won't happen - errors are in the OutputEvent
pass
# Result will always be an instance of OutputEvent
if result.error:
print(f"Error: {result.error['message']}")
```
Errors will always have this structure:
```python theme={"dark"}
error = {
"type": "ValueError",
"message": "Invalid input provided",
"traceback": "Traceback (most recent call last):\\n..."
}
```
# Runnables
Source: https://docs.timbal.ai/core-concepts/runnables
Executable primitives that provide consistent interfaces for all Timbal components
## What is a Runnable?
A Runnable is an executable unit capable of processing inputs and producing outputs through an async generator interface. It works as a wrapper that turns any callable into a standardized, traceable, and composable execution unit.
All runnables provide a unified interface and execution pattern, enabling seamless composition regardless of their underlying implementation:
* **[Tools](#)** - Function wrappers with automatic schema generation and explicit parameter control
* **[Agents](#)** - Autonomous execution units that orchestrate LLM interactions with tool calling
* **[Workflows](#)** - Programmable execution pipelines that orchestrate step-by-step processing
All runnables must have a unique `name`. This name is used for tracing, debugging, and referencing the runnable in workflows, agents, and other components. Whether you're creating a Tool, an Agent, or a Workflow, the `name` parameter is required and must be unique within your application context.
Here's how to create a basic Tool:
```python theme={"dark"}
from timbal import Tool
def add(a: float, b: float) -> float:
"""Add two numbers together."""
return a + b
add_tool = Tool(
name="add_tool",
handler=add,
)
```
## Parameter Handling and Basic Execution
Runnables can be called as a regular python function. You can **pass parameters as keyword arguments**, and they'll be automatically mapped to the appropriate parameters in the underlying functions:
```python theme={"dark"}
result = await add_tool(a=5, b=3).collect() # Returns 8
```
You can also set default parameter values when creating the runnable:
```python theme={"dark"}
add_tool = Tool(
name="add_tool",
handler=add,
default_params={"b": 3}
)
result = await add_tool(a=5).collect() # Returns 8
```
Default parameters can also be callables that are evaluated at runtime:
```python theme={"dark"}
import random
add_tool = Tool(
name="add_tool",
handler=add,
default_params={"b": lambda: random.randint(0, 10)}
)
result = await add_tool(a=5).collect() # Returns a number between 5 - 15
```
Note that runtime parameters always override default values, including callable defaults:
```python theme={"dark"}
result = await add_tool(a=5, b=10).collect() # Returns 15
```
In the above example, `b=10` is passed at runtime and overrides the callable default that would generate a random number.
# Tracing & Observability
Source: https://docs.timbal.ai/core-concepts/tracing
Comprehensive execution tracing with input/output/error/timing capture for complete observability
Events are **not persisted to memory** - they are **immutable** temporary notifications that stream during execution. They provide immediate feedback about what's happening (*e.g.* start, progress chunks, completion) but are not stored permanently. Events are consumed as they're generated and are ideal for real-time monitoring, progress tracking, and streaming responses. For permanent storage and analysis, this information is captured in Traces.
## Traces
A Trace is the core data structure that captures execution information for every runnable execution, providing a complete audit trail of what happened. This tracing system enables complete observability into your application's behavior and performance.
For the previous LLM example, a single **span** inside the run's trace would look like this:
```python theme={"dark"}
from timbal.state.tracing.span import Span
llm_span = Span(
path="agent.llm",
call_id="068c4458383678be800031537a8df42e",
parent_call_id="068c4458382e708f8000cc0f9b19d970",
t0=1640995200000, # Start time (Unix ms)
t1=1640995201000, # End time (Unix ms)
input={
"model": "openai/gpt-4o-mini",
"messages": [{
"role": "user",
"content": [{"type": "text", "text": "Hello"}],
}],
},
output={
"role": "assistant",
"content": [{"type": "text", "text": "Hello! How can I assist you today?"}],
},
usage={
"openai/gpt-4o-mini:input_tokens": 32,
"openai/gpt-4o-mini:output_tokens": 10,
},
metadata={
"type": "LLM",
"model_provider": "openai",
"model_name": "gpt-4o-mini",
"ttft": 1.218775834015105, # Time To First Token
"tps": 103.44992571967803, # Tokens Per Second
},
)
```
Spans are stored in a `Trace` — a dict keyed by `call_id`. Access them during or after a run via `get_run_context()._trace` or helper methods like `step_span()`.
That's a single span, but spans don't usually come in isolation — they're organized into larger execution units: Runs.
## Runs
A Run represents the complete execution of your application - from the moment you invoke a runnable until it finishes. Think of it as the lifecycle of a single request or operation, whether that's a simple function call or a complex multi-step agent workflow.
Every Run is composed of a collection of spans:
* **Simple execution**: Running a single tool generates one span — the tool's input, output, timing, and metadata
* **Complex execution**: Running an agent generates multiple spans as it decides which tools to use, calls LLMs for reasoning, executes tools, and processes results
All spans within a run share the same `run_id` (via `RunContext.id`), creating a complete execution history that you can analyze, debug, or replay.
### The Run Context
Traces are stored in the RunContext - a central hub that manages all run data. You can access traces during execution, control storage behavior, and use this system to share data between different parts of your run.
More on this in the [Context](./context) section.
# Overview
Source: https://docs.timbal.ai/deployment/index
Run and deploy Timbal projects locally or in production
A Timbal project is three layers:
| Layer | Directory | Runtime |
| ------------- | ------------------- | ---------------------------------------------- |
| **UI** | `ui/` | React + Vite (Bun) |
| **API** | `api/` | Elysia (Bun) — routes requests to workforce |
| **Workforce** | `workforce//` | Python agents/workflows (`timbal.server.http`) |
Each workforce member has its own `timbal.yaml` manifest. The API and UI discover workforce services through env vars (`TIMBAL_START_WORKFORCE`, etc.) that `timbal start` wires up automatically.
## Local development
For day-to-day work, use [`timbal start`](/quickstart#run-locally). It:
* Detects `ui/`, `api/`, and every `workforce//` with a valid `timbal.yaml`
* Installs dependencies (`bun install`, `uv sync`)
* Picks free ports (defaults: UI `3737`, API `3000`, workforce from `4455`)
* Starts all services and multiplexes logs to one terminal
* Exposes interactive commands (`o` open UI, `s` status, `r` restart, `f`/`m` log focus/mute, `q` quit)
Override ports or env vars when needed:
```bash theme={"dark"}
timbal start --ui-port 4000 --api-port 3001 --port my-agent=4500
timbal start --env OPENAI_API_KEY=sk-... --env my-agent:DEBUG=1
timbal start --env-file .env.staging --env-file api:.env.api
```
## Environment variables
`timbal start` composes a separate environment for each service before spawning it. You do not need `--env-file` for the standard project layout — these files are **auto-loaded** when present:
| File | Scope |
| ----------------------- | ---------------------------------------------- |
| `/.env` | All services (UI, API, every workforce member) |
| `workforce//.env` | That workforce member only |
Missing files are skipped silently.
Override or add variables at the CLI:
```bash theme={"dark"}
timbal start --env DEBUG=1 # all services
timbal start --env api:LOG_LEVEL=debug # API only
timbal start --env-file .env.staging # extra file, all services
timbal start --env-file my-agent:.env.local # extra file, one member
```
`--env` and `--env-file` accept an optional scope prefix: `ui:`, `api:`, or a workforce member name. Unscoped entries apply globally.
### Precedence (low → high)
Later layers win over earlier ones:
1. Timbal built-ins (`TIMBAL_LOG_FORMAT`, `TIMBAL_API_KEY`, etc. from your CLI profile)
2. Inherited shell environment (`PORT` from your shell is ignored so it does not leak into services)
3. Auto-loaded `/.env`, then auto-loaded `workforce//.env` (members only)
4. `--env-file` (in flag order; scoped files apply only to that service)
5. `--env` (in flag order; scoped entries apply only to that service)
6. Runtime wiring that always wins (`PORT`, `TIMBAL_START_*`, `TIMBAL_WORKFORCE`)
Restart services (`r` in the `timbal start` terminal, or quit and re-run) after changing `.env` files — changes are read at startup, not hot-reloaded.
`.env` parsing is intentionally minimal: `KEY=VALUE` lines, optional `export` prefix, optional quotes, `#` comments. No variable expansion, line continuations, or shell escapes. For complex values, set the variable in your shell or use `--env`.
Vite and Bun may also read `ui/.env` and `api/.env` from those directories at dev time. Project-root `.env` is still the usual place for secrets shared across the stack (model API keys, integration tokens).
## Production
For production you have two paths:
Managed hosting — connect your git repo, deploy a branch, platform runs UI, API, and workforce for you
Run UI, API, and workforce on your own infra — you own process lifecycle, ports, env wiring, and logging
Validate locally with `timbal start` before deploying. The same project layout is what the platform builds from your repo.
# Timbal Platform
Source: https://docs.timbal.ai/deployment/platform
Deploy Timbal projects to managed hosting
The [Timbal Platform](https://app.timbal.ai) runs your project's UI, API, and workforce components for you. There is no local `timbal build` or Docker workflow — you push code to a connected git repo and deploy from the platform.
## Prerequisites
* A project scaffolded with [`timbal create`](/quickstart#create-a-timbal-project)
* Code pushed to a git remote the platform can access (typically GitHub)
* [Timbal CLI](/installation) installed (`timbal configure` for local credentials if needed)
* The project tested locally with [`timbal start`](/quickstart#run-locally)
## How deployment works
Each platform **project** maps to a git repository. **Environments** map to git branches (e.g. `main` → production, `staging` → staging). When you deploy an environment, the platform:
1. Checks out the requested git **rev** (branch)
2. Reads each `workforce//timbal.yaml` manifest
3. Builds and runs the UI, API, and every workforce member as separate deployable units
4. Routes traffic through a shared gateway (`proj-env-.deployments.timbal.ai`)
Scaffold locally, develop with `timbal start`, then push to your remote:
```bash theme={"dark"}
timbal create my-project
cd my-project
git remote add origin git@github.com:your-org/my-project.git
git push -u origin main
```
In the [Timbal Platform](https://app.timbal.ai), create a project linked to that repository. Create environments for the branches you want to deploy (production, staging, etc.).
Add environment variables and secrets in the platform dashboard — model API keys, integration tokens, anything sensitive. These are injected at runtime and are not baked into your repo.
Reference them in code with standard env var names (e.g. `OPENAI_API_KEY`).
Trigger a deploy for the target environment from the platform UI (or via the [Deploy API](/api-reference/projects/environments/deploy)). The platform deploys the git rev for that environment's branch.
Deployments are idempotent on the same rev. Re-deploy after pushing new commits to pick up changes.
## `timbal.yaml` manifest
Every workforce member needs a `timbal.yaml`. `timbal create` generates one automatically — do not remove the `_id` or `_type` fields.
```yaml workforce//timbal.yaml icon="y" theme={"dark"}
# Auto-generated. Do not modify _id / _type by hand unless you know what you're doing.
_id: "a1b2c3d4-..."
_type: "agent" # or "workflow"
build:
# Optional: apt packages for the platform build environment
# system_packages:
# - "libgl1-mesa-glx"
# Optional: commands run after the environment is set up
# run:
# - "echo env is ready!"
# Which runnable to serve (relative to this directory)
fqn: "agent.py::agent"
```
| Field | Purpose |
| ----------------------- | ---------------------------------------------------------------------- |
| `_id` | Stable manifest UUID — used for routing, logs, and deployment identity |
| `_type` | `agent` or `workflow` |
| `fqn` | Import spec for the Python runnable (`file.py::object`) |
| `build.system_packages` | Ubuntu packages installed during platform build |
| `build.run` | Shell commands run after dependency install |
Add additional workforce members locally with `timbal add` — each gets its own directory and manifest.
## Accessing a deployed project
Once live, the environment gateway exposes:
* **UI** at the environment root
* **API** at `/api/...`
* **Workforce** at `/api/workforce//...` (proxied to each member's HTTP server)
For programmatic access outside the gateway, the platform also exposes run endpoints:
```http theme={"dark"}
POST https://dev.timbal.ai/orgs/{org_id}/apps/{app_id}/runs/collect
POST https://dev.timbal.ai/orgs/{org_id}/apps/{app_id}/runs/stream
```
See the [API Reference](/api-reference/introduction) for auth, request shapes, and SDK usage.
## Monitoring and logs
The platform dashboard provides:
* **Deployment status** per component (UI, API, workforce members)
* **Runtime logs** filtered by component
* **Resource usage** and deploy history
You can also fetch logs via the [environment logs API](/api-reference/projects/environments/logs).
Even when self-hosting workforce runtimes, you can point tracing at the platform — see [Tracing](/core-concepts/tracing).
# Self-Hosted
Source: https://docs.timbal.ai/deployment/self-hosted
Run Timbal projects on your own infrastructure
Self-hosting means you run the same three layers a `timbal create` project contains — **UI**, **API**, and **workforce** — without `timbal start` managing them. You own process lifecycle, port allocation, env wiring, restarts, and log aggregation.
There is no `timbal build` or CLI-generated Docker image. You bring your own process manager (systemd, supervisord, Kubernetes, etc.) and production build pipeline.
## Prerequisites
* A project created with [`timbal create`](/quickstart#create-a-timbal-project)
* [Bun](https://bun.sh) for `api/` and `ui/`
* [uv](https://docs.astral.sh/uv/) for workforce Python deps
* `timbal[server]` installed in each workforce member's venv
## What `timbal start` does (that you must replicate)
`timbal start` is the reference implementation. When self-hosting, reproduce the same wiring:
### 1. Workforce members
For each `workforce//` with a `timbal.yaml`:
```bash theme={"dark"}
cd workforce/
uv sync
uv run -m timbal.server.http --port --import_spec
```
`` comes from `timbal.yaml` (e.g. `agent.py::agent`). The HTTP server exposes:
```http theme={"dark"}
POST http://localhost:/run # collect (returns final OutputEvent)
POST http://localhost:/stream # SSE event stream
GET http://localhost:/healthcheck
```
### 2. API
```bash theme={"dark"}
cd api
bun install
PORT= bun run dev # dev; use your production start command in prod
```
### 3. UI (if present)
```bash theme={"dark"}
cd ui
bun install
bun run dev --port # dev; use your production build/serve in prod
```
### 4. Cross-service env vars
The API and UI need to know where workforce members live. `timbal start` injects these — set them yourself when self-hosting:
| Variable | Example | Purpose |
| ------------------------ | ------------------------------------- | --------------------------------------------- |
| `TIMBAL_START_WORKFORCE` | `a1b2c3d4-...:4455,b5c6d7e8-...:4456` | Manifest `_id` → port map (comma-separated) |
| `TIMBAL_START_API_PORT` | `3000` | API port (for services that call back to API) |
| `TIMBAL_START_UI_PORT` | `3737` | UI port |
| `PORT` | per-service | Port each Bun service binds to |
`TIMBAL_START_WORKFORCE` uses the `_id` from each member's `timbal.yaml`, not the directory name.
Also pass through model keys and integration secrets (`OPENAI_API_KEY`, etc.). Locally, `timbal start` auto-loads `/.env` and `workforce//.env` into each process; when self-hosting you must export or inject those variables yourself (e.g. via your process manager or a secrets store). See [Environment variables](/deployment#environment-variables).
If any of `TIMBAL_START_*` are missing, platform SDK helpers that resolve service URLs will fail. Mirror what `timbal start` sets before debugging routing issues.
## Production considerations
### Process management
Run each component under a supervisor that restarts on crash:
* One process per workforce member
* One for API
* One for UI (if used)
Use health checks against `/healthcheck` on workforce HTTP servers.
### Logging
`timbal start` multiplexes stdout/stderr into one terminal with per-service prefixes and `f`/`m` focus controls. Self-hosted, you need your own approach:
* Separate log files or streams per component
* Structured logging if shipping to Datadog / CloudWatch / etc.
* Correlate by request ID if the API fans out to multiple workforce members
### Networking
* Put a reverse proxy (nginx, Caddy, etc.) in front of UI and API for TLS
* Workforce members can stay internal — only the API needs to reach them
* Lock down ports so workforce HTTP servers aren't exposed publicly
### Builds
The scaffold uses `bun run dev` for API/UI. For production, use whatever production build your scaffolded `api/` and `ui/` packages define (static UI build + API server, etc.). The workforce side stays `uv run -m timbal.server.http` with a pinned `uv sync` environment.
### Observability
Platform tracing still works from self-hosted runtimes — configure a [tracing provider](/core-concepts/tracing) to export spans to Timbal or your own OTLP endpoint.
## When to self-host
Self-hosting makes sense when you need full control over networking, data residency, or custom orchestration. For most teams, the [Timbal Platform](/deployment/platform) is less operational overhead — same project layout, no component wiring to maintain.
# Overview
Source: https://docs.timbal.ai/evals/index
Automated testing framework for Timbal agents and runnables
## What are Evals?
Evals are automated tests that validate your agent's behavior, outputs, and execution patterns. They help you ensure your agents perform correctly and consistently across different scenarios.
## Why Evals Matter
AI agents are non-deterministic - the same input can yield different results. Evals help you:
* **Validate outputs**: Ensure agents produce correct responses
* **Check tool usage**: Verify agents use the right tools with correct inputs
* **Monitor performance**: Track execution time and token usage
* **Catch regressions**: Prevent breaking changes during development
* **Test execution patterns**: Validate sequential and parallel tool execution
## How Evals Work
Timbal's eval system uses a YAML-based test definition format with a powerful validator system:
```yaml theme={"dark"}
- name: time_in_madrid
description: Test that agent returns the time in Madrid
runnable: agent.py::agent
tags: ["datetime", "smoke"]
timeout: 30000
params:
prompt: "what time is it in madrid"
output:
type!: "string"
contains!: ":"
pattern!: "\\d{1,2}:\\d{2}"
elapsed:
lt!: 6000
seq!:
- llm
- get_datetime
- llm
```
### Core Components
20+ validators for checking outputs, patterns, types, and more
Validate execution sequences and parallel tool calls
AI-powered semantic validation for natural language
Command-line interface for running and discovering evals
## Prerequisites
Before running evals, ensure the following environment variables are set:
* `TIMBAL_API_KEY` - Your Timbal API key
* `TIMBAL_API_HOST` - The Timbal API host URL
* `TIMBAL_ORG_ID` - Your organization ID
These can be set in your environment or in a `.env` file in your project directory.
## Quick Start
### 1. Create a test file
Create a file named `eval_greeting.yaml`:
```yaml theme={"dark"}
- name: greeting_test
description: Verify the agent greets users appropriately
runnable: agent.py::my_agent
params:
prompt: "Hi there!"
output:
not_null!: true
type!: "string"
prompt!: "The response greets the user politely"
elapsed:
lt!: 5000
```
### 2. Run your evals
```bash theme={"dark"}
python -m timbal.evals.cli path/to/eval_greeting.yaml
```
### 3. View results
The CLI displays pytest-style output with pass/fail status, duration, and detailed failure information:
```
========================= timbal evals =========================
collected 1 eval
eval_greeting.yaml
greeting_test ......................................... PASSED (0.45s)
tags: greeting, basic
├── output
│ ├── not_null! ✓
│ ├── type! ✓
│ └── prompt! ✓
└── elapsed
└── lt! ✓
========================= 1 passed in 0.45s =========================
```
## Eval Structure
Each eval consists of:
| Field | Description | Required |
| ------------- | -------------------------------------- | -------- |
| `name` | Unique identifier for the eval | Yes |
| `runnable` | Path to the runnable (`file.py::name`) | Yes |
| `params` | Input parameters for the runnable | No |
| `description` | Human-readable description | No |
| `tags` | List of tags for filtering | No |
| `timeout` | Maximum execution time in milliseconds | No |
| `output` | Validators for the final output | No |
| `elapsed` | Validators for total execution time | No |
| `seq!` | Sequence flow validator | No |
Eval names must be unique across all eval files. The CLI will error if duplicate names are found.
See [Writing Evals](/evals/writing-evals) for the complete syntax reference.
## Next Steps
Learn the full eval syntax and best practices
Complete reference for all validators
CLI options and CI/CD integration
# Running Evals
Source: https://docs.timbal.ai/evals/running-evals
CLI usage, file discovery, and CI/CD integration
Timbal provides a command-line interface for discovering and running evals. This guide covers all CLI options and integration patterns.
## Basic Usage
Run evals from the command line:
```bash theme={"dark"}
# Run all evals in a directory
python -m timbal.evals.cli path/to/evals/
# Run a specific eval file
python -m timbal.evals.cli path/to/eval_search.yaml
# Run a single eval by name (pytest-style)
python -m timbal.evals.cli path/to/eval_search.yaml::my_eval_name
# Run with a specific runnable
python -m timbal.evals.cli --runnable agent.py::my_agent path/to/evals/
```
## CLI Options
| Option | Description |
| -------------------- | ------------------------------------------------------------------ |
| `path` | Path to eval file or directory (positional) |
| `--runnable` | Fully qualified name of the runnable (`file.py::name`) |
| `--log-level` | Logging level (DEBUG, INFO, WARNING, ERROR) |
| `-s`, `--no-capture` | Disable stdout/stderr capture (show output live) |
| `-t`, `--tags` | Filter evals by tags (comma-separated) |
| `-f`, `--format` | Output format: `pretty` (default) or `json` (streams JSONL events) |
| `-j`, `--jobs` | Run up to N evals concurrently (default 1) |
| `-o`, `--output` | Write the full results document as JSON to a file (`-` for stdout) |
| `-V`, `--version` | Show version information |
### Runnable Specification
The runnable can be specified in two ways:
**1. Via CLI flag:**
```bash theme={"dark"}
python -m timbal.evals.cli --runnable agents/search.py::search_agent evals/
```
**2. Per-eval in YAML:**
```yaml theme={"dark"}
- name: test_search
runnable: agents/search.py::search_agent
params:
prompt: "Find products"
output:
not_null!: true
```
Per-eval runnable overrides the CLI flag.
### Tag Filtering
Filter evals by tags using `-t` or `--tags`:
```bash theme={"dark"}
# Run only evals with 'smoke' tag
python -m timbal.evals.cli -t smoke evals/
# Run evals matching ANY of the specified tags
python -m timbal.evals.cli -t smoke,fast evals/
# Combine with other options
python -m timbal.evals.cli --tags regression -s evals/
```
Evals matching **any** of the specified tags will run.
### Output Capture
By default, stdout/stderr from your agent is captured and only shown on failure. Use `-s` to see output live:
```bash theme={"dark"}
# Captured (default) - cleaner output
python -m timbal.evals.cli evals/
# Live output - useful for debugging
python -m timbal.evals.cli -s evals/
```
### Streaming JSON Output
Use `--format json` to stream structured events instead of the rich terminal report. One JSON object is written per line (JSONL) to stdout, and each `result` event is emitted **as the eval completes** — human-readable output goes to stderr, so stdout stays parseable:
```bash theme={"dark"}
python -m timbal.evals.cli evals/ --format json
```
```json theme={"dark"}
{"event": "start", "total": 2, "evals": [{"name": "greeting_test", "path": "evals/eval_basic.yaml"}, ...]}
{"event": "result", "name": "greeting_test", "passed": true, "duration": 1.2, "output": ..., "validators": [...]}
{"event": "result", "name": "search_test", "passed": false, "duration": 3.4, ...}
{"event": "summary", "total": 2, "passed": 1, "failed": 1, "total_duration": 4.6}
```
#### Event Reference
Every line is a JSON object with an `event` field: `start` (once, before any eval runs), `result` (once per eval), `summary` (once, at the end).
**`start`**
| Field | Type | Description |
| ------- | ---- | ------------------------------------------- |
| `total` | int | Number of evals that will run |
| `evals` | list | `{name, path}` for each eval, in file order |
**`result`** — one per eval. With `--jobs 1` results arrive in file order; with `--jobs > 1` they arrive in **completion order**, so use `name` to correlate.
| Field | Type | Description |
| ----------------- | -------------- | --------------------------------------------------------------- |
| `name` | string | Eval name |
| `path` | string | Eval file the eval was loaded from |
| `description` | string \| null | Eval description |
| `tags` | list | Eval tags |
| `passed` | bool | Overall verdict (no error and all validators passed) |
| `duration` | float | Wall-clock seconds for this eval |
| `error` | object \| null | `{type, message, traceback}` when the run itself failed |
| `params` | object | Input params the runnable was called with |
| `output` | any | Final runnable output (messages serialize to `{role, content}`) |
| `usage` | object | Token counts keyed by `provider/model:metric` |
| `validators` | list | Per-validator verdicts (see below) |
| `captured_stdout` | string | Stdout captured during the eval (empty with `-s`) |
| `captured_stderr` | string | Stderr captured during the eval (empty with `-s`) |
Each entry in `validators`:
| Field | Type | Description |
| -------------- | -------------- | ------------------------------------------------------------------ |
| `target` | string | Span/property path the validator ran against (e.g. `agent.output`) |
| `name` | string | Validator name (e.g. `contains!`, `prompt!`) |
| `value` | any | Expected value from the eval YAML |
| `passed` | bool | Whether the check passed |
| `evaluated` | bool | `false` when the validator was invalid and never executed |
| `error` | string \| null | Failure reason |
| `actual_value` | any | Resolved value (populated for LLM validators) |
**`summary`**
| Field | Type | Description |
| ----------------------------- | ----- | -------------------------------- |
| `total` / `passed` / `failed` | int | Eval counts |
| `total_duration` | float | Sum of eval durations in seconds |
The document written by `-o/--output` has the same shape: the `summary` fields at the top level plus a `results` list, where each entry matches the `result` event (minus the `event` key).
### Parallel Execution
Use `-j`/`--jobs` to run evals concurrently. Results are reported in **completion order**, which pairs naturally with `--format json` for streaming consumers:
```bash theme={"dark"}
# Run up to 4 evals at a time, streaming results as they finish
python -m timbal.evals.cli evals/ --format json -j 4
```
Per-eval stdout/stderr capture works in parallel mode — output is attributed to the eval that produced it.
### Saving Results
Use `-o`/`--output` to write the full results document as JSON after the run:
```bash theme={"dark"}
# Write to a file (works with both formats)
python -m timbal.evals.cli evals/ -o results.json
# Print the document to stdout (pretty format only; report goes to stderr)
python -m timbal.evals.cli evals/ -o -
```
## File Discovery
### Naming Patterns
Timbal discovers eval files matching these patterns:
* `eval*.yaml` - e.g., `eval_search.yaml`, `evals.yaml`
* `*eval.yaml` - e.g., `search_eval.yaml`, `my_eval.yaml`
### Directory Structure
```
project/
├── agents/
│ ├── search.py
│ └── support.py
└── evals/
├── eval_search.yaml
├── eval_support.yaml
└── regression/
├── eval_smoke.yaml
└── eval_full.yaml
```
Run all evals:
```bash theme={"dark"}
python -m timbal.evals.cli evals/
```
Run specific subset:
```bash theme={"dark"}
python -m timbal.evals.cli evals/regression/
```
### Configuration File
Create an `evalconf.yaml` in your project root for shared configuration:
```yaml theme={"dark"}
# evalconf.yaml
runnable: agents/main.py::agent
log_level: INFO
```
Timbal walks up the directory tree looking for `evalconf.yaml`.
## Output Format
### Successful Run
```
========================= timbal evals =========================
collected 5 evals
eval_search.yaml
basic_search .......................................... PASSED (0.45s)
tags: search, smoke
├── output
│ ├── not_null! ✓
│ └── contains! ✓
└── elapsed
└── lt! ✓
advanced_search ....................................... PASSED (0.82s)
tags: search, regression
├── output
│ ├── prompt! ✓
│ └── min_length! ✓
└── seq!
├── llm ✓
├── search_products ✓
└── llm ✓
========================= 5 passed in 2.34s =========================
```
### Failed Run
```
========================= timbal evals =========================
collected 3 evals
eval_validation.yaml
input_validation ...................................... FAILED (0.23s)
tags: validation
├── output
│ ├── not_null! ✓
│ └── contains! ✗
========================= FAILURES =========================
eval_validation.yaml::input_validation
-----------------------------------------
Captured stdout:
Processing input...
Captured stderr:
Warning: deprecated function used
Failed validators:
- output -> contains!
Expected: "validated"
Actual: "The input was processed successfully"
Error: Value does not contain expected substring
========================= 1 failed, 2 passed in 1.56s =========================
```
## Exit Codes
| Code | Meaning |
| ---- | ---------------------------- |
| 0 | All evals passed |
| 1 | One or more evals failed |
| 2 | Configuration or setup error |
## CI/CD Integration
### GitHub Actions
```yaml theme={"dark"}
# .github/workflows/evals.yaml
name: Run Evals
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
evals:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install -e .
pip install timbal
- name: Run evals
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
python -m timbal.evals.cli --runnable agent.py::agent evals/ -j 4 -o results.json
- name: Upload results
if: always()
uses: actions/upload-artifact@v4
with:
name: eval-results
path: results.json
```
### GitLab CI
```yaml theme={"dark"}
# .gitlab-ci.yml
evals:
stage: test
image: python:3.11
script:
- pip install -e .
- pip install timbal
- python -m timbal.evals.cli --runnable agent.py::agent evals/
variables:
OPENAI_API_KEY: $OPENAI_API_KEY
```
### Pre-commit Hook
```bash theme={"dark"}
#!/bin/bash
# .git/hooks/pre-commit
echo "Running evals..."
python -m timbal.evals.cli evals/smoke/
if [ $? -ne 0 ]; then
echo "Evals failed. Commit aborted."
exit 1
fi
```
## Debugging
### Verbose Output
```bash theme={"dark"}
# Show debug logging
python -m timbal.evals.cli --log-level DEBUG evals/
# Show live output
python -m timbal.evals.cli -s evals/
```
### Run Single Eval
Test a specific eval during development:
```bash theme={"dark"}
# Run one file
python -m timbal.evals.cli evals/eval_search.yaml
# Run a specific eval by name (pytest-style syntax)
python -m timbal.evals.cli evals/eval_search.yaml::basic_search
```
### Environment Variables
Set environment variables for testing:
```bash theme={"dark"}
# Via shell
export API_KEY="test-key"
python -m timbal.evals.cli evals/
# Or in eval file
- name: test_with_key
runnable: agent.py::agent
env:
API_KEY: "test-key"
params:
prompt: "Fetch data"
output:
not_null!: true
```
## Best Practices
Separate fast smoke tests from slow regression tests:
```
evals/
├── smoke/ # Fast tests, run on every commit
│ └── eval_basic.yaml
└── regression/ # Comprehensive tests, run nightly
└── eval_full.yaml
```
```bash theme={"dark"}
# Quick check
python -m timbal.evals.cli evals/smoke/
# Full suite
python -m timbal.evals.cli evals/
```
Check exit codes in scripts:
```bash theme={"dark"}
python -m timbal.evals.cli evals/
status=$?
if [ $status -eq 0 ]; then
echo "All evals passed!"
elif [ $status -eq 1 ]; then
echo "Some evals failed"
exit 1
else
echo "Configuration error"
exit 2
fi
```
Prevent hanging tests with timeouts (in milliseconds):
```yaml theme={"dark"}
- name: slow_test
runnable: agent.py::agent
timeout: 60000 # 60 second timeout
params:
prompt: "Complex query"
output:
not_null!: true
```
Organize evals with tags:
```yaml theme={"dark"}
- name: smoke_test
runnable: agent.py::agent
tags:
- smoke
- quick
params:
prompt: "Hi"
output:
not_null!: true
```
# Comparison Validators
Source: https://docs.timbal.ai/evals/validators/comparison
Validators for equality, containment, patterns, and numeric comparisons
Comparison validators check values against expected content using various matching strategies.
All string comparison validators support [transforms](/evals/validators#transforms) and [negation](/evals/validators#negation).
## Equality
### eq!
Checks for exact equality between the actual and expected values.
```yaml theme={"dark"}
output:
eq!: "Hello, world!"
```
| Parameter | Type | Description |
| --------- | ---- | ------------------------ |
| value | any | The exact value to match |
```yaml theme={"dark"}
# String equality
output:
eq!: "Success"
# Case-insensitive equality with transform
output:
eq!:
value: "success"
transform: lowercase
# Numeric equality
get_calculate:
output:
eq!: 42
# Boolean equality
validate_input:
output:
eq!: true
# In span input
get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
```
### ne!
Checks that the actual value does NOT equal the expected value. This is the negated form of `eq!`.
```yaml theme={"dark"}
output:
ne!: "error"
```
| Parameter | Type | Description |
| --------- | ---- | ----------------------------- |
| value | any | The value that must NOT match |
```yaml theme={"dark"}
# Ensure response is not empty
output:
ne!: ""
# Case-insensitive not-equal
output:
ne!:
value: "error"
transform: lowercase
```
## String Matching
### contains!
Checks if the actual value contains the expected substring or item.
```yaml theme={"dark"}
output:
contains!: "success"
```
| Parameter | Type | Description |
| --------- | ---- | ------------------------- |
| value | any | Substring or item to find |
```yaml theme={"dark"}
# Substring check
output:
contains!: "order confirmed"
# Case-insensitive contains with transform
output:
contains!:
value: "success"
transform: lowercase
# Check output contains time format
output:
contains!: ":"
```
### not\_contains!
Checks that the actual value does NOT contain the expected substring or item.
```yaml theme={"dark"}
output:
not_contains!: "error"
```
| Parameter | Type | Description |
| --------- | ---- | ------------------------------------- |
| value | any | Substring or item that must be absent |
```yaml theme={"dark"}
# Ensure no error messages
output:
not_contains!: "error"
# Case-insensitive with transform
output:
not_contains!:
value: "error"
transform: lowercase
# Ensure no sensitive data in tool input
log_message:
input:
message:
not_contains!: "password"
```
### contains\_all!
Checks if the actual value contains ALL of the expected substrings or items.
```yaml theme={"dark"}
output:
contains_all!: ["time", "date", "timezone"]
```
| Parameter | Type | Description |
| --------- | ---- | ------------------------------------------------- |
| value | list | List of substrings/items that must ALL be present |
```yaml theme={"dark"}
# Check all required fields mentioned
output:
contains_all!: ["name", "email", "phone"]
# Case-insensitive with transform
output:
contains_all!:
value: ["madrid", "time"]
transform: lowercase
```
### not\_contains\_all!
Checks that the actual value does NOT contain all of the specified items (at least one must be missing). This is the negated form of `contains_all!`.
```yaml theme={"dark"}
output:
not_contains_all!: ["Paris", "London", "Tokyo"]
```
| Parameter | Type | Description |
| --------- | ---- | ----------------------------------------------- |
| value | list | List of items where at least one must be absent |
```yaml theme={"dark"}
# Ensure not all competitor names are mentioned
output:
not_contains_all!: ["CompetitorA", "CompetitorB", "CompetitorC"]
# With transform
output:
not_contains_all!:
value: ["error", "warning", "critical"]
transform: lowercase
```
### contains\_any!
Checks if the actual value contains AT LEAST ONE of the expected substrings or items.
```yaml theme={"dark"}
output:
contains_any!: ["success", "completed", "done"]
```
| Parameter | Type | Description |
| --------- | ---- | ----------------------------------------------------------- |
| value | list | List of substrings/items where at least one must be present |
```yaml theme={"dark"}
# Check for any success indicator
output:
contains_any!: ["success", "ok", "completed"]
# Case-insensitive with transform
output:
contains_any!:
value: ["yes", "confirmed", "approved"]
transform: lowercase
```
### not\_contains\_any!
Checks that the actual value contains NONE of the specified items. This is the negated form of `contains_any!`.
```yaml theme={"dark"}
output:
not_contains_any!: ["error", "failed", "exception"]
```
| Parameter | Type | Description |
| --------- | ---- | ------------------------------------- |
| value | list | List of items that must ALL be absent |
```yaml theme={"dark"}
# Ensure no error-related words
output:
not_contains_any!: ["error", "failed", "exception", "invalid"]
# With transform
output:
not_contains_any!:
value: ["ERROR", "FAILED"]
transform: uppercase
```
### pattern!
Checks if the actual value matches a regular expression pattern.
```yaml theme={"dark"}
output:
pattern!: "Order #\\d{6}"
```
| Parameter | Type | Description |
| --------- | ------ | -------------------------- |
| value | string | Regular expression pattern |
```yaml theme={"dark"}
# Match time format
output:
pattern!: "\\d{1,2}:\\d{2}"
# Match ISO date format
get_datetime:
output:
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
# Match price format
output:
pattern!: "\\$\\d+\\.\\d{2}"
```
Regular expressions use Python's `re` module syntax. Remember to escape special characters in YAML (use `\\d` instead of `\d`).
### not\_pattern!
Checks that the actual value does NOT match the regular expression pattern. This is the negated form of `pattern!`.
```yaml theme={"dark"}
output:
not_pattern!: "^Error:"
```
| Parameter | Type | Description |
| --------- | ------ | ---------------------------------------------- |
| value | string | Regular expression pattern that must NOT match |
```yaml theme={"dark"}
# Ensure no error prefix
output:
not_pattern!: "^Error:"
# Ensure no failed status
output:
not_pattern!: "\\bfailed\\b"
# Case-insensitive with transform
output:
not_pattern!:
value: "^error"
transform: lowercase
```
### starts\_with!
Checks if the actual value starts with the expected prefix.
```yaml theme={"dark"}
output:
starts_with!: "Hello"
```
| Parameter | Type | Description |
| --------- | ------ | --------------- |
| value | string | Expected prefix |
```yaml theme={"dark"}
# Check timezone prefix
get_datetime:
input:
timezone:
starts_with!: "Europe/"
# Check response starts correctly
output:
starts_with!: "The current time"
# Case-insensitive with transform
output:
starts_with!:
value: "the current"
transform: lowercase
```
### not\_starts\_with!
Checks that the actual value does NOT start with the expected prefix. This is the negated form of `starts_with!`.
```yaml theme={"dark"}
output:
not_starts_with!: "Error"
```
| Parameter | Type | Description |
| --------- | ------ | ------------------------------- |
| value | string | Prefix that must NOT be present |
```yaml theme={"dark"}
# Ensure no error prefix
output:
not_starts_with!: "Error:"
# Ensure no apology
output:
not_starts_with!: "Sorry"
# Case-insensitive with transform
output:
not_starts_with!:
value: "error"
transform: lowercase
```
### ends\_with!
Checks if the actual value ends with the expected suffix.
```yaml theme={"dark"}
output:
ends_with!: "."
```
| Parameter | Type | Description |
| --------- | ------ | --------------- |
| value | string | Expected suffix |
```yaml theme={"dark"}
# Check timezone suffix
get_datetime:
input:
timezone:
ends_with!: "rid" # Matches "Madrid"
# Check file extension
extract_file:
output:
filename:
ends_with!: ".pdf"
# Case-insensitive with transform
output:
ends_with!:
value: "completed."
transform: lowercase
```
### not\_ends\_with!
Checks that the actual value does NOT end with the expected suffix. This is the negated form of `ends_with!`.
```yaml theme={"dark"}
output:
not_ends_with!: "error"
```
| Parameter | Type | Description |
| --------- | ------ | ------------------------------- |
| value | string | Suffix that must NOT be present |
```yaml theme={"dark"}
# Ensure no error suffix
output:
not_ends_with!: "failed"
# Case-insensitive with transform
output:
not_ends_with!:
value: "error"
transform: lowercase
```
## Numeric Comparisons
Numeric validators support integers, floats, and date strings.
### lt!
Checks if the actual value is less than the expected value.
```yaml theme={"dark"}
elapsed:
lt!: 5000
```
| Parameter | Type | Description |
| --------- | --------------------- | ----------------------- |
| value | number or date string | Upper bound (exclusive) |
### lte!
Checks if the actual value is less than or equal to the expected value.
```yaml theme={"dark"}
llm:
usage:
input_tokens:
lte!: 500
```
| Parameter | Type | Description |
| --------- | --------------------- | ----------------------- |
| value | number or date string | Upper bound (inclusive) |
### gt!
Checks if the actual value is greater than the expected value.
```yaml theme={"dark"}
get_datetime:
output:
gt!: "2025-12-15T00:45:12"
```
| Parameter | Type | Description |
| --------- | --------------------- | ----------------------- |
| value | number or date string | Lower bound (exclusive) |
### gte!
Checks if the actual value is greater than or equal to the expected value.
```yaml theme={"dark"}
search_results:
output:
count:
gte!: 1
```
| Parameter | Type | Description |
| --------- | --------------------- | ----------------------- |
| value | number or date string | Lower bound (inclusive) |
```yaml theme={"dark"}
# Execution time limits
elapsed:
lt!: 6000
# Token usage limits
llm:
usage:
input_tokens:
lte!: 500
output_tokens:
lte!: 1000
# Per-span timing
get_datetime:
elapsed:
lt!: 1000
gte!: 10
# Date comparison
get_datetime:
output:
gt!: "2025-12-15T00:45:12"
lte!: "2026-01-01"
```
## Combining Comparison Validators
Combine validators for comprehensive checks:
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 50
contains!: "order"
not_contains!: "error"
pattern!: "#\\d{6}"
ends_with!: "."
```
### Range Checks
```yaml theme={"dark"}
elapsed:
gte!: 100
lt!: 5000
llm:
usage:
input_tokens:
gt!: 0
lte!: 500
```
### Span Input/Output Validation
```yaml theme={"dark"}
get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
starts_with!: "Europe/"
ends_with!: "rid"
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
gt!: "2025-01-01"
lte!: "2026-01-01"
```
# Flow Validators
Source: https://docs.timbal.ai/evals/validators/flow
Validators for checking execution sequences and parallel execution
Flow validators check the execution patterns of your agent - the order in which tools are called and whether operations run in parallel.
## seq!
Validates that execution spans match a sequence pattern. This is a top-level validator that checks the order of tool calls.
```yaml theme={"dark"}
seq!:
- llm
- search_products
- llm
```
The `seq!` validator checks that the specified spans appear in order in the execution trace.
### Basic Sequence
```yaml theme={"dark"}
seq!:
- llm # First: LLM call
- get_datetime # Then: datetime tool
- llm # Then: another LLM call
```
### Wildcard Patterns
Use wildcards to match flexible sequences:
| Pattern | Description |
| ------- | ------------------------------- |
| `..` | Exactly 1 span |
| `...` | Any number of spans (0 or more) |
| `n..m` | Between n and m spans |
| `n..` | At least n spans |
| `..m` | At most m spans |
```yaml theme={"dark"}
seq!:
- llm
- ... # Any spans in between
- send_email
```
```yaml theme={"dark"}
# Exactly one span between
seq!:
- validate_input
- .. # Exactly 1 span
- process_result
# Any number of spans between
seq!:
- llm
- ... # 0 or more spans
- llm
# Between 1 and 3 spans
seq!:
- llm
- 1..3 # 1 to 3 spans
- complete
# At least 1 span between
seq!:
- initialize
- 1.. # 1 or more spans
- finalize
# At most 2 spans between
seq!:
- fetch_data
- ..2 # 0 to 2 spans
- transform_data
```
### Span Validation Within Sequence
Validate span inputs, outputs, and timing within the sequence:
```yaml theme={"dark"}
seq!:
- llm:
elapsed:
lte!: 40000
usage:
input_tokens:
lte!: 1000
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
starts_with!: "Europe/"
ends_with!: "rid"
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
gt!: "2025-12-15T00:45:12"
- llm
```
### any! Pattern
Match any span meeting certain criteria:
```yaml theme={"dark"}
seq!:
- llm
- any!:
min: 1 # At least 1 match
max: 3 # At most 3 matches
contains: # Must match one of these
- validate_input
- check_format
not_contains: # Must not match these
- delete_user
- process_result
```
| Field | Type | Description |
| -------------- | ------- | -------------------------------- |
| `min` | integer | Minimum number of matching spans |
| `max` | integer | Maximum number of matching spans |
| `contains` | list | Span names that can match |
| `not_contains` | list | Span names that must not match |
```yaml theme={"dark"}
# At least one validation step
seq!:
- receive_input
- any!:
min: 1
contains:
- validate_email
- validate_phone
- validate_address
- save_record
# Any number of retry attempts (0-3)
seq!:
- initial_request
- any!:
min: 0
max: 3
contains:
- retry_request
- final_response
# Exclude certain operations
seq!:
- process_data
- any!:
min: 1
not_contains:
- delete_data
- drop_table
- complete
```
### Nested parallel! in Sequences
Check for parallel execution within a sequence:
```yaml theme={"dark"}
seq!:
- llm
- parallel!:
- get_datetime
- get_weather
- get_stock_price
- llm
```
With span validation inside parallel:
```yaml theme={"dark"}
seq!:
- 1..1
- parallel!:
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
- get_weather
- ...
```
## parallel!
Validates that specified spans executed in parallel (overlapping time ranges). This can be used as a top-level validator or nested within `seq!`.
```yaml theme={"dark"}
parallel!:
- get_datetime
- get_weather
- get_stock_price
```
| Field | Type | Description |
| ----------- | ------- | --------------------------------------------- |
| (list) | list | Span names that must execute in parallel |
| `tolerance` | integer | Timing tolerance in milliseconds (default: 0) |
| `spans` | list | Span names (when using tolerance) |
### Basic Parallel Check
```yaml theme={"dark"}
parallel!:
- fetch_user
- fetch_orders
- fetch_preferences
```
This validates that all three spans had overlapping execution times.
### With Tolerance
Network latency and scheduling can cause slight timing variations. Use tolerance to account for this:
```yaml theme={"dark"}
parallel!:
tolerance: 100 # 100ms tolerance
spans:
- fetch_user
- fetch_orders
- fetch_preferences
```
### With Span Validation
Validate inputs/outputs of parallel spans:
```yaml theme={"dark"}
parallel!:
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
output:
type!: "string"
- get_weather:
input:
city:
eq!: "Madrid"
- get_stock_price
```
## Complete Examples
### Basic Flow Validation
```yaml theme={"dark"}
- name: datetime_query
runnable: agent.py::agent
params:
prompt: "what time is it in madrid"
output:
contains!: ":"
pattern!: "\\d{1,2}:\\d{2}"
seq!:
- llm
- get_datetime
- llm
```
### Comprehensive Flow Validation
```yaml theme={"dark"}
- name: full_datetime_flow
runnable: agent.py::agent
params:
prompt: "what time is it in madrid"
output:
type!: "string"
min_length!: 10
contains!: ":"
pattern!: "\\d{1,2}:\\d{2}"
prompt!: "The response states the current time in Madrid"
elapsed:
lt!: 6000
seq!:
- llm:
elapsed:
lte!: 40000
usage:
input_tokens:
lte!: 1000
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
starts_with!: "Europe/"
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
- llm
```
### Parallel Tool Execution
```yaml theme={"dark"}
- name: parallel_data_fetch
runnable: agent.py::agent
params:
prompt: "Get my dashboard data"
parallel!:
- fetch_user
- fetch_orders
- fetch_recommendations
```
### Mixed Sequential and Parallel
```yaml theme={"dark"}
- name: complex_flow
runnable: agent.py::agent
params:
prompt: "Process my order"
seq!:
- llm
- validate_order
- parallel!:
- check_inventory
- calculate_shipping
- apply_discounts
- ...
- llm
```
## How Parallel Detection Works
Two spans are considered parallel if their execution times overlap:
```
Span A: |-------|
Span B: |-------|
→ Parallel (overlapping)
Span A: |-------|
Span B: |-------|
→ Sequential (no overlap)
```
With tolerance, spans starting within the tolerance window are still considered parallel:
```
Tolerance: 100ms
Span A: |-------|
Span B: |-------| (started 50ms after A ended)
→ Parallel (within tolerance)
```
## Tips
Use `seq!` with wildcards (`...`) for flexible matching when you care about the order of key operations but not every intermediate step.
Flow validators require execution tracing to be enabled. Tracing is automatically enabled when running evals.
Be careful with strict timing requirements in `parallel!`. Network conditions and system load can affect timing. Use appropriate tolerance values.
# Overview
Source: https://docs.timbal.ai/evals/validators/index
Complete reference for all Timbal eval validators
Validators are the core of Timbal's eval system. They check specific properties of your agent's execution and determine whether an eval passes or fails.
## Validator Syntax
All validators use the `name!` suffix convention:
```yaml theme={"dark"}
output:
type!: "string"
contains!: "hello"
min_length!: 10
```
The `!` suffix distinguishes validators from regular YAML keys.
## Transforms
Transforms allow you to normalize values before validation. This is useful for case-insensitive matching or handling whitespace.
```yaml theme={"dark"}
output:
contains!:
value: "success"
transform: lowercase
```
### Available Transforms
| Transform | Description | Example |
| --------------------- | ----------------------------------- | ------------------- |
| `lowercase` | Convert to lowercase | `"Hello" → "hello"` |
| `uppercase` | Convert to uppercase | `"Hello" → "HELLO"` |
| `trim` | Remove leading/trailing whitespace | `" hi " → "hi"` |
| `collapse_whitespace` | Replace multiple spaces with single | `"a b" → "a b"` |
### Chaining Transforms
Apply multiple transforms in order:
```yaml theme={"dark"}
output:
contains!:
value: "hello world"
transform: [trim, lowercase, collapse_whitespace]
```
Transforms are applied left-to-right: first `trim`, then `lowercase`, then `collapse_whitespace`.
## Negation
Most validators support negation using the `not_` prefix or the `negate` field.
### Using Aliases
```yaml theme={"dark"}
output:
not_contains!: "error"
not_starts_with!: "Error:"
ne!: "failed"
```
### Using the negate Field
```yaml theme={"dark"}
output:
contains!:
value: "error"
negate: true
```
### Combining Negation with Transforms
```yaml theme={"dark"}
output:
not_contains!:
value: "ERROR"
transform: uppercase
```
## Validator Categories
String matching, patterns, and equality checks
Type checking, JSON, and format validation
Length constraints and bounds
AI-powered checks for claims, meaning, and language
Execution sequence and parallelism
## Quick Reference
| Validator | Category | Description | Example |
| ------------------- | ---------- | --------------------------------- | ----------------------------------------------------- |
| `eq!` | Comparison | Exact equality | `eq!: "hello"` |
| `ne!` | Comparison | Not equal | `ne!: "error"` |
| `contains!` | Comparison | Substring/item check | `contains!: "world"` |
| `not_contains!` | Comparison | Absence check | `not_contains!: "error"` |
| `contains_all!` | Comparison | Contains all items | `contains_all!: ["a", "b"]` |
| `not_contains_all!` | Comparison | Missing at least one | `not_contains_all!: ["x", "y"]` |
| `contains_any!` | Comparison | Contains at least one | `contains_any!: ["a", "b"]` |
| `not_contains_any!` | Comparison | Contains none | `not_contains_any!: ["x", "y"]` |
| `pattern!` | Comparison | Regex match | `pattern!: "\\d+"` |
| `not_pattern!` | Comparison | Regex non-match | `not_pattern!: "^Error"` |
| `starts_with!` | Comparison | Prefix check | `starts_with!: "Hello"` |
| `not_starts_with!` | Comparison | No prefix match | `not_starts_with!: "Error"` |
| `ends_with!` | Comparison | Suffix check | `ends_with!: "."` |
| `not_ends_with!` | Comparison | No suffix match | `not_ends_with!: "error"` |
| `lt!` | Comparison | Less than | `lt!: 100` |
| `lte!` | Comparison | Less than or equal | `lte!: 100` |
| `gt!` | Comparison | Greater than | `gt!: 0` |
| `gte!` | Comparison | Greater than or equal | `gte!: 1` |
| `type!` | Type | Type checking | `type!: "string"` |
| `not_type!` | Type | Type exclusion | `not_type!: "null"` |
| `json!` | Type | Valid JSON | `json!: true` |
| `email!` | Type | Email format | `email!: true` |
| `not_null!` | Type | Non-null check | `not_null!: true` |
| `length!` | Length | Exact length | `length!: 10` |
| `min_length!` | Length | Minimum length | `min_length!: 5` |
| `max_length!` | Length | Maximum length | `max_length!: 100` |
| `prompt!` | LLM | Statement/claim is true in output | `prompt!: "The response explains the refund process"` |
| `semantic!` | LLM | Semantic match | `semantic!: "greeting"` |
| `not_semantic!` | LLM | Semantic non-match | `not_semantic!: "rude"` |
| `language!` | LLM | Language detection | `language!: "en"` |
| `not_language!` | LLM | Language exclusion | `not_language!: "fr"` |
| `seq!` | Flow | Execution sequence | See [Flow](/evals/validators/flow) |
| `parallel!` | Flow | Parallel execution | See [Flow](/evals/validators/flow) |
## Where Validators Apply
### Output Validation
```yaml theme={"dark"}
output:
type!: "string"
not_null!: true
contains!: "success"
```
### Timing Validation
```yaml theme={"dark"}
elapsed:
lt!: 5000
gte!: 100
```
### Span Validation
```yaml theme={"dark"}
get_weather:
input:
city:
eq!: "Madrid"
output:
type!: "object"
elapsed:
lt!: 2000
```
### Usage Validation
```yaml theme={"dark"}
llm:
usage:
input_tokens:
lte!: 500
output_tokens:
lte!: 1000
```
The token field names depend on the model provider:
* **OpenAI**: Use `input_text_tokens` and `output_text_tokens` (e.g., `input_text_tokens: lte!: 500`, `output_text_tokens: lte!: 1000`)
* **Anthropic**: Use `input_tokens` and `output_tokens` (e.g., `input_tokens: lte!: 500`, `output_tokens: lte!: 1000`)
## Combining Validators
### Multiple Validators on One Target
Apply multiple validators to the same target:
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 10
contains!: "success"
not_contains!: "error"
```
All validators must pass for the eval to succeed.
### Nested Validation
Validate nested properties in span inputs/outputs:
```yaml theme={"dark"}
search_products:
input:
query:
contains!: "laptop"
options:
limit:
lte!: 100
category:
eq!: "electronics"
output:
results:
min_length!: 1
```
## Error Messages
When a validator fails, it provides detailed error information:
```
FAILED: output
Validator: contains!
Expected: "success"
Actual: "The operation failed due to network error"
Error: Value does not contain expected substring
```
For LLM validators:
```
FAILED: output
Validator: prompt!
Expected: "The response greets the user politely"
Actual: "What do you want?"
Reason: The response is curt and lacks politeness
```
# Length Validators
Source: https://docs.timbal.ai/evals/validators/length
Validators for checking length constraints on strings, lists, and collections
Length validators check the size of strings, lists, and other collections.
Length validators support [transforms](/evals/validators#transforms) when validating strings. Transforms are applied before measuring length.
## length!
Checks that the value has exactly the specified length.
```yaml theme={"dark"}
generate_code:
output:
length!: 6
```
| Parameter | Type | Description |
| --------- | ------- | --------------------- |
| value | integer | Exact required length |
```yaml theme={"dark"}
# Exact code length
generate_pin:
output:
length!: 4
# Fixed-length ID
create_id:
output:
length!: 36 # UUID length
# Specific list size
get_top_results:
output:
length!: 5
```
## min\_length!
Checks that the value has at least the specified length.
```yaml theme={"dark"}
output:
min_length!: 10
```
| Parameter | Type | Description |
| --------- | ------- | ----------------------- |
| value | integer | Minimum required length |
```yaml theme={"dark"}
# Ensure substantive response
output:
min_length!: 50
# Require at least one result
search_products:
output:
min_length!: 1
# Validate input length
validate_password:
input:
password:
min_length!: 8
```
## max\_length!
Checks that the value has at most the specified length.
```yaml theme={"dark"}
output:
max_length!: 1000
```
| Parameter | Type | Description |
| --------- | ------- | ---------------------- |
| value | integer | Maximum allowed length |
```yaml theme={"dark"}
# Limit response length
output:
max_length!: 500
# Limit results returned
search_products:
output:
max_length!: 10
# Username length limit
validate_username:
input:
username:
max_length!: 32
```
## Combining Length Validators
### Length Range
```yaml theme={"dark"}
output:
min_length!: 50
max_length!: 500
```
### With Other Validators
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 10
max_length!: 1000
contains!: "result"
```
## Common Patterns
### Response Quality Checks
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 20 # Not too short
max_length!: 2000 # Not too long
```
### Array Size Validation
```yaml theme={"dark"}
search_products:
output:
type!: "array"
min_length!: 1
max_length!: 50
recommend:
output:
length!: 3 # Exactly 3 recommendations
```
### Code Generation
```yaml theme={"dark"}
generate_pin:
output:
type!: "string"
length!: 4
pattern!: "\\d{4}"
generate_code:
output:
type!: "string"
length!: 6
pattern!: "[A-Z0-9]{6}"
```
## Using Transforms
Transforms are applied before measuring length:
```yaml theme={"dark"}
# Trim whitespace before checking length
output:
min_length!:
value: 10
transform: trim
# Collapse whitespace then check length
output:
max_length!:
value: 100
transform: [trim, collapse_whitespace]
```
## Notes on Length Calculation
* **Strings**: Length is the number of characters
* **Arrays**: Length is the number of items
* **Objects**: Length is the number of keys
```yaml theme={"dark"}
# String length (characters)
output: # "Hello" has length 5
length!: 5
# Array length (items)
items: # [1, 2, 3] has length 3
length!: 3
# Object length (keys)
config: # {"a": 1, "b": 2} has length 2
length!: 2
```
# LLM Validators
Source: https://docs.timbal.ai/evals/validators/llm
AI-powered validators for checking claims, meaning, and language
LLM validators use AI models to evaluate content that can't be easily checked with exact matching. They're ideal for validating natural language outputs where wording may vary but meaning should be consistent.
LLM validators support [transforms](/evals/validators#transforms). Transforms are applied to the content before sending to the LLM for evaluation.
## prompt!
Validates whether a natural-language statement about the output is true based on the actual text.
```yaml theme={"dark"}
output:
prompt!: "The response clearly explains the refund process"
```
| Parameter | Type | Description |
| --------- | ------ | ----------------------------------------------------------- |
| value | string | A statement or claim that should be true in the actual text |
The prompt validator asks an LLM to check if your statement is supported by the actual output. It focuses on factual consistency rather than exact wording.
`prompt!` is the recommended general-purpose LLM validator. A statement has clear truth conditions ("The agent asked about the budget" is either supported by the text or it isn't), which makes judge results more consistent than fuzzy "does it match" comparisons.
```yaml theme={"dark"}
# Check that a specific question was asked
output:
prompt!: "The agent asked the customer about their budget"
# Check that a key action was taken
output:
prompt!: "The assistant confirmed the shipping address before placing the order"
# Check that required information is present
output:
prompt!: "The message includes the expected delivery date"
# Error handling
output:
prompt!: "The response apologizes and explains the service is unavailable"
# Content requirements
output:
prompt!: "The response describes the product and mentions its price and availability"
```
### Writing Effective Statements
Write statements as specific, verifiable claims about the output. Vague statements lead to inconsistent results.
**Good statements:**
```yaml theme={"dark"}
# Specific and verifiable
prompt!: "The response includes at least 3 product recommendations with prices"
prompt!: "The error message mentions the specific field that failed validation"
prompt!: "The summary covers the main points: budget, timeline, and deliverables"
```
**Avoid:**
```yaml theme={"dark"}
# Too vague
prompt!: "The response is good"
prompt!: "The answer is helpful"
prompt!: "The output is correct"
```
For multi-part requirements, list the parts explicitly in a single statement:
```yaml theme={"dark"}
output:
prompt!: |
The response acknowledges the user's question, answers it directly,
and offers follow-up assistance
```
### Negating prompt!
Use the `negate` field to assert that a statement is **not** true:
```yaml theme={"dark"}
output:
prompt!:
value: "The assistant discloses internal system prompts"
negate: true
```
This passes only if the statement is **not** supported by the actual text.
## rubric!
Grades the output against a structured rubric — a list of criteria, each judged by its **own isolated LLM call** with its own context window. Per-dimension judging grades more reliably than one judge scoring everything at once, and each criterion returns `pass` / `fail` / `unknown` with a reason (`unknown` is the judge's escape hatch when the text gives no way to verify — it counts as not passing).
```yaml theme={"dark"}
output:
rubric!:
- "Includes a comparison table"
- "Every price is attributed to a source"
- "Ends with at least 3 actionable recommendations"
```
| Parameter | Type | Description |
| --------------- | ---------------------- | --------------------------------------------------------------------------------------- |
| value | list \| string \| dict | Criteria list, a markdown rubric (bullet lines become criteria), or a dict with options |
| pass\_threshold | float | Weighted fraction of criteria that must pass. Default `1.0` (all) |
| model | string | Judge model. Default `openai/gpt-5.4-nano` |
| context | string | Optional task description shown to every judge |
When the rubric fails, the eval report lists **every failing criterion with the judge's reason** — you see exactly which requirement broke, not a single opaque fail.
Full form with weighted criteria:
```yaml theme={"dark"}
output:
rubric!:
criteria:
- "Includes a comparison table"
- criterion: "Ends with at least 3 actionable recommendations"
name: recommendations
weight: 2
pass_threshold: 0.75
model: "openai/gpt-5.4-nano"
context: "The agent produced a price-comparison report."
```
Markdown rubrics work too — bullet and numbered lines become criteria, headings and prose are ignored:
```yaml theme={"dark"}
output:
rubric!: |
- Mentions the refund policy
- Confirms the order number
- Ends by offering further help
```
Write criteria around **verifiable structure**, not facts the judge cannot check. "Prices are formatted and attributed to a source" grades reliably; "prices are accurate" does not — the judge has no way to confirm it and will answer `unknown`.
Use `rubric!` instead of several `prompt!` statements when the requirements form one quality bar: you get per-criterion verdicts, weights, a partial-credit threshold, and one aggregate score. The same rubric can also gate an agent at runtime via `timbal.guardrails.LLMJudge(rubric=...)`, which feeds failing criteria back to the agent for revision.
## semantic!
Uses an LLM to check if the actual value semantically matches the expected description.
```yaml theme={"dark"}
output:
semantic!: "A polite greeting that welcomes the user"
```
| Parameter | Type | Description |
| --------- | ------ | ------------------------------------------------ |
| value | string | Natural language description of expected content |
The semantic validator sends the actual value and your description to an LLM, which determines if they match semantically.
If your check can be phrased as a verifiable statement, prefer [`prompt!`](#prompt). Use `semantic!` when you're matching the output against a *description* of its overall meaning, tone, or style rather than asserting a specific fact about it.
```yaml theme={"dark"}
# Tone
output:
semantic!: "A professional response suitable for a business context"
# Style matching
output:
semantic!: "A casual, friendly greeting that matches the user's informal tone"
# Overall content shape
output:
semantic!: "An apologetic message explaining the service is unavailable"
```
## not\_semantic!
Checks that the content does NOT semantically match the description. This is the negated form of `semantic!`.
```yaml theme={"dark"}
output:
not_semantic!: "An error message or apology"
```
| Parameter | Type | Description |
| --------- | ------ | ------------------------------------------------ |
| value | string | Natural language description that must NOT match |
```yaml theme={"dark"}
# Ensure response is not an error
output:
not_semantic!: "An error message or failure notification"
# Ensure not rude or dismissive
output:
not_semantic!: "A rude, dismissive, or unhelpful response"
# Ensure not off-topic
output:
not_semantic!: "A response about unrelated topics"
```
## language!
Checks that the content is written in a specific language.
```yaml theme={"dark"}
output:
language!: "en"
```
| Parameter | Type | Description |
| --------- | ------ | ---------------------- |
| value | string | Expected language code |
```yaml theme={"dark"}
# Verify language output
output:
language!: "en"
output:
language!: "es"
output:
language!: "fr"
output:
language!: "ja"
```
### Common Language Codes
| Language | Code |
| ---------- | ------ |
| English | `"en"` |
| Spanish | `"es"` |
| French | `"fr"` |
| German | `"de"` |
| Italian | `"it"` |
| Portuguese | `"pt"` |
| Chinese | `"zh"` |
| Japanese | `"ja"` |
| Korean | `"ko"` |
| Arabic | `"ar"` |
## not\_language!
Checks that the content is NOT written in a specific language. This is the negated form of `language!`.
```yaml theme={"dark"}
output:
not_language!: "fr"
```
| Parameter | Type | Description |
| --------- | ------ | --------------------------------- |
| value | string | Language code that must NOT match |
```yaml theme={"dark"}
# Ensure response is not in French
output:
not_language!: "fr"
# Ensure English-only output
output:
language!: "en"
not_language!: "es"
```
## LLM vs Exact Matching
Use LLM validation (`prompt!`, `semantic!`) when:
* Output wording can vary but meaning must be consistent
* Testing for tone, style, or completeness
* Validating summaries or explanations
Use exact matching (`eq!`, `contains!`) when:
* Specific words or phrases must appear
* Validating structured data
* Checking for exact values
```yaml theme={"dark"}
# Use prompt for flexible content
output:
prompt!: "The response confirms the order was placed successfully"
# Use contains for required terms
output:
contains!: "Order #"
# Combine both approaches
output:
contains!: "confirmed"
prompt!: "The response confirms the order and includes the order details"
```
## Common Patterns
### Multi-Language Support Testing
```yaml theme={"dark"}
- name: responds_in_spanish
runnable: agent.py::agent
params:
prompt: "Hola, necesito ayuda"
output:
language!: "es"
prompt!: "The response offers help to the user"
- name: responds_in_french
runnable: agent.py::agent
params:
prompt: "Bonjour, j'ai besoin d'aide"
output:
language!: "fr"
prompt!: "The response offers help to the user"
```
### Behavioral Checks
```yaml theme={"dark"}
- name: refund_process_explained
runnable: agent.py::agent
params:
prompt: "I want a refund"
output:
prompt!: "The response acknowledges the refund request and explains the refund process"
- name: address_confirmed_before_order
runnable: agent.py::agent
params:
prompt: "Order this to my usual address"
output:
prompt!: "The assistant confirmed the shipping address before placing the order"
```
### Tone and Style Validation
```yaml theme={"dark"}
- name: professional_tone
runnable: agent.py::agent
params:
prompt: "I want a refund"
output:
semantic!: "A professional, empathetic response"
- name: casual_tone
runnable: agent.py::agent
params:
prompt: "Hey what's up"
output:
semantic!: "A casual, friendly greeting that matches the user's informal tone"
```
### Completeness Checks
```yaml theme={"dark"}
output:
prompt!: |
The response acknowledges the user's question, answers it directly,
provides additional context, and offers follow-up assistance
```
### Error Message Quality
```yaml theme={"dark"}
- name: helpful_error_message
runnable: agent.py::agent
params:
prompt: "Buy product XYZ123"
output:
prompt!: |
The response clearly states the product was not found, suggests
possible alternatives or corrections, and offers help finding
the right product
```
## Using Transforms
Transforms normalize content before LLM evaluation:
```yaml theme={"dark"}
# Normalize whitespace before the LLM check
output:
prompt!:
value: "The response greets the user professionally"
transform: [trim, collapse_whitespace]
# Lowercase before language detection
output:
language!:
value: "en"
transform: lowercase
```
## Combining with Other Validators
```yaml theme={"dark"}
output:
# Structure checks
not_null!: true
type!: "string"
min_length!: 50
# Content checks
contains!: "order"
not_contains!: "error"
# LLM validation
prompt!: "The response confirms the order and includes an estimated delivery date"
# Language check
language!: "en"
```
## Choosing Models
LLM validators use Timbal agents under the hood. By default they use `openai/gpt-5.4-nano`, but you can override the model per-validator in YAML.
```yaml theme={"dark"}
output:
# Use a larger OpenAI model for factual checks
prompt!:
value: "The assistant clearly explains the refund process"
model: "openai/gpt-5.2"
# Use an Anthropic model for language + semantics
language!:
value: "es"
model: "anthropic/claude-sonnet-4-6"
# Use a Gemini model for description matching
semantic!:
value: "A helpful, on-topic answer"
model: "google/gemini-2.0-flash"
```
LLM validators rely on **structured output**. For Anthropic, structured-output support is currently in beta and only available on a limited set of models (for example: Claude Sonnet 4.5/4.6, Claude Opus 4.5/4.6, Claude Haiku 4.5). Make sure you pick one of these Anthropic variants when you set the `model` field.
You can otherwise pick any model supported by Timbal’s `Agent` – the `model` field here is passed directly through to the underlying agent used for the validator, subject to the provider’s structured-output limitations.
## Cost Considerations
LLM validators make API calls to language models, which incur costs. To optimize:
1. **Use structural validators first**: Check `not_null!`, `contains!`, etc. before LLM validation
2. **Be specific in statements**: Reduces need for retries
3. **Group LLM checks**: One detailed statement vs. multiple simple ones
4. **Use for critical paths**: Reserve LLM validation for important behavioral checks
```yaml theme={"dark"}
# Efficient: structural checks catch obvious failures quickly
output:
not_null!: true # Fast, free
type!: "string" # Fast, free
min_length!: 20 # Fast, free
contains!: "order" # Fast, free
prompt!: "The response confirms the order completely" # LLM call only if above pass
```
# Type Validators
Source: https://docs.timbal.ai/evals/validators/type
Validators for type checking, JSON, email, and null validation
Type validators check the type and format of values, ensuring data conforms to expected structures.
Type validators generally don't use transforms since they check data types rather than string content. See [Validators Overview](/evals/validators#transforms) for transform documentation.
## not\_null!
Checks that a value exists and is not null/None.
```yaml theme={"dark"}
output:
not_null!: true
```
| Parameter | Type | Description |
| --------- | ------- | --------------------------------------------------- |
| value | boolean | `true` to require non-null, `false` to require null |
```yaml theme={"dark"}
# Ensure output exists
output:
not_null!: true
# Ensure tool output exists
get_weather:
output:
not_null!: true
# Ensure specific field exists
search_results:
output:
items:
not_null!: true
```
Use `not_null!: true` as a first validator to ensure the target exists before applying other validators.
## type!
Checks that a value is of a specific type.
```yaml theme={"dark"}
output:
type!: "string"
```
| Parameter | Type | Description |
| --------- | ------ | ----------- |
| value | string | Type name |
### Supported Types
| Type | Matches |
| -------- | ---------------------- |
| `string` | Strings |
| `int` | Integers |
| `float` | Floating-point numbers |
| `number` | Any numeric type |
| `bool` | Boolean values |
| `array` | Lists/arrays |
| `object` | Dictionaries/objects |
| `null` | None/null values |
```yaml theme={"dark"}
# String output
output:
type!: "string"
# Numeric result
calculate:
output:
type!: "number"
# Array of results
search_products:
output:
type!: "array"
# Object response
get_user:
output:
type!: "object"
# Boolean flag
validate:
output:
type!: "bool"
```
## not\_type!
Checks that a value is NOT of a specific type. This is the negated form of `type!`.
```yaml theme={"dark"}
output:
not_type!: "null"
```
| Parameter | Type | Description |
| --------- | ------ | ----------------------------- |
| value | string | Type name that must NOT match |
```yaml theme={"dark"}
# Ensure output is not null
output:
not_type!: "null"
# Ensure result is not a string (expect structured data)
process_data:
output:
not_type!: "string"
# Ensure not returning an error object
validate:
output:
not_type!: "object"
```
## json!
Checks that a string value is valid JSON.
```yaml theme={"dark"}
output:
json!: true
```
| Parameter | Type | Description |
| --------- | ------- | ---------------------------- |
| value | boolean | `true` to require valid JSON |
```yaml theme={"dark"}
# Ensure output is valid JSON
output:
json!: true
# Combine with content checks
output:
json!: true
contains!: '"status"'
contains!: '"success"'
```
The `json!` validator only checks that the string is valid JSON. It does not validate the JSON structure. Use `contains!` or `pattern!` for content checks.
## email!
Checks that a string value is a valid email format.
```yaml theme={"dark"}
extract_email:
output:
email!: true
```
| Parameter | Type | Description |
| --------- | ------- | ------------------------------------ |
| value | boolean | `true` to require valid email format |
```yaml theme={"dark"}
# Validate extracted email
extract_contact:
output:
email:
email!: true
```
## Common Patterns
### Structured Output Validation
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
json!: true
contains!: '"id"'
contains!: '"status"'
```
### Type-Safe Numeric Results
```yaml theme={"dark"}
calculate:
output:
type!: "number"
gte!: 0
lte!: 100
```
### Span Output Type Checking
```yaml theme={"dark"}
get_datetime:
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
search_products:
output:
type!: "array"
min_length!: 1
```
### Validating Nested Fields
```yaml theme={"dark"}
get_user:
output:
type!: "object"
# Or validate specific nested fields
get_user:
output:
profile:
name:
type!: "string"
not_null!: true
email:
email!: true
```
# Writing Evals
Source: https://docs.timbal.ai/evals/writing-evals
Learn how to write effective evals for your Timbal agents
This guide covers the complete syntax for writing evals, from basic tests to complex multi-validator scenarios.
## File Structure
Evals are defined in YAML files. Each file can contain multiple eval definitions:
```yaml theme={"dark"}
# eval_search.yaml
- name: basic_search
description: Test basic product search
runnable: agents/search.py::search_agent
params:
prompt: "Find me a laptop"
output:
contains!: "laptop"
type!: "string"
- name: search_with_filters
description: Test search with price filters
runnable: agents/search.py::search_agent
params:
prompt: "Find laptops under $1000"
output:
pattern!: "\\$\\d+"
```
## Eval Definition
### Required Fields
```yaml theme={"dark"}
- name: my_eval_name
runnable: path/to/agent.py::agent_name
```
Eval names must be unique across all eval files in your project. The CLI will error if duplicate names are found.
### Optional Fields
```yaml theme={"dark"}
- name: complete_example
description: "A thorough test of greeting behavior"
runnable: agent.py::agent
tags:
- greeting
- smoke-test
timeout: 30000 # Milliseconds
env:
API_KEY: "test-key"
params:
prompt: "Hello"
output:
prompt!: "The response greets the user in a friendly way"
elapsed:
lt!: 5000
```
## Params Structure
The `params` field contains input parameters passed to your runnable:
### Simple Prompt
```yaml theme={"dark"}
params:
prompt: "What's the weather like?"
```
### With Messages
Use `messages` to establish conversation history for multi-turn testing:
```yaml theme={"dark"}
params:
messages:
- role: user
content: "What's the weather like in New York?"
- role: assistant
content: "It's currently 15°C and raining in New York."
- role: user
content: "Should I bring an umbrella?"
```
The agent receives the full conversation history and responds to the last message. This is useful for testing context retention, memory, and whether the agent avoids redundant tool calls when context is already available.
When using `messages` instead of `prompt`, the agent's input key will be `messages` rather than `prompt`.
### Additional Parameters
Pass any additional parameters your agent accepts:
```yaml theme={"dark"}
params:
prompt: "Search for products"
max_results: 10
include_reviews: true
category: "electronics"
```
## Validating Output
The `output` section validates the final response from your agent:
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 10
contains!: "success"
not_contains!: "error"
pattern!: "Order #\\d{6}"
prompt!: "The response confirms the order and includes the order details"
```
Multiple validators can be combined - all must pass.
## Validating Timing
The `elapsed` section validates total execution time in milliseconds:
```yaml theme={"dark"}
elapsed:
lt!: 5000 # Less than 5 seconds
gte!: 100 # At least 100ms (not instant)
```
## Validating Tool Spans
Validate specific tools by their name:
```yaml theme={"dark"}
get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
starts_with!: "Europe/"
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
elapsed:
lt!: 1000
search_products:
input:
query:
contains!: "laptop"
limit:
lte!: 100
output:
type!: "array"
min_length!: 1
```
## Validating Token Usage
Access usage metrics on LLM spans:
```yaml theme={"dark"}
llm:
usage:
input_tokens:
lte!: 500
output_tokens:
lte!: 1000
```
The token field names depend on the model provider:
* **OpenAI**: Use `input_text_tokens` and `output_text_tokens` (e.g., `input_text_tokens: lte!: 500`, `output_text_tokens: lte!: 1000`)
* **Anthropic**: Use `input_tokens` and `output_tokens` (e.g., `input_tokens: lte!: 500`, `output_tokens: lte!: 1000`)
For multi-model scenarios, tokens are automatically summed across models.
## Flow Validators
### Sequence Validation
Use `seq!` to validate the order of tool execution:
```yaml theme={"dark"}
seq!:
- llm
- search_products
- llm
```
With wildcards for flexible matching:
```yaml theme={"dark"}
seq!:
- llm
- ... # Any number of spans
- send_email
```
### Parallel Validation
Use `parallel!` to validate concurrent execution:
```yaml theme={"dark"}
parallel!:
- get_weather
- get_datetime
- get_stock_price
```
### Nested Flow Validation
Combine sequence and parallel:
```yaml theme={"dark"}
seq!:
- llm
- parallel!:
- fetch_user
- fetch_orders
- llm
```
### Span Validation Within Sequence
Validate span inputs/outputs within the sequence:
```yaml theme={"dark"}
seq!:
- llm:
elapsed:
lt!: 3000
usage:
input_tokens:
lte!: 500
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
output:
type!: "string"
- llm
```
## Complete Example
```yaml theme={"dark"}
- name: time_query
description: "Test time query with validation"
runnable: "agent.py::agent"
tags: ["datetime", "smoke"]
timeout: 30000
params:
prompt: "what time is it in madrid"
# Validate final output
output:
type!: "string"
min_length!: 10
contains!: ":"
pattern!: "\\d{1,2}:\\d{2}"
prompt!: "The response states the current time in Madrid"
# Validate timing
elapsed:
lt!: 6000
# Validate execution sequence
seq!:
- llm:
elapsed:
lte!: 40000
usage:
input_tokens:
lte!: 1000
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
starts_with!: "Europe/"
output:
type!: "string"
pattern!: "^\\d{4}-\\d{2}-\\d{2}"
- llm
```
## Wildcard Patterns in Sequences
| Pattern | Description |
| ------- | ------------------------------- |
| `..` | Exactly 1 span |
| `...` | Any number of spans (0 or more) |
| `n..m` | Between n and m spans |
| `n..` | At least n spans |
| `..m` | At most m spans |
```yaml theme={"dark"}
seq!:
- llm
- 1..3 # 1 to 3 spans
- validate_input
- ... # Any number
- send_response
```
## Tags and Filtering
Use tags to organize and filter evals:
```yaml theme={"dark"}
- name: smoke_test_greeting
tags:
- smoke
- greeting
- quick
runnable: agent.py::agent
params:
prompt: "Hi"
output:
not_null!: true
```
## Environment Variables
Set environment variables for specific evals:
```yaml theme={"dark"}
- name: test_with_api_key
runnable: agent.py::agent
env:
API_KEY: "test-key-123"
DEBUG: "true"
params:
prompt: "Fetch data"
output:
not_null!: true
```
## Best Practices
Names should clearly indicate what's being tested:
```yaml theme={"dark"}
# Good
- name: search_returns_relevant_products
- name: handles_empty_search_results
- name: validates_timezone_input
# Avoid
- name: test1
- name: search_test
```
Use multiple validators to thoroughly test behavior:
```yaml theme={"dark"}
output:
not_null!: true
type!: "string"
min_length!: 50
contains!: "product"
not_contains!: "error"
prompt!: "The response recommends products and includes their prices"
```
Don't just check the output - validate how the agent got there:
```yaml theme={"dark"}
output:
prompt!: "The response provides weather information for Madrid"
seq!:
- llm
- get_weather:
input:
city:
eq!: "Madrid"
- llm
```
For outputs that can vary in wording but should convey the same meaning:
```yaml theme={"dark"}
# Instead of exact matching
output:
eq!: "The current time in Madrid is 14:30."
# Use an LLM-checked statement
output:
prompt!: "The response indicates the current time in Madrid"
```
Prefer `prompt!` (a verifiable statement about the output) for behavioral checks. Use `semantic!` when matching against a description of tone or style, e.g. `semantic!: "A professional, empathetic response"`.
Create specific evals for error conditions:
```yaml theme={"dark"}
- name: handles_invalid_timezone
runnable: agent.py::agent
params:
prompt: "what time is it in xyzland"
output:
prompt!: "The response indicates the timezone or location is not recognized"
```
## File Naming Conventions
Timbal discovers eval files matching these patterns:
* `eval*.yaml` - e.g., `eval_search.yaml`, `evals.yaml`
* `*eval.yaml` - e.g., `search_eval.yaml`, `my_eval.yaml`
Organize evals by feature or agent:
```
evals/
├── eval_search.yaml
├── eval_support.yaml
├── eval_checkout.yaml
└── regression/
├── eval_smoke.yaml
└── eval_full.yaml
```
# Adaptive System Prompts
Source: https://docs.timbal.ai/examples/agents/adaptive-system-prompts
Adapt agent behavior based on detected context or origin
Combine `pre_hook` with a dynamic system prompt function to adapt agent behavior based on detected context.
This example shows how to detect the origin platform and adjust the system prompt accordingly:
```python theme={"dark"}
from timbal import Agent
from timbal.state import get_run_context
def get_system_prompt() -> str:
client_chain = get_run_context().current_span().client_chain
prompts = {
"whatsapp": "You are a WhatsApp business assistant. Keep responses concise and friendly. Use emojis appropriately.",
"slack": "You are a Slack bot for internal team communication. Be professional but approachable. Use Slack formatting.",
}
return prompts.get(client_chain, "You are a helpful assistant.")
def pre_hook():
span = get_run_context().current_span()
prompt_text = span.input.get("prompt")
if "whatsapp" in prompt_text:
span.client_chain = "whatsapp"
elif "slack" in prompt_text:
span.client_chain = "slack"
else:
span.client_chain = "unknown"
agent = Agent(
name="ContextAwareAgent",
model="openai/gpt-4o-mini",
system_prompt=get_system_prompt,
pre_hook=pre_hook
)
# Different origins get different system prompts
result1 = await agent(prompt="whatsapp: Hi! I need help").collect()
result2 = await agent(prompt="slack: @bot Can you help?").collect()
result3 = await agent(prompt="Hi! I need help").collect()
```
## Key Features
* **Pre-hook Detection**: Use `pre_hook` to detect context or origin from message content
* **Dynamic System Prompts**: System prompt adapts based on detected context
* **Platform-Specific Responses**: Different tones and formats per platform
* **Combined Pattern**: Shows how to combine hooks with dynamic system prompts
# Approval-Required Tools
Source: https://docs.timbal.ai/examples/agents/approval-required-tools
Pause an agent for human approval, capture the gate, and resume with a decision — including redaction and audit fields
This example wires up a refund tool that gates above \$100, captures the `ApprovalEvent` mid-stream, and resumes with a decision. See the full reference in the [Human in the Loop](/human-in-the-loop) section.
```python theme={"dark"}
import asyncio
from timbal import Agent, Tool
from timbal.types.approval import ApprovalResolution
from timbal.types.events import ApprovalEvent
def refund_customer(amount: int, customer_id: str) -> str:
return f"refunded ${amount} to {customer_id}"
refund = Tool(
handler=refund_customer,
requires_approval=lambda amount, **_: amount > 100,
approval_prompt=lambda amount, customer_id: (
f"Approve refunding ${amount} to {customer_id}?"
),
approval_redact_keys=["customer_id"],
)
agent = Agent(
name="support_agent",
model="openai/gpt-5",
tools=[refund],
)
async def main() -> None:
# First call — agent will try to refund and pause for approval.
pending: list[ApprovalEvent] = []
async for event in agent(prompt="Refund $250 to customer C-42"):
if isinstance(event, ApprovalEvent):
pending.append(event)
# Show the reviewer the redacted prompt + input.
for event in pending:
print(event.prompt, event.input)
# event.input["customer_id"] == "***" (redacted)
# Second call — resume with the reviewer's decision.
decisions = {
event.approval_id: ApprovalResolution(
approved=True,
approver_id="user_42",
comment="Verified invoice and customer history.",
)
for event in pending
}
result = await agent(
prompt="Refund $250 to customer C-42",
resume=decisions,
).collect()
print(result.output.collect_text())
# Refund executed, agent reports success.
asyncio.run(main())
```
## Resuming From a Different Process
To approve in a UI now and resume in a worker later, configure a durable tracing provider and pass `parent_id`:
```python theme={"dark"}
from pathlib import Path
from timbal.state.tracing.providers import JsonlTracingProvider
provider = JsonlTracingProvider.configured(_path=Path("traces.jsonl"))
agent = Agent(
name="support_agent",
model="openai/gpt-5",
tools=[refund],
tracing_provider=provider,
)
# In the worker, after the reviewer decided:
result = await agent(
prompt="Refund $250 to customer C-42",
parent_id=paused_run_id,
resume={approval_id: True},
).collect()
if result.status.reason == "approval_already_claimed":
# Another worker already resumed this approval. Safe to no-op.
return
```
`JsonlTracingProvider` (and `SqliteTracingProvider`) implement durable `(parent_id, approval_id)` claims, so two workers racing on the same gate will not both execute the handler.
## Denying With a Reason
When the agent calls a denied tool, Timbal converts the denial into a `ToolResultContent` so the model can react (apologize, escalate, try another path) instead of crashing:
```python theme={"dark"}
result = await agent(
prompt="Refund $250 to customer C-42",
resume={
approval_id: ApprovalResolution(
approved=False,
reason="Refund exceeds policy limit.",
approver_id="user_42",
)
},
).collect()
```
For direct tool calls (no agent), denial returns `status.reason == "approval_denied"` and the handler does not run.
## Key Features
* **Callable policy** — `requires_approval=lambda amount, **_: amount > 100` runs against the validated handler input
* **Redaction** — `approval_redact_keys` masks fields in the public approval surface; the handler still receives unredacted input
* **Audit fields** — `approver_id`, `comment`, `decided_at` persist under `span.metadata["approval"]["resolution"]`
* **Durable resume** — pair with `JsonlTracingProvider` / `SqliteTracingProvider` and `parent_id` to span processes
* **Duplicate protection** — `claim_approval` ensures a single worker resumes each gate
# Asking the User
Source: https://docs.timbal.ai/examples/agents/ask-user
Interaction tools built on suspend() — free text, single/multi choice, confirmation, structured forms, ratings, uploads, and generative-UI review — each with its own payload shape
`suspend()` lets a tool pause the run and hand control to the user, then resume with whatever they send back. Each tool picks a `kind` (the frontend's renderer discriminator) and a `payload` (what to render). The resume value can be **any JSON type** — a string, a bool, a list, a dict — so the same primitive covers everything from a yes/no to a multi-field form.
See the full reference in the [Human in the Loop](/human-in-the-loop) section. This page is a catalog of shapes you can copy.
## A catalog of interaction tools
```python theme={"dark"}
from timbal import suspend
def ask_text(question: str) -> str:
"""Ask the user an open-ended question. Resumes with their typed answer."""
return suspend({"question": question}, kind="ask_text")
def ask_choice(question: str, options: list[str]) -> str:
"""Ask the user to pick exactly one option. Renders as a single-select."""
return suspend({"question": question, "options": options}, kind="ask_choice")
def ask_user_multi(question: str, options: list[str]) -> list[str]:
"""Ask the user to pick any number of options. Resumes with a list."""
return suspend({"question": question, "options": options}, kind="ask_user_multi")
def confirm(action: str) -> bool:
"""Ask the user to confirm before proceeding. Resumes with a bool."""
return bool(suspend({"action": action}, kind="confirm"))
def ask_form(title: str, fields: list[dict]) -> dict:
"""Collect several values at once. Resumes with a {field_name: value} dict."""
return suspend({"title": title, "fields": fields}, kind="ask_form")
def ask_rating(question: str, scale: int = 5) -> int:
"""Ask for a rating on a 1..scale scale. Resumes with an int."""
return int(suspend({"question": question, "scale": scale}, kind="ask_rating"))
def request_upload(prompt: str, accept: list[str]) -> str:
"""Ask the user to upload a file. Resumes with a URL or file id."""
return suspend({"prompt": prompt, "accept": accept}, kind="request_upload")
def review_chart(title: str, series: list[dict]) -> dict:
"""Generative UI: render a chart for the user and wait for their tweaks.
Resumes with {"approved": bool, "edits": {...}}."""
return suspend({"title": title, "series": series}, kind="review_chart")
```
`suspend` is exported at the top level (`from timbal import suspend`). Ready-made `ask_user`, `ask_user_multi`, and `confirm` also ship in `timbal.tools` (`from timbal.tools import ask_user, ask_user_multi, confirm`).
The handler **re-executes from the top** on resume, so put `suspend()` before any non-idempotent side-effect (or make everything before it idempotent). For irreversible actions, gate them with [`requires_approval`](/human-in-the-loop/approval-gates) instead — that pauses *before* any handler code runs.
## Payload shapes at a glance
This is the contract your frontend renders against. Switch your UI on `kind`, render `payload`, and send the matching resume value back keyed by `interaction_id`.
| `kind` | `payload` shape | resume value |
| ---------------- | --------------------------------------------------- | -------------------------------------- |
| `ask_text` | `{ "question": str }` | `str` |
| `ask_choice` | `{ "question": str, "options": [str] }` | `str` (one option) |
| `ask_user_multi` | `{ "question": str, "options": [str] }` | `[str]` |
| `confirm` | `{ "action": str }` | `bool` |
| `ask_form` | `{ "title": str, "fields": [{name, label, type}] }` | `{ name: value }` |
| `ask_rating` | `{ "question": str, "scale": int }` | `int` |
| `request_upload` | `{ "prompt": str, "accept": [str] }` | `str` (url / file id) |
| `review_chart` | `{ "title": str, "series": [...] }` | `{ "approved": bool, "edits": {...} }` |
## Driving the loop
Give an agent whichever interaction tools fit your product, then run the pause/resume loop. The agent decides which tool to call; you render the payload and resume with the value.
```python theme={"dark"}
import asyncio
from timbal import Agent
from timbal.types.events import InteractionEvent, OutputEvent
agent = Agent(
name="onboarding_assistant",
model="openai/gpt-5",
tools=[ask_text, ask_choice, ask_user_multi, confirm, ask_form, ask_rating],
system_prompt=(
"You onboard new users. Gather what you need by calling the interaction "
"tools — never guess a value the user hasn't given you."
),
)
def render_and_collect(kind: str, payload: dict):
"""Your UI. Return the value matching the kind (see the table above)."""
if kind == "confirm":
return True
if kind == "ask_choice":
return payload["options"][0]
if kind == "ask_user_multi":
return payload["options"][:2]
if kind == "ask_form":
return {f["name"]: "..." for f in payload["fields"]}
if kind == "ask_rating":
return payload["scale"]
return "Acme Inc." # ask_text / request_upload / ...
async def main() -> None:
prompt = "Help me set up my workspace."
while True:
pending: list[InteractionEvent] = []
final: OutputEvent | None = None
async for event in agent(prompt=prompt, parent_id=getattr(main, "_run_id", None)):
if isinstance(event, InteractionEvent):
pending.append(event)
if isinstance(event, OutputEvent) and event.path == "onboarding_assistant":
final = event
# Finished — no more questions.
if final.status.reason != "input_required":
print(final.output.collect_text())
return
# Answer every pending question and resume from this run.
main._run_id = final.run_id
resume = {
it.interaction_id: render_and_collect(it.kind, it.payload)
for it in pending
}
result = await agent(prompt=prompt, parent_id=final.run_id, resume=resume).collect()
if result.status.reason != "input_required":
print(result.output.collect_text())
return
main._run_id = result.run_id
asyncio.run(main())
```
A single turn can open **multiple** interactions at once (the model calls several tools in parallel). You receive one `InteractionEvent` per question and send all answers back in one `resume` map: `{ "": ..., "": ... }`. Approval gates ride the same channel — mix freely.
## Structured form example
`ask_form` is the workhorse for collecting several values in one round-trip instead of a chain of questions:
```python theme={"dark"}
def collect_company_profile() -> dict:
return ask_form(
title="Company profile",
fields=[
{"name": "company_name", "label": "Company name", "type": "text"},
{"name": "team_size", "label": "Team size", "type": "number"},
{"name": "industry", "label": "Industry", "type": "select",
"options": ["SaaS", "Fintech", "Healthcare", "Other"]},
{"name": "wants_newsletter", "label": "Subscribe to updates?", "type": "boolean"},
],
)
```
The emitted `InteractionEvent.payload` is exactly the dict you passed; resume with the filled values:
```python theme={"dark"}
resume = {
interaction_id: {
"company_name": "Acme Inc.",
"team_size": 25,
"industry": "SaaS",
"wants_newsletter": True,
}
}
```
## Generative UI: render, then continue
`review_chart` shows how the same mechanism powers "render something, let the user tweak it, then keep going". The tool emits the data to draw; the resume value carries the user's edits back into the run:
```python theme={"dark"}
def propose_dashboard(metric: str) -> dict:
decision = review_chart(
title=f"{metric} over the last 30 days",
series=[{"label": metric, "points": fetch_points(metric)}],
)
# decision == {"approved": True, "edits": {"range": "90d"}}
return decision
```
If the user changes the range, you get `{"approved": True, "edits": {"range": "90d"}}` back and the handler continues with their choice — no separate "apply" endpoint needed.
## Resuming across processes
Everything above works in-process. To pause in a browser now and resume in a worker later, configure a durable provider and pass `parent_id` on resume:
```python theme={"dark"}
from pathlib import Path
from timbal.state.tracing.providers import JsonlTracingProvider
agent = Agent(
name="onboarding_assistant",
model="openai/gpt-5",
tools=[ask_text, ask_form, confirm],
tracing_provider=JsonlTracingProvider.configured(_path=Path("traces.jsonl")),
)
# Later, in any process:
result = await agent(
prompt="Help me set up my workspace.",
parent_id=paused_run_id,
resume={interaction_id: "Acme Inc."},
).collect()
```
See [Resuming a paused run](/human-in-the-loop/resuming) and [Client integration (HTTP)](/human-in-the-loop/client-integration) for the durable providers and the HTTP `/stream` wire contract your frontend talks to.
# Audio
Source: https://docs.timbal.ai/examples/agents/audio
AI agents can process audio files for transcription and analysis using speech-to-text capabilities
Transcribe audio files in a `pre_hook` before processing.
This example uses OpenAI's transcription API, but you can use any transcription service (ElevenLabs, Google Cloud, or custom implementations).
The `[Audio]` prefix is added to the transcribed text to clearly indicate that the content originated from an audio file. This helps the agent understand the context and source of the information, which can be useful for:
* **Context Awareness**: The agent knows the text came from audio transcription
* **Mixed Content**: When combining text and audio in the same prompt, the prefix distinguishes transcribed content
* **Traceability**: Makes it easier to track which parts of the conversation came from audio vs. text input
```python theme={"dark"}
from timbal import Agent
from timbal.state import get_run_context
from timbal.types.file import File
from timbal.types.content import content_factory
import os
from openai import AsyncOpenAI
async def stt(audio_file: File) -> str:
"""Transcribe an audio file."""
client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
transcript = await client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
)
return transcript.text
async def pre_hook():
"""Transcribe audio files before processing."""
span = get_run_context().current_span()
prompt = span.input.get("prompt")
# Transcribe audio file and add prefix
if (isinstance(prompt, File) and
prompt.__content_type__ and
prompt.__content_type__.startswith("audio/")):
transcription = await stt(prompt)
span.input["prompt"] = content_factory(f"[Audio]: {transcription}")
agent = Agent(
name="AudioAgent",
model="openai/gpt-4.1-mini",
pre_hook=pre_hook
)
audio_file = File.validate("/path/to/recording.wav")
result = await agent(prompt=audio_file).collect()
```
This example uses OpenAI's transcription API directly. For more advanced features like language detection, timestamps, and better error handling, refer to the [OpenAI Audio API documentation](https://platform.openai.com/docs/guides/speech-to-text).
## Key Features
* **Pre-hook Transcription**: Audio is transcribed before the agent processes it
* **Any Model**: Works with any text model, not just audio-capable ones
* **Flexible Providers**: Use any transcription service (OpenAI, ElevenLabs, or custom)
* **Audio Prefix**: The "\[Audio]" prefix clearly indicates transcribed content
* **File Support**: Works with local files, URLs, and base64 data
# Content-Aware Tools
Source: https://docs.timbal.ai/examples/agents/content-aware-tools
Use ToolSet to resolve tools dynamically based on content type - automatically select image or PDF tools
Use `ToolSet` to resolve tools dynamically at runtime. This example shows how tools automatically adapt based on the content type:
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.core.tool_set import ToolSet
from timbal.state import get_run_context
from timbal.types.file import File
def analyze_image(image_path: str) -> str:
"""Analyze an image using computer vision."""
return f"Image analysis for: {image_path}"
def analyze_pdf(pdf_path: str) -> str:
"""Analyze a PDF document."""
return f"Analysis for: {pdf_path}"
def get_file_type(file_url: File) -> str:
"""Detect file type from content type or extension."""
content_type = file_url.__content_type__
if content_type and content_type.startswith("image/"):
return "image"
elif content_type and content_type.startswith("application/pdf"):
return "pdf"
return "unknown"
class FileTypeBasedToolSet(ToolSet):
async def resolve(self) -> list[Tool]:
span = get_run_context().current_span()
prompt = span.input.get("prompt", "")
tools = []
if isinstance(prompt, File):
file_type = get_file_type(prompt)
if file_type == "image":
tools = [Tool(handler=analyze_image)]
elif file_type == "pdf":
tools = [Tool(handler=analyze_pdf)]
return tools
agent = Agent(
name="FileTypeAwareAgent",
model="openai/gpt-5.2",
tools=[FileTypeBasedToolSet()]
)
# It does not have any tool available
result1 = await agent(
prompt="What is Python?",
).collect()
# Image analysis tool added
result2 = await agent(
prompt=File.validate('/path/to/image.jpg')
).collect()
# PDF analysis tool added
result3 = await agent(
prompt=File.validate('/path/to/document.pdf'),
).collect()
```
The `resolve()` method is called before each LLM call. It detects the file type and returns different tools:
* **Text/no file**: No tools available
* **Image files**: Adds `analyze_image`
* **PDF files**: Adds `analyze_pdf`
## Key Features
* **File Type Detection**: Automatically detects file type from content type
* **Context-Aware Tools**: Only relevant tools are exposed based on content type
* **Token Efficiency**: Reduces token usage by showing only necessary tools
* **Flexible**: Easily extend to support more file types
# CSV
Source: https://docs.timbal.ai/examples/agents/csv
AI agents can analyze and process CSV data files for insights and data manipulation
Agents automatically handle CSV files when included in prompts. First validate your CSV file with Timbal's `File` type, then pass it alongside text in a list:
```python theme={"dark"}
from timbal import Agent
from timbal.types.file import File
agent = Agent(
name="DataAgent",
model="openai/gpt-5",
system_prompt="Analyze CSV data and provide insights."
)
# Validate CSV file and analyze
csv_file = File.validate("path/to/data.csv")
result = await agent(
prompt=["What insights can you find in this data?", csv_file]
).collect()
print(result.output.collect_text())
```
## Key Features
* **Automatic Processing**: CSV files are automatically parsed and converted to the correct format
* **Data Analysis**: Extract insights, patterns, and summaries from structured data
* **File Support**: Works with local files, URLs, and base64 data
# Custom Functions in System Prompts
Source: https://docs.timbal.ai/examples/agents/custom-functions-system-prompts
Use your own functions to build dynamic system prompts that include real-time data and custom logic
Pass a function to `system_prompt` that builds the prompt using your custom functions:
```python theme={"dark"}
from datetime import datetime
from timbal import Agent
from timbal.state import get_run_context
def get_system_prompt() -> str:
span = get_run_context().current_span()
current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
user_id = span.input.get("user_id", "unknown")
return f"""You are a helpful assistant.
Current time: {current_time}
User: {user_id}"""
agent = Agent(
name="TimeAwareAgent",
model="openai/gpt-4o-mini",
system_prompt=get_system_prompt
)
```
Your function is executed each time the agent runs, ensuring the system prompt always contains up-to-date information.
## Key Features
* **Custom Functions**: Define your own functions to fetch and format data
* **Automatic Execution**: Your function is called each time the agent runs
* **Access Runtime Context**: Use `get_run_context()` to access current execution data and user inputs
* **Full Control**: Build the system prompt dynamically with any logic you need
* **Real-time Data**: Perfect for integrating APIs, databases, or any dynamic data source
# Word
Source: https://docs.timbal.ai/examples/agents/docx
AI agents can read and analyze Microsoft Word documents for content extraction and processing
Agents automatically handle DOCX files when included in prompts. First validate your DOCX file with Timbal's `File` type, then pass it alongside text in a list:
```python theme={"dark"}
from timbal import Agent
from timbal.types.file import File
agent = Agent(
name="DocumentAgent",
model="gemini/gemini-2.5-pro",
system_prompt="Analyze Word documents and extract key information."
)
# Validate DOCX file and analyze
docx_file = File.validate("path/to/document.docx")
result = await agent(
prompt=["Summarize this document and highlight the main sections", docx_file]
).collect()
print(result.output.collect_text())
```
## Key Features
* **Automatic Processing**: DOCX files are automatically parsed and converted to readable format
* **Document Analysis**: Extract text, structure, and insights from Word documents
* **File Support**: Works with local files, URLs, and base64 data
# Email
Source: https://docs.timbal.ai/examples/agents/emls
AI agents can process and summarize email files for quick content extraction and analysis
Agents can analyze EML files to extract key information and provide summaries. First validate your EML file with Timbal's `File` type, then pass it alongside text in a list:
```python theme={"dark"}
from timbal import Agent
from timbal.types.file import File
agent = Agent(
name="EmailSummarizer",
model="openai/gpt-4.1-mini",
system_prompt="Summarize emails and extract key information like sender, subject, main points, and action items."
)
# Validate EML file and summarize
eml_file = File.validate("path/to/email.eml")
result = await agent(
prompt=["Summarize this email and extract the key points", eml_file]
).collect()
print(result.output.collect_text())
```
## Key Features
* **Email Summarization**: Extract key information from email content
* **Structured Analysis**: Identify sender, subject, main points, and action items
* **File Support**: Works with local files, URLs, and base64 data
# Execution Behavior
Source: https://docs.timbal.ai/examples/agents/execution-behavior
Validate agent execution flow, tool usage, and sequence of operations
Execution behavior validation ensures your agent follows the expected workflow. Verify tool selection, parameter values, execution order, and sequence compliance using flow validators.
## Example
This example demonstrates how to validate complex execution flows with parallel tool execution and sequential processing using nested `seq!` and `parallel!` validators.
### Eval Configuration
```yaml evals.yaml theme={"dark"}
- name: eval_travel_assistant_workflow
description: Fetch data in parallel then processes sequentially
runnable: agent.py::agent
params:
prompt: "I'm planning a trip to Madrid. What's the current time, the weather forecast, and flight prices?"
seq!:
- llm
- parallel!:
- get_datetime:
input:
timezone:
eq!: "Europe/Madrid"
- get_weather:
input:
city:
eq!: "Madrid"
- search_flights:
input:
destination:
contains!: "Madrid"
- llm
```
### Agent Implementation
```python agent.py theme={"dark"}
from timbal import Agent
async def get_datetime(timezone: str) -> str:
"""Get current datetime for a timezone."""
return f"2024-01-15 14:30:00 {timezone}"
async def get_weather(city: str) -> str:
"""Get weather forecast for a city."""
weather_data = {
"Madrid": "Sunny, 18°C",
"Barcelona": "Cloudy, 15°C"
}
return weather_data.get(city, "Weather data unavailable")
async def search_flights(destination: str) -> str:
"""Search for flights to a destination."""
return f"Found flights to {destination}"
agent = Agent(
name="travel_assistant",
model="openai/gpt-5.2",
tools=[get_datetime, get_weather, search_flights],
)
```
### Running Evaluations
```bash theme={"dark"}
python -m timbal.evals.cli evals.yaml
```
## How It Works
1. **Initial Processing**: The agent starts with an LLM call to understand the request.
2. **Parallel Execution**: Two tools (`get_datetime` and `get_weather`) execute in parallel using `parallel!`, improving efficiency by fetching independent data simultaneously.
3. **Sequential Processing**: After parallel execution completes, `search_flights` runs sequentially, as it may depend on the previous results.
4. **Final Processing**: A final LLM call synthesizes all the gathered information into a response.
5. **Validation**: The `seq!` validator ensures:
* Tools execute in the correct order
* Parallel tools run simultaneously (overlapping execution times)
* Tool inputs match expected values using nested validators
## Evaluation Results
### Successful Validation
When the agent follows the expected workflow with parallel and sequential execution:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
PASSED evals.yaml::eval_travel_assistant_workflow [4.65s]
└── travel_assistant
└── ✓ seq!
├── llm
├── ✓ parallel!
│ ├── get_datetime
│ │ └── ✓ input.timezone.eq! ("Europe/Madrid")
│ └── get_weather
│ └── ✓ input.city.eq! ("Madrid")
├── search_flights
│ └── ✓ input.destination.contains! ("Madrid")
└── llm
============================= 1 passed in 4.65s ==============================
```
### Failed Validation
When the agent doesn't follow the expected execution pattern:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
FAILED evals.yaml::eval_travel_assistant_workflow [4.36s]
└── travel_assistant
└── ✗ seq!
├── llm
├── ✗ parallel!
│ ├── get_datetime
│ │ └── ✓ input.timezone.eq! ("Europe/Madrid")
│ └── get_weather
│ └── ✓ input.city.eq! ("Madrid")
├── search_flights
│ └── ✓ input.destination.contains! ("Madrid")
└── llm
!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1 failed in 4.36s !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
```
## Key Features
* **Nested Flow Validation**: Combine `seq!` and `parallel!` to validate complex execution patterns
* **Parallel Execution**: Use `parallel!` to ensure independent tools run simultaneously for better performance
* **Sequential Processing**: Use `seq!` to enforce order when tools depend on previous results
* **Tool Input Validation**: Validate tool input parameters using nested validators within flow validators
* **Span Validation**: Validate individual span properties (input, output, elapsed, usage) within sequences
* **Workflow Compliance**: Ensure agents follow expected execution patterns and optimize tool usage
# Image Generation
Source: https://docs.timbal.ai/examples/agents/image-generation
AI agents can generate images from text descriptions using OpenAI's image generation tools
Agents can create images from text prompts using image generation tools. This example uses a post hook to save generated images to disk and return file paths:
```python theme={"dark"}
import os
from openai import AsyncOpenAI
from pydantic import Field
from timbal import Agent, Tool
from timbal.state import get_run_context
from timbal.types.file import File
async def gen_images(
prompt: str = Field(
...,
description="A text description of the desired image(s). Max length: 32000 characters.",
max_length=32000,
),
) -> list[File]:
client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
kwargs = {}
if prompt is not None:
kwargs["prompt"] = prompt
res = await client.images.generate(
model="gpt-image-1",
**kwargs,
)
files = [File.validate(f"data:image/png;base64,{image.b64_json}") for image in res.data]
return files
def post_hook():
"""Simple post hook to save files to disk."""
span = get_run_context().current_span()
if hasattr(span.output, '__iter__') and span.output:
saved_files = []
for i, file_obj in enumerate(span.output):
if hasattr(file_obj, 'to_disk'):
# Save to specific directory
output_dir = Path("generated_images")
output_dir.mkdir(exist_ok=True)
file_path = output_dir / f"image_{i+1}.png"
file_obj.to_disk(file_path)
saved_files.append(str(file_path))
if saved_files:
span.output = saved_files
# Create tool with post hook
gen_images_with_save = Tool(
name="gen_images",
handler=gen_images,
post_hook=post_hook
)
agent = Agent(
name="ImageAgent",
model="openai/gpt-4.1",
tools=[gen_images_with_save],
system_prompt="Generate images from text descriptions when requested."
)
result = await agent(
prompt="Create a picture of a cat."
).collect()
print("Generated image saved to:", result.output.collect_text())
```
## Key Features
* **High Quality**: Generate professional-quality images
* **Customizable**: Control size, style, and format
* **File Output**: Returns image files ready for use
* **Post Hooks**: Automatically save files to disk and return file paths
# Images
Source: https://docs.timbal.ai/examples/agents/images
AI agents can analyze and understand images by processing visual content alongside text instructions
Agents automatically handle image files when included in prompts. First validate your image file with Timbal's `File` type, then pass it alongside text in a list:
```python theme={"dark"}
from timbal import Agent
from timbal.types.file import File
agent = Agent(
name="VisionAgent",
model="anthropic/claude-sonnet-4-6", # Vision-capable model
max_tokens=256,
system_prompt="Analyze images and provide detailed descriptions."
)
# Validate multiple image files and analyze
image1 = File.validate("path/to/image1.jpg")
image2 = File.validate("path/to/image2.png")
image3 = File.validate("path/to/image3.jpeg")
result = await agent(
prompt=["Analyze these images and provide a description of each one", image1, image2, image3]
).collect()
print(result.output.collect_text())
```
Use vision-capable models for image processing. Check the [Model Reference](/models/overview) for models with vision support.
## Key Features
* **Automatic Processing**: Images are automatically converted to the correct format for vision models
* **Multi-modal**: Combine text and images in the same conversation
* **File Support**: Works with local files, URLs, and base64 data
# Input Parameters
Source: https://docs.timbal.ai/examples/agents/input-validation
Pass input parameters to agents and validate the output response
Parameters in `params` become part of the agent's input, and you can only validate the resulting output response.
## Example
This example demonstrates how to pass input parameters to the agent and validate the output. The `premium_user` parameter is passed via `params` and becomes part of the agent's input, which can be accessed in the system prompt function.
### Eval Configuration
```yaml evals.yaml theme={"dark"}
- name: eval_input_has_points
description: Pass premium_user parameter and validate output contains points information
runnable: agent.py::agent
params:
prompt: "Hello, I'm a premium user"
premium_user: True
output:
contains!: "10"
```
### Agent Implementation
```python agent.py theme={"dark"}
from timbal import Agent
from timbal.state import get_run_context
def get_system_prompt() -> str:
"""Build system prompt with premium_user status from input."""
span = get_run_context().current_span()
premium_user = span.input.get("premium_user", False)
return f"""You are a points system assistant.
Premium users have 10 points.
Premium user: {premium_user}"""
agent = Agent(
name="points_agent",
model="openai/gpt-4.1-mini",
system_prompt=get_system_prompt
)
```
### Running Evaluations
```bash theme={"dark"}
python -m timbal.evals.cli evals.yaml
```
## How It Works
1. **Parameters**: The `premium_user: True` parameter is passed via `params` and becomes part of the agent's input.
2. **System Prompt**: The `get_system_prompt()` function accesses the `premium_user` value from the input using `get_run_context()` and builds the system prompt dynamically.
3. **Output Validation**: The output is validated to ensure it contains "10" (the points for premium users).
## Evaluation Results
The CLI displays pytest-style output with pass/fail status:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
PASSED evals.yaml::eval_input_has_points [0.45s]
└── points_agent
└── ✓ output.contains! ("10")
============================= 1 passed in 0.45s ==============================
```
When validation fails, the CLI shows detailed error information:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
FAILED evals.yaml::eval_input_has_points [0.48s]
└── points_agent
└── ✗ output.contains! ("10")
Expected: "10"
Actual: "You currently have 0 points..."
!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1 failed in 1.94s !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
```
## Key Features
* **Input Parameters**: Pass parameters directly to the agent via `params` - these become part of the agent's input
* **Dynamic System Prompts**: Use callable functions for `system_prompt` to access input parameters and build prompts dynamically
* **Output Validation**: Validate agent responses using output validators - input parameters cannot be validated directly
# Multi-turn Conversations
Source: https://docs.timbal.ai/examples/agents/multi-turn-conversation
Test agent memory, context retention, and complex conversational flows across multiple interactions
Multi-turn conversation testing ensures your agent maintains context across multiple interactions. Use `params.messages` to establish conversation history and validate the agent's response to the final message.
## Example
This example demonstrates how to test multi-turn conversations by passing a full conversation history via `params.messages` and validating that the agent uses context appropriately.
### Eval Configuration
```yaml evals.yaml theme={"dark"}
- name: eval_weather_advice
description: Test agent uses weather data to provide appropriate advice
runnable: agent.py::agent
params:
messages:
- role: user
content: "What's the weather like in New York?"
- role: assistant
content: "It's currently 15°C and raining in New York."
- role: user
content: "Should I bring an umbrella?"
output:
contains!:
value: "yes"
transform: lowercase
seq!:
- llm
# Only llm should be called
# get_weather should not be called since context is available
```
### Agent Implementation
```python agent.py theme={"dark"}
from timbal import Agent
def get_weather(location: str) -> str:
"""Get current weather information for a specific location."""
weather_data = {
"New York": "15°C and raining",
"London": "12°C and cloudy",
"Tokyo": "22°C and sunny"
}
return weather_data.get(location, f"Weather data not available for {location}")
agent = Agent(
name="weather_agent",
model="openai/gpt-5.2",
system_prompt="You are a helpful weather assistant.",
tools=[get_weather],
)
```
### Running Evaluations
```bash theme={"dark"}
python -m timbal.evals.cli evals.yaml
```
## How It Works
1. **Conversation History**: The `params.messages` array establishes the full conversation history, including previous user messages and assistant responses.
2. **Context Usage**: The agent receives the entire conversation history, so it can remember what was said in previous turns and answer accordingly.
3. **Output Validation**: The `output` validator checks that the agent's response contains the expected content (e.g., "yes" for the umbrella question).
4. **Sequence Validation**: The `seq!` validator ensures the agent only calls `llm` and doesn't unnecessarily call `get_weather` again, since the weather information is already available in the conversation history.
## Evaluation Results
### Successful Validation
When the agent remembers context and provides the correct response:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
PASSED evals.yaml::eval_weather_advice [1.06s]
└── weather_agent
├── ✓ seq!
│ └── llm
└── ✓ output.contains! ("yes") ⤳ lowercase
============================= 1 passed in 1.06s ==============================
```
### Failed Validation
When the agent doesn't use context or provides an incorrect response:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
FAILED evals.yaml::eval_weather_advice [2.91s]
└── weather_agent
├── ✗ seq!
│ └── llm
└── ✗ output.contains! ("yes") ⤳ lowercase
!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1 failed in 2.91s !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
```
## Key Features
* **Conversation History**: Use `params.messages` to establish full conversation context
* **Context Memory**: Agents receive the entire conversation history and can remember previous interactions
* **Sequence Validation**: Use `seq!` to verify agents don't call tools unnecessarily when context is available
* **Response Continuity**: Ensure agents build logically on previous interactions
# Output Validation
Source: https://docs.timbal.ai/examples/agents/output-validation
Validate agent responses using content checks, format validation, and semantic evaluation
Output validation ensures your agent produces correct, well-formatted responses. You can validate content structure, exclude unwanted text, match patterns with regex, and use LLM-powered semantic evaluation.
## Example
This example demonstrates how to validate agent outputs using multiple validators including content checks, format validation, timing, and usage metrics.
### Eval Configuration
```yaml evals.yaml theme={"dark"}
- name: eval_creative_writer_response
description: Validate creative writing agent provides well-structured stories
runnable: agent.py::agent
params:
prompt: "Write a story about a robot learning to paint"
output:
contains_all!: ["Title", "Story", "Lesson"]
not_contains!: ["error", "failed"]
pattern!: "^Title: .+"
elapsed:
lt!: 15000
llm:
usage:
output_text_tokens:
lte!: 500
```
In this example, we use `output_text_tokens` instead of `output_tokens` because the agent uses OpenAI (`openai/gpt-5.2`). For Anthropic models, use `output_tokens` instead. See [Validating Token Usage](/evals/writing-evals#validating-token-usage) for more details.
### Agent Implementation
```python agent.py theme={"dark"}
from timbal import Agent
agent = Agent(
name="creative_writer",
model="openai/gpt-5.2",
system_prompt="""You are a creative writing assistant.
For any story request, always provide:
1. A compelling title
2. A complete short story (2 sentences)
3. A moral or lesson
Format your response as:
Title: [story title]
Story: [complete narrative]
Lesson: [moral or takeaway]"""
)
```
### Running Evaluations
```bash theme={"dark"}
python -m timbal.evals.cli evals.yaml
```
## How It Works
1. **Output Validation**: Multiple validators check the agent's response for required content (`contains_all!`), excluded content (`not_contains!`), and format (`pattern!`).
2. **Timing Validation**: The `elapsed` validator ensures the agent responds within the specified time limit.
3. **Usage Validation**: Span-level validators track resource consumption, such as token usage for LLM calls.
4. **Combined Validators**: All validators must pass for the eval to succeed.
## Evaluation Results
### Successful Validation
When all validators pass:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
PASSED evals.yaml::eval_creative_writer_response [0.52s]
└── creative_writer
├── ✓ output.contains_all! (["Title", "Story", "Lesson"])
├── ✓ output.not_contains! (["error", "failed"])
├── ✓ output.pattern! ("^Title: .+")
├── ✓ elapsed.lt! (15000)
└── ✓ llm.usage.output_text_tokens.lte! (500)
============================= 1 passed in 0.52s ==============================
```
### Failed Validation
When any validator fails:
```
──────────────────── Timbal Evals ────────────────────
collected 1 evals from 1 file
FAILED evals.yaml::eval_creative_writer_response [6.14s]
└── creative_writer
├── ✗ output.contains_all! (["Title", "Story", "Lesson"])
├── ✓ output.not_contains! (["error", "failed"])
├── ✗ output.pattern! ("^Title: .+")
├── ✓ elapsed.lt! (15000)
└── ✓ llm.usage.output_text_tokens.lte! (500)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1 failed in 6.14s !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
```
## Key Features
* **Content Validation**: Verify required keywords (`contains_all!`) and exclude unwanted content (`not_contains!`)
* **Format Validation**: Ensure responses follow expected structure with `pattern!` regex validation
* **Time Validation**: Monitor execution time with `elapsed` validators (`lt!`, `lte!`, etc.)
* **Usage Validation**: Track resource consumption with span-level `usage` validators (e.g., `llm.usage.output_text_tokens` for OpenAI or `llm.usage.output_tokens` for Anthropic)
* **Combined Validators**: Use multiple validators together - all must pass for the eval to succeed
# PDF
Source: https://docs.timbal.ai/examples/agents/pdf
AI agents can read and analyze PDF documents for content extraction and processing
Agents automatically handle PDF files when included in prompts. First validate your PDF file with Timbal's `File` type, then pass it alongside text in a list:
```python theme={"dark"}
from timbal import Agent
from timbal.types.file import File
agent = Agent(
name="DocumentAgent",
model="openai/gpt-4.1", # Vision-capable model
system_prompt="Analyze PDF documents and extract key information."
)
# Validate PDF file and analyze
pdf_file = File.validate("path/to/document.pdf")
result = await agent(
prompt=["Summarize this document and highlight the main points", pdf_file]
).collect()
print(result.output.collect_text())
```
PDFs are converted to images, so vision-capable models are required. Check the [Model Reference](/models/overview) for models with vision support.
## Key Features
* **Automatic Processing**: PDF files are automatically converted to images and parsed for text content
* **Document Analysis**: Extract text, structure, and insights from PDF documents
* **File Support**: Works with local files, URLs, and base64 data
# Web Search
Source: https://docs.timbal.ai/examples/agents/search
AI agents can search the web for real-time information using OpenAI or Anthropic models
Agents can search the web for current information using the built-in `WebSearch` tool. This tool works with **OpenAI** and **Anthropic** models.
See the [Model Reference](/models/overview) to check which specific models support web search.
```python theme={"dark"}
from timbal import Agent
from timbal.tools import WebSearch
# Works with OpenAI or Anthropic models
agent = Agent(
name="SearchAgent",
model="openai/gpt-5.1",
tools=[WebSearch()],
system_prompt="Search the web for current information."
)
# Agent will automatically use search when needed
result = await agent(
prompt="When was the last match of Liverpool?"
).collect()
```
## Key Features
* **Real-time Information**: Access current web data and news
* **Automatic Tool Selection**: Agent decides when to search
* **Citation Support**: Includes source links in responses
# Semantic Search with Embeddings
Source: https://docs.timbal.ai/examples/agents/semantic-search
AI agents can create embeddings and perform intelligent semantic search through natural conversation
Agents can set up semantic search capabilities and find relevant content through simple conversations. The agent handles embedding creation and meaning-based search automatically:
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.platform.kbs.embeddings import create_embedding, list_embedding_models
from timbal.platform.kbs.tables import create_table, get_table, get_tables, import_csv, search_table
from timbal.state import RunContext, set_run_context
from timbal.types.file import File
agent = Agent(
name="ResearchAgent",
model="openai/gpt-4.1-mini",
system_prompt=("""
You are a research assistant. Help users set up semantic search and find relevant articles. Be conversational and helpful.
Configuration:
- org_id: '1'
- kb_id: '70'
If a table doesn't exist, create it immediately without asking for confirmation.
If an embedding already exists, skip creating it and proceed with the search.
Always use these configuration values when calling KB functions.
"""),
tools=[
Tool(handler=create_table, description="Use it to create the articles table. ALWAYS use 'text' data type for all columns. Match exactly the CSV file columns."),
Tool(handler=import_csv, description="Use it to import the CSV file itself to the table."),
Tool(handler=list_embedding_models, description="Use it to list the available embedding models."),
Tool(handler=create_embedding, description="Use it to create embeddings for semantic search. Only create if they don't already exist. Use an available model."),
Tool(handler=search_table, description="Use it to search articles using semantic search. First you have to know the column name of the embeddings."),
Tool(handler=get_table, description="Use it to get the table definitio."),
Tool(handler=get_tables, description="Use it to get the table definition"),
]
)
# First conversation: Setting up the knowledge base
csv_file = File.validate("/path/to/documents.csv")
result1 = await agent(
prompt=["I need to set up a technical support knowledge base. Create an 'articles' table with the columns from the CSV file /path/to/documents.csv and add the CSV file to the table.", csv_file]
).collect()
# Second conversation: Creating embeddings for semantic search
result2 = await agent(
prompt="Now create embeddings for the technical content so I can do semantic search on support documents."
).collect()
# Reset context to simulate a fresh conversation (no memory of previous steps)
run_context = RunContext() # Creates a new empty context
set_run_context(run_context) # Replaces current context, wiping all memory
# Third conversation: Performing semantic search
result3 = await agent(
prompt="Search in articles table for 'How can I install music software on my computer?'"
).collect()
print(result3.output.collect_text())
```
## Key Features
* **Natural Setup**: Create tables, add content, and set up embeddings through conversation
* **Semantic Understanding**: Find content by meaning, not exact keywords
* **Automatic Embeddings**: Agent creates vector embeddings for intelligent search
* **Contextual Matching**: Understand synonyms, related concepts, and intent
* **Flexible Queries**: Natural language queries work better than keyword searches
# SQL Queries & Data Management
Source: https://docs.timbal.ai/examples/agents/sql-queries
AI agents can add data to knowledge bases and then query it through natural conversation
Agents can manage your knowledge base data through simple conversations. Add records naturally, then ask questions about your data - the agent handles all the complex SQL operations behind the scenes:
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.platform.kbs.tables import create_table, import_records, query, get_tables, get_table
agent = Agent(
name="SupportAnalyst",
model="openai/gpt-4.1-mini",
system_prompt=("""
You are a support data manager. Help users add tickets to the database and analyze the data. Be conversational and helpful.
Configuration:
- org_id: 'your-org-id'
- kb_id: 'your-kb-id'
- table_name: 'tickets'
If the tickets table doesn't exist, create it immediately without asking for confirmation.
Always use these configuration values when calling KB functions.
"""),
tools=[
Tool(handler=get_tables, description="Use it to check if a table it is already created"),
Tool(handler=get_table, description="Use it to get the table definition"),
Tool(handler=create_table, description="Use it to create the table"),
Tool(handler=query, description="Use it to query the table"),
Tool(handler=import_records, description="Use it to import records to the table")
] # Add multiple KB functions as tools
)
# First conversation: Adding records to the knowledge base
await agent(
prompt="I need to add a new support ticket. Customer ID 103 reported a login issue with high priority."
).collect()
# Second conversation: Querying the data we just added
result2 = await agent(
prompt="How many high priority tickets do we have from customer 103?"
).collect()
print(result2.output.collect_text())
# There is currently 1 high priority ticket from customer 103. Would you like to see more details or do anything else?
```
![Tickets table in Timbal Platform]()
*The tickets table as it appears in the Timbal Platform interface, showing the record added by the agent.*
## Key Features
* **Natural Conversations**: Add data and ask questions in plain English
* **Automatic SQL Generation**: Agent converts your requests into complex SQL queries
* **Data Management**: Add individual records or import bulk CSV files
* **Real-time Analysis**: Get instant insights from your data
* **PostgreSQL Power**: Full database capabilities behind simple conversations
# Text-to-Speech
Source: https://docs.timbal.ai/examples/agents/tts
AI agents can convert text to speech using OpenAI's TTS capabilities
Agents can generate audio from text using text-to-speech tools. Add the TTS tool to your agent's tools list:
```python theme={"dark"}
import os
from openai import AsyncOpenAI
from pydantic import Field
from timbal import Agent
from timbal.state import get_run_context
from timbal.types.file import File
async def tts(
text: str = Field(
...,
description="The text to convert to speech.",
),
) -> File:
client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
kwargs = {
"input": text,
'model': 'tts-1',
'voice': 'alloy'
}
response = await client.audio.speech.create(**kwargs)
return File.validate(
response.content,
{"extension": ".mp3"}
)
async def post_hook():
"""Post hook to extract and return the audio file."""
span = get_run_context().current_span()
span.output = await tts(text=span.output.collect_text())
span.output.to_disk("audio.mp3")
agent = Agent(
name="VoiceAgent",
model="openai/gpt-4.1-mini",
post_hook=post_hook
)
# Agent will generate audio files
await agent(
prompt="What is the capital of France?"
).collect()
```
## Key Features
* **Multiple Voices**: Choose from various voice options
* **Audio Formats**: Support for MP3, WAV, and other formats
* **File Output**: Returns audio files for download or playback
# Mail Assistant
Source: https://docs.timbal.ai/examples/guides/mail-assistant
AI-powered Gmail integration that triages emails and autonomously drafts intelligent responses
This guide shows how to:
* Connect to the Gmail API to retrieve incoming messages
* Use agents to generate draft replies
## Setup Requirements
Install the required Google API client libraries:
```python theme={"dark"}
google-api-core
google-api-python-client
google-auth
google-auth-oauthlib
```
Go to the [Google Cloud Console](https://console.cloud.google.com/) and create a new project or select an existing one.
1. Navigate to **APIs & Services** > **Library**
2. Search for "Gmail API"
3. Click on **Gmail API** and then **Enable**
There are two ways to authenticate with Gmail API. Choose the one that fits your use case:
## Method 1: OAuth 2.0 (Personal/Desktop Apps)
**Use this if:**
* You're building a personal application
* You want users to authenticate with their own Google account
* You're using a personal Gmail account (not Google Workspace)
### Setup Steps:
**1. Create OAuth 2.0 Credentials**
1. Go to **APIs & Services** > **Credentials**
2. Click **Create Credentials** > **OAuth client ID**
3. Choose **Desktop app** as the application type
4. Download the credentials JSON file and save it as `credentials.json`
**2. Configure OAuth Scopes**
The following scopes are required:
```python theme={"dark"}
SCOPES = [
"https://www.googleapis.com/auth/gmail.readonly",
"https://www.googleapis.com/auth/gmail.compose"
]
```
**3. Generate Access Token**
Run this script once to generate `token.json`:
```python theme={"dark"}
from google_auth_oauthlib.flow import InstalledAppFlow
SCOPES = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.compose'
]
flow = InstalledAppFlow.from_client_secrets_file('credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
with open('token.json', 'w') as token:
token.write(creds.to_json())
print("token.json generated successfully!")
```
This will open a browser window for Google sign-in.
**4. Initialize in Code**
```python theme={"dark"}
from google.oauth2.credentials import Credentials
from google.auth.transport.requests import Request
from googleapiclient.discovery import build
import os
SCOPES = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.compose'
]
def initialize_gmail():
"""Initialize Gmail API connection with OAuth"""
try:
creds = None
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
print("No valid credentials found.")
return None
gmail_service = build('gmail', 'v1', credentials=creds)
# Test connection and get profile
profile = gmail_service.users().getProfile(userId='me').execute()
print(f"Connected to Gmail: {profile['emailAddress']}")
return gmail_service
except Exception as e:
print(f"Error initializing Gmail API: {e}")
return None
```
***
## Method 2: Service Account (Google Workspace)
**Use this if:**
* You have a Google Workspace account
* You need to access Gmail on behalf of multiple users
* You're building a server-side application
**Requirements:**
* Google Workspace account (not personal Gmail)
* Google Workspace Admin access for domain-wide delegation
### Setup Steps:
**1. Create a Service Account**
1. Go to **IAM & Admin** > **Service Accounts**
2. Click **CREATE SERVICE ACCOUNT**
3. Enter name: `gmail-service-account`
4. Add description: `Service account for Gmail API access`
5. Click **CREATE AND CONTINUE**, then **DONE**
6. Note the service account email: `gmail-service-account@your-project-id.iam.gserviceaccount.com`
**2. Create and Download Service Account Key**
1. Click on your service account
2. Go to **KEYS** tab
3. Click **ADD KEY** > **Create new key**
4. Select **JSON** and click **CREATE**
5. Save the downloaded file as `credentials.json`
6. **Keep this file secure!**
**3. Configure Domain-Wide Delegation**
Requires Google Workspace Admin privileges
1. Open `credentials.json` and copy the `client_id` value (e.g., `103635629912027933995`)
2. Go to [Google Workspace Admin Console](https://admin.google.com/)
3. Navigate to **Security** > **Access and data control** > **API Controls**
4. Click **Manage Domain Wide Delegation** > **Add new**
5. Paste the **Client ID** and add scopes:
```
https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/gmail.compose
```
6. Click **Authorize**
**4. Initialize Service Account in Code**
```python theme={"dark"}
from google.oauth2 import service_account
from googleapiclient.discovery import build
SCOPES = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.compose'
]
CREDENTIALS_FILE = 'credentials.json'
DELEGATED_USER = 'your-email@yourdomain.com'
def initialize_gmail():
"""Initialize Gmail API connection with service account"""
try:
credentials = service_account.Credentials.from_service_account_file(
CREDENTIALS_FILE, scopes=SCOPES)
delegated_credentials = credentials.with_subject(DELEGATED_USER)
gmail_service = build('gmail', 'v1', credentials=delegated_credentials)
# Test connection and get profile
profile = gmail_service.users().getProfile(userId='me').execute()
print(f"Connected to Gmail: {profile['emailAddress']}")
return gmail_service
except Exception as e:
print(f"Error initializing Gmail API: {e}")
return None
```
## Implementation
This section walks through the essential components needed to implement Gmail integration with Timbal.
For the complete example, visit our [GitHub repository](https://github.com/timbal-ai/timbal/tree/main/examples/gmail).
### Gmail Initialization
Establish a connection to the Gmail API using your credentials. Choose the method that matches your setup:
```python theme={"dark"}
def initialize_gmail(self):
"""Initialize Gmail API connection with OAuth"""
try:
creds = None
token_file = 'token.json'
if os.path.exists(token_file):
creds = Credentials.from_authorized_user_file(token_file, self.scopes)
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
print("Refreshing expired credentials...")
creds.refresh(Request())
else:
print("No valid credentials found.")
return False
self.gmail_service = build('gmail', 'v1', credentials=creds)
# Test connection and get initial history ID
profile = self.gmail_service.users().getProfile(userId='me').execute()
print(f"Connected to Gmail: {profile['emailAddress']}")
# Get initial history ID from profile
self.last_history_id = profile.get('historyId')
if self.last_history_id:
print(f"Starting from history ID: {self.last_history_id}")
else:
print("No history ID found, will monitor from now")
return True
except Exception as e:
print(f"Error initializing Gmail API: {e}")
return False
```
```python theme={"dark"}
def initialize_gmail(self):
"""Initialize Gmail API connection with Service Account"""
try:
CREDENTIALS_FILE = 'credentials.json'
DELEGATED_USER = 'your-email@yourdomain.com'
credentials = service_account.Credentials.from_service_account_file(
CREDENTIALS_FILE, scopes=self.scopes)
delegated_credentials = credentials.with_subject(DELEGATED_USER)
self.gmail_service = build('gmail', 'v1', credentials=delegated_credentials)
# Test connection and get initial history ID
profile = self.gmail_service.users().getProfile(userId='me').execute()
print(f"Connected to Gmail: {profile['emailAddress']}")
# Get initial history ID from profile
self.last_history_id = profile.get('historyId')
if self.last_history_id:
print(f"Starting from history ID: {self.last_history_id}")
else:
print("No history ID found, will monitor from now")
return True
except Exception as e:
print(f"Error initializing Gmail API: {e}")
return False
```
### Email monitoring
Monitor the Gmail inbox for new messages using polling:
```python theme={"dark"}
async def check_for_new_messages(self):
"""Check for new messages since last check"""
try:
# Check for new messages since last history ID
history = self.gmail_service.users().history().list(
userId='me',
startHistoryId=self.last_history_id,
historyTypes=['messageAdded']
).execute()
new_messages = []
for history_record in history.get('history', []):
for message_added in history_record.get('messagesAdded', []):
message_id = message_added['message']['id']
message_details = self.get_message_details(message_id)
if message_details:
# Filter out draft messages
if not self.is_draft_message(message_details):
new_messages.append(message_details)
else:
print(f"📝 Skipping draft message: {message_details.get('subject', 'No Subject')}")
if new_messages:
print(f"\n🔔 Found {len(new_messages)} new message(s) at {datetime.now().strftime('%H:%M:%S')}")
for message in new_messages:
await self.generate_draft(message)
# Update history ID
if history.get('history'):
self.last_history_id = history['history'][-1]['id']
except HttpError as error:
print(f"Error checking for new messages: {error}")
async def start_monitoring(self):
"""Start monitoring Gmail for new messages using polling"""
while True:
await self.check_for_new_messages()
await asyncio.sleep(10) # Check new emails every 10 seconds
```
### Create a Timbal Agent for intelligent email responses
Create an Agent to generate emails responses:
```python theme={"dark"}
agent = Agent(
name="email_response_generator",
model="openai/gpt-4o-mini",
system_prompt="You are an assistant that helps draft professional email responses.",
post_hook=self.save_draft,
)
```
### Call a Timbal Agent to generate the draft
Process incoming emails by creating the prompt and invoking the Agent to generate a response:
```python theme={"dark"}
def _create_email_prompt(self, email):
# Extract email information
subject = email.get('subject', 'No Subject')
sender = email.get('from', 'Unknown Sender')
body = email.get('body', '')
snippet = email.get('snippet', '')
content = body if body else snippet
prompt = f"""
You are an AI assistant that helps draft professional email responses.
Please write a helpful and appropriate response to this email:
**Received Email:**
From: {sender}
Subject: {subject}
Content: {content}
**Instructions:**
- Write a professional response
- Do not assume anything that is not explicitly stated in the email
- Address any questions or requests appropriately
- Keep the tone friendly but business-appropriate
- If you need more information, ask clarifying questions
- End with a professional closing
- Keep the response concise but complete
**Your Response:**
"""
return prompt.strip()
async def generate_draft(self, message):
prompt = self._create_email_prompt(message)
# Generate and create draft
try:
response_text = await self.agent(prompt=prompt, input_email=message).collect()
if response_text:
print("Generated Response:")
print("-" * 40)
print(response_text)
print("-" * 40)
print("Draft created successfully!")
print("="*60)
else:
print("Failed to generate response")
except Exception as e:
print(f"Error generating/creating draft: {e}")
print("="*60)
```
### Save the draft to the emails thread
Define the post-hook function that automatically saves the AI-generated response as a draft reply in the original email thread:
```python theme={"dark"}
def save_draft(self):
"""Create a draft response to the original message"""
span = get_run_context().current_span()
original_message = span.input['input_email']
response_text = span.output.collect_text()
try:
# Extract sender email from the original message
from_header = original_message.get('from', '')
if '<' in from_header and '>' in from_header:
# Extract email from "Name " format
sender_email = from_header.split('<')[1].split('>')[0].strip()
else:
# Assume the entire from header is the email
sender_email = from_header.strip()
# Create response subject
original_subject = original_message.get('subject', '')
if not original_subject.startswith('Re:'):
response_subject = f"Re: {original_subject}"
else:
response_subject = original_subject
# Create draft message
draft_message = self._create_message(sender_email, response_subject, response_text)
# Create the draft with thread ID to link it to the original conversation
draft_body = {
'message': {
'raw': draft_message
}
}
# Include thread ID if available to link the draft to the original conversation
if original_message.get('thread_id'):
draft_body['message']['threadId'] = original_message['thread_id']
draft = self.gmail_service.users().drafts().create(
userId='me',
body=draft_body
).execute()
print(f"Draft ID: {draft['id']}")
print(f"To: {sender_email}")
print(f"Subject: {response_subject}")
return draft
except Exception as e:
print(f"Error creating draft: {e}")
return None
```
## Key Features
* **Real-time Processing**: Constantly polls new emails
* **Intelligent Responses**: Uses AI to generate contextually appropriate draft replies
* **History Tracking**: Maintains state to avoid processing duplicate emails or generated drafts
## Troubleshooting
* **Missing scopes**: Verify all required Gmail API scopes are included
* **OAuth issues**: Check that `credentials.json` and `token.json` files exist and are valid
* **Service Account issues**: Verify domain-wide delegation is configured and `DELEGATED_USER` has correct email
* **Rate limits**: Gmail API has daily quotas. Check your usage.
* **Permission denied**: For Service Accounts, ensure domain-wide delegation is authorized with correct Client ID
* **Import errors**: Ensure all required packages are installed
* **File errors**: Check that your credentials file exists and is properly formatted
# Slack Bot
Source: https://docs.timbal.ai/examples/guides/slack-bot
Build a Slack bot that responds in real time via webhooks, with an AI agent and threaded conversations
This guide shows how to create a Slack bot using Timbal that can:
* Respond to messages in real-time via webhooks
* Process user requests intelligently with AI agents
* Handle threaded conversations
## Setup Requirements
Install the Slack SDK:
```bash theme={"dark"}
pip install slack-sdk
```
For all available Slack API methods and capabilities, see the [Slack API Methods Reference](https://docs.slack.dev/reference/methods).
1. Go to [Slack API](https://api.slack.com/apps) and click **Create New App**
2. Choose **From scratch**
3. Enter your app name and select your workspace
1. Go to **OAuth & Permissions**
2. Add the following **Bot Token Scopes** based on Timbal handlers you'll use:
```
# Message Operations
chat:write # Send messages
chat:write.public # Send messages to channels bot isn't in
im:write # Send direct messages
# Channel & Conversation Operations
channels:read # View basic channel information
channels:history # Get messages from channels
groups:read # View basic private channel information
im:read # View basic direct message information
im:history # Get direct message history
# App Mentions & Reactions
app_mentions:read # Receive app mention events
reactions:read # Read message reactions
# File Operations
files:read # Download files from Slack
files:write # Upload files to Slack
```
After adding or modifying scopes, you'll need to reinstall the app to your workspace for the changes to take effect.
1. In **OAuth & Permissions**, click **Install to Workspace**
2. Copy the **Bot User OAuth Token** (starts with `xoxb-`)
3. Set as environment variable: `SLACK_BOT_TOKEN`
1. Go to **App Home** in your Slack app settings
2. Under **Show Tabs**, enable **Home Tab**
3. Check **Allow users to send Slash commands and messages from the messages tab**
4. This enables users to send direct messages to your bot
1. In your Slack workspace, start a direct message with your bot
2. Click on the bot's name at the top of the chat
3. In the profile panel, find the **Member ID** (format: `U12345678`)
4. Set this as environment variable: `SLACK_BOT_USER_ID=U12345678`
The Bot User ID is required to prevent infinite loops by filtering out the bot's own messages.
## Implementation
### Setting up the Slack Bot Configuration
First, set up the Slack configuration with your bot's user ID and authentication token.
Create a `.env` file in your project root:
```bash .env theme={"dark"}
SLACK_BOT_USER_ID=U12345678 # Your actual bot user ID
SLACK_BOT_TOKEN=xoxb-your-bot-token-here # Your bot token
```
Then configure your Python code:
```python theme={"dark"}
import os
from dotenv import load_dotenv
from slack_sdk import WebClient
# Load environment variables from .env
load_dotenv()
SLACK_BOT_USER_ID = os.getenv("SLACK_BOT_USER_ID")
client = WebClient(token=os.getenv("SLACK_BOT_TOKEN"))
```
### Creating the Agent
Create the Timbal agent with pre and post hooks for Slack integration:
```python theme={"dark"}
from timbal import Agent
agent = Agent(
name="SlackAgent",
system_prompt="You are a tech support AI assistant. Help users with their technical questions clearly and concisely.",
model="openai/gpt-4.1-mini",
pre_hook=pre_hook,
post_hook=post_hook,
)
```
### Pre-hook: Processing Incoming Messages
The pre-hook handles incoming Slack webhook events and extracts relevant information.
**Parameter Name Flexibility**: The `_webhook` parameter name is completely custom. You can use any name you prefer:
* `await agent(slack_data=body).collect()`
* `await agent(webhook_payload=body).collect()`
* `await agent(event_data=body).collect()`
Just ensure the same name is used consistently in both the agent call and the pre\_hook check.
```python theme={"dark"}
from timbal.state import get_run_context
from timbal.errors import bail
async def pre_hook():
"""Process incoming Slack webhook events before agent execution."""
span = get_run_context().current_span()
# Check if this is a Slack webhook event
# Note: "_webhook" matches the parameter name used when calling agent(_webhook=body)
if "_webhook" not in span.input:
return # Not triggered by Slack
slack_event = span.input["_webhook"]["event"]
# Ignore bot's own messages to prevent infinite loops
if slack_event.get("user") == SLACK_BOT_USER_ID:
raise bail()
# Extract and validate message text
text = slack_event.get("text", "")
if not text:
raise bail()
# Store Slack context for response
span.slack_channel = slack_event["channel"]
span.slack_thread_ts = slack_event.get("thread_ts") # Reply in same thread
span.input["prompt"] = text
```
### Post-hook: Sending Responses
The post-hook sends the agent's response back to Slack:
```python theme={"dark"}
from timbal.state import get_run_context
def post_hook():
"""Send agent response back to Slack channel."""
span = get_run_context().current_span()
# Only process Slack webhook events
if "_webhook" not in span.input:
return
# Extract response context
slack_channel = span.slack_channel
slack_thread_ts = span.slack_thread_ts
reply = span.output.collect_text()
# Send response if not empty
if reply.strip():
client.chat_postMessage(
channel=slack_channel,
text=reply,
thread_ts=slack_thread_ts,
)
```
### Complete Example
Here's the full implementation ready to use:
```python agent.py theme={"dark"}
import os
from dotenv import load_dotenv
from slack_sdk import WebClient
from timbal import Agent
from timbal.errors import bail
from timbal.state import get_run_context
# Load environment variables from .env
load_dotenv()
SLACK_BOT_USER_ID = os.getenv("SLACK_BOT_USER_ID")
client = WebClient(token=os.getenv("SLACK_BOT_TOKEN"))
async def pre_hook():
"""Process incoming Slack webhook events before agent execution."""
span = get_run_context().current_span()
# Only process Slack webhook events
# Note: "_webhook" must match the parameter name used when calling agent(_webhook=body)
if "_webhook" not in span.input:
return
slack_event = span.input["_webhook"]["event"]
# Prevent infinite loops by ignoring bot's own messages
if slack_event.get("user") == SLACK_BOT_USER_ID:
raise bail()
# Extract message text
text = slack_event.get("text", "")
if not text:
raise bail()
# Store context for response
span.slack_channel = slack_event["channel"]
span.slack_thread_ts = slack_event.get("thread_ts")
span.input["prompt"] = text
def post_hook():
"""Send agent response back to Slack."""
span = get_run_context().current_span()
if "_webhook" not in span.input:
return
# Get response and send to Slack
reply = span.output.collect_text()
slack_channel = span.slack_channel
slack_thread_ts = span.slack_thread_ts
if reply.strip():
client.chat_postMessage(
channel=slack_channel,
text=reply,
thread_ts=slack_thread_ts,
)
# Create the Slack agent
agent = Agent(
name="SlackAgent",
system_prompt="You are a tech support AI assistant. Help users with their technical questions clearly and concisely.",
model="openai/gpt-4.1-mini",
pre_hook=pre_hook,
post_hook=post_hook,
)
```
## Running the Agent
Timbal provides built-in ngrok integration for easy local development. To use this feature:
1. Create an account at [ngrok.com](https://ngrok.com/)
2. Install ngrok on your system and the Python package:
```bash theme={"dark"}
pip install pyngrok
```
3. Configure your authtoken:
```bash theme={"dark"}
ngrok config add-authtoken YOUR_AUTHTOKEN
```
Create a simple FastAPI server to handle Slack's URL verification and webhook events:
```python server.py theme={"dark"}
import uvicorn
from dotenv import load_dotenv
from fastapi import FastAPI, Request
from pyngrok import ngrok
# Import our Slack agent to handle webhook events
from agent import agent
load_dotenv()
app = FastAPI()
@app.post("/")
async def slack_events(request: Request):
body = await request.json()
if body.get("type") == "url_verification":
return {"challenge": body.get("challenge")}
# Process actual Slack events here
if body.get("type") == "event_callback":
await agent(_webhook=body).collect() # _webhook is a custom parameter name you define
return {"status": "ok"}
if __name__ == "__main__":
port = 8000
public_url = ngrok.connect(port, "http")
print(f"Public URL: {public_url}")
uvicorn.run(app, host="0.0.0.0", port=port)
```
1. Run the script to start your Slack bot:
```bash theme={"dark"}
python server.py
```
Or using uv (recommended):
```bash theme={"dark"}
uv run server.py
```
2. Copy the ngrok public URL displayed in the terminal (e.g., `https://abc123.ngrok-free.app`)
1. Go to **Event Subscriptions** and toggle **Enable Events**
2. Set **Request URL** to our webhook endpoint (e.g., `https://abc123.ngrok-free.app/`)
3. Subscribe to **Bot Events** - these determine what events Slack will send to your bot:
```
message.channels # Bot receives messages in public channels it's added to
message.groups # Bot receives messages in private channels it's added to
message.im # Bot receives direct messages sent to it
app_mention # Bot receives messages that mention it (@botname)
reaction_added # Bot receives events when reactions are added to messages
```
Your Slack bot is now ready! 🚀 Start a direct message with your bot or mention it in a channel to begin chatting with your AI assistant.
## Key Features
* **Real-time Responses**: Instantly processes Slack messages via webhooks
* **Thread Support**: Maintains conversation context in Slack threads
* **Loop Prevention**: Automatically ignores bot's own messages
* **Flexible AI**: Uses any LLM model supported by Timbal
* **Easy Integration**: Minimal setup with existing Slack handlers
## Troubleshooting
* Check webhook URL is publicly accessible and returns 200 status
* Ensure bot has correct permissions in Slack workspace
* Confirm `SLACK_BOT_USER_ID` matches your bot's actual user ID
* Test webhook endpoint manually with curl or Postman
* Double-check `SLACK_BOT_USER_ID` is correct (found in Slack app settings)
* Ensure pre-hook properly filters bot messages with `bail()`
* Verify webhook events aren't being duplicated
* Check that bot doesn't respond to its own message events
* Check that OpenAI API key is valid and has sufficient credits
* Verify agent model name is correct (e.g., `openai/gpt-4o-mini`)
* Ensure pre-hook is setting `span.input["prompt"]` correctly
* Add logging to debug webhook payload structure
* Ensure all required packages are installed (`slack-sdk fastapi uvicorn pyngrok python-dotenv`)
* Verify ngrok is installed and configured with authtoken
* Make sure port 8000 is not already in use by another process
* Ensure Slack Request URL matches your POST endpoint route with "/" (e.g., `https://abc123.ngrok-free.app/`)
# WhatsApp Assistant
Source: https://docs.timbal.ai/examples/guides/whatsapp-assistant
Build a WhatsApp assistant with webhooks, conversation persistence (Knowledge Base or JSONL), and support for text, audio, images, and documents
This guide shows how to create a WhatsApp assistant with Timbal that can:
* Respond to messages in real time via webhooks
* Persist conversations using Timbal Knowledge Bases or JSONL
* Handle text, audio, images, and documents
## Setup Requirements
Go to the Meta console and open your app's WhatsApp section: [Meta WhatsApp App Dashboard](https://developers.facebook.com/apps/).
1. Set the **Callback URL** to your endpoint, e.g., `https://yourdomain.com/`
2. Set a **Verify Token** (keep it secret)
3. In your app code, use the same token (see `WHATSAPP_VERIFY_TOKEN` in the example below)
4. Click **Verify and Save**
In the WhatsApp product settings, subscribe to message-related events (e.g., messages, statuses).
Create a `.env` file and set:
```bash .env theme={"dark"}
# Timbal platform (if using KB persistence)
ORG_ID=your_org_id
KB_ID=your_kb_id
# WhatsApp API
WHATSAPP_ACCESS_TOKEN=your_long_lived_token
WHATSAPP_PHONE_NUMBER_ID=your_phone_number_id
# Storage mode for this template: JSONL | timbal
WHATSAPP_STORAGE_MODE=timbal
# Webhook verification
WHATSAPP_VERIFY_TOKEN=your_verify_token
# Server
HOST=0.0.0.0
PORT=4343
# Optional: enable ngrok tunnel
ENABLE_NGROK=true
```
The included `whatsapp_tools.send_whatsapp_message` currently references `WHATSAPP_ACCESS_TOKEN`/`WHATSAPP_PHONE_NUMBER_ID`. Ensure these are set. If you see mismatched names in your local template (e.g., `WHATSAPP_TOKEN`, `PHONE_NUMBER_ID`), set both pairs to be safe.
## Minimal Webhook App (FastAPI)
Use a simple FastAPI app to receive the webhook and pass the full payload to the agent. The agent will process everything in `pre_hook` and send responses in `post_hook`.
```python app.py theme={"dark"}
from fastapi import FastAPI, Request, Response
from agent import agent
import os
import uvicorn
from dotenv import load_dotenv
load_dotenv()
app = FastAPI()
VERIFY_TOKEN = os.getenv("WHATSAPP_VERIFY_TOKEN")
@app.get("/")
async def verify(request: Request):
token = request.query_params.get("hub.verify_token")
challenge = request.query_params.get("hub.challenge")
if token == VERIFY_TOKEN and challenge:
return Response(content=challenge, media_type="text/plain")
return Response(content="Verification failed", status_code=403)
@app.post("/")
async def webhook(request: Request):
_webhook = await request.json()
await agent(_webhook=_webhook).collect()
return {"status": "ok"}
def _maybe_start_ngrok(port: int) -> None:
enable = os.getenv("ENABLE_NGORK", "false")
if enable != "true":
return
try:
from pyngrok import ngrok
public_url = ngrok.connect(port, "http")
print(f"ngrok public url: {public_url}")
except Exception as e:
print(f"Failed to start ngrok: {e}")
if __name__ == "__main__":
host = os.getenv("HOST", "0.0.0.0")
try:
port = int(os.getenv("PORT", "4343"))
except Exception:
port = 4343
_maybe_start_ngrok(port)
uvicorn.run("app:app", host=host, port=port, log_level="info")
```
Expose locally (optional):
```bash theme={"dark"}
ngrok http 4343
```
Then set the ngrok HTTPS URL as the Callback URL in the Meta console.
## Creating the Agent
```python agent.py theme={"dark"}
import os
import json
from timbal import Agent
from timbal.state import get_run_context
from whatsapp_tools import send_whatsapp_message
def _value(payload: dict) -> dict | None:
try:
return payload["entry"][0]["changes"][0]["value"]
except Exception:
return None
def _prompt_from_message(msg: dict) -> str:
t = msg.get("type")
if t == "text":
return (msg.get("text") or {}).get("body", "").strip()
if t == "image":
mid = (msg.get("image") or {}).get("id")
cap = (msg.get("image") or {}).get("caption", "")
return f"[image:{mid}] {cap}".strip()
if t == "audio":
mid = (msg.get("audio") or {}).get("id")
return f"[audio:{mid}]"
if t == "document":
fn = (msg.get("document") or {}).get("filename", "document")
return f"[document:{fn}]"
return f"[{t or 'unknown'}]"
def _append_jsonl(path: str, rec: dict) -> None:
try:
with open(path, "a", encoding="utf-8") as f:
f.write(json.dumps(rec, ensure_ascii=False) + "\n")
except Exception:
pass
def _recent_history(path: str, user_phone: str, limit: int = 10) -> list[str]:
out = []
try:
if not os.path.exists(path):
return []
with open(path, "r", encoding="utf-8") as f:
rows = [json.loads(l) for l in f if l.strip()]
for r in reversed(rows):
if r.get("user_phone") == user_phone:
try:
payload = json.loads(r.get("message") or "{}")
out.append(f"{r.get('direction')}: {payload.get('content','')}")
except Exception:
continue
if len(out) >= limit:
break
return list(reversed(out))
except Exception:
return []
def pre_hook():
"""Build professional prompt from WhatsApp metadata, message, and optional history."""
span = get_run_context().current_span()
payload = span.input.get("_webhook")
if not payload:
bail("No payload found in webhook")
value = payload["entry"][0]["changes"][0]["value"]
if not value:
bail("No value found in webhook")
messages = value.get("messages", [])
if not messages:
bail("No messages found in webhook")
contacts = value.get("contacts", [])
profile = contacts[0].get("profile", {})
name = profile.get("name", "")
msg = messages[0]
from_number = msg.get("from")
message_id = msg.get("id")
phone_number_id = value.get("metadata").get("phone_number_id")
prompt = _prompt_from_message(msg)
msg_type = msg.get("type")
rec = {
"id": msg.get("id"),
"user_phone": from_number,
"direction": "inbound",
"message_type": msg_type,
"message": json.dumps({"type": msg_type, "content": prompt}, ensure_ascii=False),
"timestamp": msg.get("timestamp"),
"conversation_id": f"{phone_number_id}_{from_number}",
"user_name": name,
}
_append_jsonl("whatsapp_messages.jsonl", rec)
span.input["prompt"] = _recent_history("whatsapp_messages.jsonl", user_phone=from_number, limit=10)
span.input["whatsapp_from_number"] = from_number
span.input["name"] = name
```
The `pre_hook` runs before invoking the agent. It extracts metadata from the webhook, builds a prompt from the incoming message, persists the inbound event, and sets values in `span.input` for the agent to use during inference.
```python agent.py theme={"dark"}
def post_hook():
"""Send the agent response via WhatsApp and persist outbound to JSONL."""
span = get_run_context().current_span()
from_number = span.input.get("whatsapp_from_number")
if not from_number:
return
try:
response_text = (span.output.collect_text() or "").strip()
except Exception:
return
if not response_text:
return
try:
send_whatsapp_message(to=from_number, message=response_text)
except Exception:
pass
_append_jsonl("whatsapp_messages.jsonl", {
"id": f"outbound_{from_number}_{datetime.now().strftime('%Y%m%d_%H%M%S')}",
"user_phone": from_number,
"direction": "outbound",
"message_type": "agent",
"message": json.dumps({"type": "text", "content": response_text}, ensure_ascii=False),
"timestamp": datetime.now().isoformat(),
"conversation_id": "",
"user_name": span.input.get("name", ""),
})
```
The `post_hook` runs after obtaining the agent's response. It sends the text via WhatsApp and persists the outbound event to JSONL.
```python agent.py theme={"dark"}
agent = Agent(
name="WhatsAppAgent",
description="A minimal WhatsApp agent for Timbal",
model="openai/gpt-4.1-mini",
pre_hook=pre_hook,
post_hook=post_hook,
)
```
### Optional: system prompt example
```text theme={"dark"}
title: "WHATSAPP AGENT from {{YOUR_BRAND}}",
description: "You are a WhatsApp agent from {{YOUR_BRAND}}. Your goal is to help users find the perfect vehicle and manage all their automotive needs.",
## WhatsApp message format
You are responding to messages in WhatsApp. Use WhatsApp's text formatting syntax when appropriate:
Text format:
- *text* = bold (for important information)
- _text_ = italic (for clarifications or soft emphasis)
- ~text~ = strikethrough (for corrections or obsolete information)
- `text` = monospace (for code, commands or technical text)
- Use triple backticks for code blocks (for longer fragments)
- > text = quote (for references or highlighted information)
- Important: Never use [View here](url) or . Send the direct link instead; WhatsApp does not render these markdown forms.
Important rules:
- Symbols must be attached to the text (no spaces)
- Every opening symbol must have its closing symbol
- You can combine formats: *_text_* for bold+italic
- Use sparingly; only when it adds value
- For lists use bullets (•), numbers (1.), or hyphens (-)
```
### Optional: Timbal KB persistence and history
If you prefer using Timbal Knowledge Bases instead of JSONL, you can import and query records with the platform helpers. These are async functions; adapt their usage to your environment accordingly.
```python theme={"dark"}
from timbal.platform.kbs.tables import import_records, query
ORG_ID = int(os.getenv("ORG_ID"))
KB_ID = int(os.getenv("KB_ID"))
async def save_record_to_kb(record: dict) -> None:
await import_records(ORG_ID, KB_ID, "whatsapp_messages", [record])
async def fetch_recent_history_from_kb(user_phone: str, limit: int = 10) -> list[str]:
rows = await query(ORG_ID, KB_ID, sql=f"""
SELECT direction, message
FROM whatsapp_messages
WHERE user_phone = '{user_phone}'
ORDER BY timestamp DESC
LIMIT {limit}
""")
lines: list[str] = []
for r in reversed(rows or []):
try:
payload = json.loads(r.get("message") or "{}")
lines.append(f"{r.get('direction')}: {payload.get('content','')}")
except Exception:
continue
return lines
```
## Media Handling Notes
* **Images**: The webhook includes an image `id`. Use the Graph API to fetch the media URL or content using your `WHATSAPP_ACCESS_TOKEN`. Provide the URL or a `File` object to the agent if needed.
* **Audio**: The webhook includes an audio `id`. You can fetch and transcribe audio, then pass the transcript as the prompt.
* **Documents**: You receive basic metadata (e.g., filename). Decide whether to fetch and analyze content or treat as a reference.
Keep the `pre_hook` minimal: extract the message type, create a concise prompt, and persist the event.
## Complete implementation
For a full, end-to-end reference (webhook app, helpers, and utilities), see the official WhatsApp examples in the Timbal repository:
* [timbal/examples/whatsapp](https://github.com/timbal-ai/timbal/tree/main/examples/whatsapp)
## Possible Improvements
Each WhatsApp webhook creates a separate agent instance. Persist every inbound message immediately (JSONL or KB), then wait a short debounce window. After that window, if storage shows a newer message for the same user, bail this run so the most recent instance answers with full context. No in-memory buffer is required.
* **Why it helps**: avoids replying to partial context when the user is still typing or sending media.
* **Trade-offs**: adds small latency; choose a short window (e.g., 800–1500 ms).
* **Example approach (JSONL)**:
```python theme={"dark"}
import os, json, time
DEBOUNCE_MS = int(os.getenv("WHATSAPP_DEBOUNCE_MS", "1200"))
def _message_ts(msg: dict) -> int:
try:
return int(msg.get("timestamp") or 0)
except Exception:
return 0
def _newer_exists_in_jsonl(path: str, user_phone: str, after_ts: int) -> bool:
try:
if not os.path.exists(path):
return False
with open(path, "r", encoding="utf-8") as f:
rows = [json.loads(l) for l in f if l.strip()]
for rec in reversed(rows):
if rec.get("user_phone") != user_phone:
continue
if rec.get("direction") != "inbound":
continue
try:
ts = int(rec.get("timestamp") or 0)
except Exception:
continue
if ts > after_ts:
return True
return False
except Exception:
return False
def pre_hook():
span = get_run_context().current_span()
payload = span.input.get("_webhook") or {}
value = (payload.get("entry") or [{}])[0].get("changes", [{}])[0].get("value", {})
msg = (value.get("messages") or [None])[0] or {}
from_number = msg.get("from", "")
current_ts = _message_ts(msg)
# 1) Persist inbound immediately (example: JSONL)
_append_jsonl("whatsapp_messages.jsonl", {
"id": msg.get("id"),
"user_phone": from_number,
"direction": "inbound",
"message_type": msg.get("type"),
"message": json.dumps({"type": msg.get("type"), "content": _prompt_from_message(msg)}, ensure_ascii=False),
"timestamp": str(current_ts),
"conversation_id": (value.get("metadata") or {}).get("phone_number_id", ""),
"user_name": (value.get("contacts") or [{}])[0].get("profile", {}).get("name", ""),
})
# 2. Wait a small window, then check for a newer message
asyncio.sleep(DEBOUNCE_MS / 1000.0)
if _newer_exists_in_jsonl("whatsapp_messages.jsonl", from_number, current_ts):
bail("Newer message detected; aborting this run")
# 3. Proceed with prompt building from recent history
span.input["prompt"] = _recent_history("whatsapp_messages.jsonl", user_phone=from_number, limit=10)
```
Use the WhatsApp Cloud API to mark messages as read. This improves UX and clears unread indicators for the user.
```python theme={"dark"}
import os, requests
def mark_as_read(message_id: str) -> None:
token = os.getenv("WHATSAPP_ACCESS_TOKEN")
phone_id = os.getenv("WHATSAPP_PHONE_NUMBER_ID")
try:
requests.post(
f"https://graph.facebook.com/v21.0/{phone_id}/messages",
headers={"Authorization": f"Bearer {token}"},
json={
"messaging_product": "whatsapp",
"status": "read",
"message_id": message_id,
},
timeout=10,
)
except Exception:
pass
```
Call `mark_as_read(message_id)` right after parsing the webhook.
You can mark the message as read and briefly show a typing indicator while the agent thinks or uses tools. Optionally send a short acknowledgement, then follow up with the final answer.
```python theme={"dark"}
import os, httpx
def send_typing_and_read(message_id: str) -> dict:
token = os.getenv("WHATSAPP_ACCESS_TOKEN")
phone_id = os.getenv("WHATSAPP_PHONE_NUMBER_ID")
if not token or not phone_id:
return {}
url = f"https://graph.facebook.com/v19.0/{phone_id}/messages"
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
}
with httpx.Client() as client:
# Mark as read
client.post(
url,
headers=headers,
json={
"messaging_product": "whatsapp",
"status": "read",
"message_id": message_id,
},
timeout=10,
)
# Show typing indicator briefly
resp = client.post(
url,
headers=headers,
json={
"messaging_product": "whatsapp",
"typing_indicator": {"type": "typing_on"},
},
timeout=10,
)
return resp.json()
```
* Call this right after parsing the webhook and before long operations.
* Keep interim notifications minimal to avoid noise.
Persist metadata for images, audio, videos, and documents so the agent can reuse them as context later (e.g., to re-attach an image or analyze it again).
Suggested fields to store: `id`, `user_phone`, `direction`, `message_type`, `media_id`, `media_url` (or object store key), `mime_type`, `caption`, `timestamp`.
```python theme={"dark"}
async def save_media_example(record: dict) -> None:
# Merge this idea with your existing save flow
await save_record_to_kb({
**record,
"message_type": "image", # or "video", "audio", "document"
# e.g. values fetched via Graph API media URL endpoint
"media_id": record.get("media_id"),
"media_url": record.get("media_url"),
"mime_type": record.get("mime_type"),
"caption": record.get("caption", ""),
})
```
Later, query recent media by `user_phone` or `conversation_id` and inject URLs or file handles into the agent context as needed.
## Running
```bash theme={"dark"}
python app.py
```
Expose with ngrok, then verify the webhook in the Meta console. New messages should invoke your agent and send responses via WhatsApp.
## Troubleshooting
* **Verify token**: Ensure the value in Meta matches `VERIFY_TOKEN` in your app
* **Public URL**: Your callback must be publicly reachable (use ngrok)
* **HTTPS**: Required in production
* **Tokens**: Confirm `WHATSAPP_ACCESS_TOKEN` and `WHATSAPP_PHONE_NUMBER_ID` are set
* **Phone ID**: Ensure your app uses the correct phone number ID
* **Permissions**: Check token permissions in Access Token Debugger
* Check `ORG_ID` and `KB_ID` if using Timbal KB
* If using JSONL, verify the JSONL file path and write permissions
# Overview
Source: https://docs.timbal.ai/examples/index
Complete guides and building patterns for Timbal AI agents
Learn to build powerful AI agents through comprehensive guides and focused examples that demonstrate real-world applications and core patterns.
## Guides
Production-ready guides that walk you through building real-world scenarios from start to finish. These comprehensive tutorials show you how to integrate Timbal with popular services and deploy to production from start to finish.
AI-powered Gmail integration that triages emails and autonomously drafts intelligent responses
Build a Slack bot that responds in real time via webhooks, with an AI agent and threaded conversations
Build a WhatsApp assistant with webhooks, conversation persistence (Knowledge Base or JSONL), and support for text, audio, images, and documents
## Agents
Quick-start examples showcasing Timbal's agent patterns. Learn specific techniques like tool integration, streaming responses, state management, and multi-agent orchestration through bite-sized implementations.
## Workflows
Step-by-step workflow examples that chain multiple operations together. Perfect for complex business processes that require explicit control flow and data transformation between steps.
# Conditional Routing
Source: https://docs.timbal.ai/examples/workflows/conditional-routing
Process data differently based on validation results
A data processing pipeline that validates input and routes to different handlers based on the result. Only the matching branch executes.
## Workflow
```python pipeline.py theme={"dark"}
from timbal import Workflow
from timbal.state import get_run_context
def validate_data(data: dict) -> str:
"""Validate data and return status."""
if not data.get("email"):
return "invalid"
if data.get("age", 0) < 18:
return "minor"
return "valid"
def process_adult(data: dict) -> dict:
"""Process adult user data."""
return {
"status": "processed",
"user": data["email"],
"tier": "adult",
}
def process_minor(data: dict) -> dict:
"""Process minor user data with restrictions."""
return {
"status": "processed",
"user": data["email"],
"tier": "minor",
"restricted": True,
}
def log_invalid(data: dict) -> dict:
"""Log invalid data."""
return {
"status": "rejected",
"reason": "missing email",
}
pipeline = (
Workflow(name="user_processor")
.step(validate_data, data={"email": "user@example.com", "age": 25})
.step(process_adult,
data=lambda: get_run_context().step_span("validate_data").input.get("data"),
when=lambda: get_run_context().step_span("validate_data").output == "valid")
.step(process_minor,
data=lambda: get_run_context().step_span("validate_data").input.get("data"),
when=lambda: get_run_context().step_span("validate_data").output == "minor")
.step(log_invalid,
data=lambda: get_run_context().step_span("validate_data").input.get("data"),
when=lambda: get_run_context().step_span("validate_data").output == "invalid")
)
```
## How It Works
```
validate_data ──┬─→ process_adult (if "valid")
├─→ process_minor (if "minor")
└─→ log_invalid (if "invalid")
```
1. **`validate_data`** — checks email and age, returns "valid", "minor", or "invalid"
2. **`process_adult`**, **`process_minor`**, or **`log_invalid`** — only the matching branch runs (`when` condition)
## Running
```python theme={"dark"}
result = await pipeline().collect()
print(result.output)
```
The output will be:
```json theme={"dark"}
{
"status": "processed",
"user": "user@example.com",
"tier": "adult"
}
```
# Parallel Fan-Out
Source: https://docs.timbal.ai/examples/workflows/parallel-fan-out
Fetch data from multiple sources in parallel and merge the results
Multiple independent data sources are fetched concurrently, then a final step merges everything into a single report.
## Workflow
```python pipeline.py theme={"dark"}
from timbal import Workflow
from timbal.state import get_run_context
def fetch_users() -> list[dict]:
"""Fetch user data."""
return [
{"id": 1, "name": "Alice", "role": "admin"},
{"id": 2, "name": "Bob", "role": "editor"},
]
def fetch_orders() -> list[dict]:
"""Fetch order data."""
return [
{"id": 101, "user_id": 1, "total": 59.99},
{"id": 102, "user_id": 2, "total": 29.99},
{"id": 103, "user_id": 1, "total": 14.50},
]
def fetch_inventory() -> list[dict]:
"""Fetch inventory data."""
return [
{"product": "Widget A", "stock": 150},
{"product": "Widget B", "stock": 0},
]
def build_report(users: list, orders: list, inventory: list) -> dict:
"""Merge all data into a summary report."""
total_revenue = sum(o["total"] for o in orders)
out_of_stock = [i["product"] for i in inventory if i["stock"] == 0]
return {
"total_users": len(users),
"total_orders": len(orders),
"revenue": total_revenue,
"out_of_stock": out_of_stock,
}
pipeline = (
Workflow(name="dashboard_report")
.step(fetch_users)
.step(fetch_orders)
.step(fetch_inventory)
.step(build_report,
users=lambda: get_run_context().step_span("fetch_users").output,
orders=lambda: get_run_context().step_span("fetch_orders").output,
inventory=lambda: get_run_context().step_span("fetch_inventory").output,
)
)
```
## How It Works
```
fetch_users ─┐
fetch_orders ─┼─→ build_report
fetch_inventory─┘
```
1. **`fetch_users`**, **`fetch_orders`**, and **`fetch_inventory`** run in **parallel** — no dependencies between them
2. **`build_report`** waits for all three to complete, then merges their outputs
## Running
```python theme={"dark"}
result = await pipeline().collect()
print(result.output)
```
The output will be:
```json theme={"dark"}
{
"total_users": 2,
"total_orders": 3,
"revenue": 104.48,
"out_of_stock": ["Widget B"]
}
```
# Sequential Steps
Source: https://docs.timbal.ai/examples/workflows/sequential-steps
Chain multiple processing steps with data passing between them
A document processing pipeline that fetches content, extracts key information, summarizes it with an LLM, and formats the final output. Each step depends on the previous one's output.
## Workflow
```python pipeline.py theme={"dark"}
from timbal import Agent, Workflow
from timbal.state import get_run_context
def fetch_content(url: str) -> str:
"""Fetch raw content from a URL."""
import urllib.request
with urllib.request.urlopen(url) as response:
return response.read().decode("utf-8")
def extract_metadata(html: str) -> dict:
"""Extract title and text from HTML content."""
import re
title_match = re.search(r"(.*?)", html)
text = re.sub(r"<[^>]+>", " ", html)
text = re.sub(r"\s+", " ", text).strip()
return {
"title": title_match.group(1) if title_match else "Untitled",
"text": text[:5000],
}
summarizer = Agent(
name="summarizer",
model="openai/gpt-5-mini",
system_prompt="Summarize the given text in 3 bullet points. Be concise."
)
def format_report(title: str, summary: str) -> str:
"""Format the final report."""
return f"# {title}\n\n{summary}"
pipeline = (
Workflow(name="document_pipeline")
.step(fetch_content, url="https://example.com")
.step(extract_metadata,
html=lambda: get_run_context().step_span("fetch_content").output)
.step(summarizer,
prompt=lambda: get_run_context().step_span("extract_metadata").output["text"])
.step(format_report,
title=lambda: get_run_context().step_span("extract_metadata").output["title"],
summary=lambda: get_run_context().step_span("summarizer").output.collect_text())
)
```
## How It Works
```
fetch_content → extract_metadata → summarizer → format_report
```
1. **`fetch_content`** — fetches raw HTML from the URL
2. **`extract_metadata`** — parses title and text from the HTML (waits for `fetch_content`)
3. **`summarizer`** — LLM summarizes the extracted text (waits for `extract_metadata`)
4. **`format_report`** — combines title and summary into a report (waits for both `extract_metadata` and `summarizer`)
Each lambda creates an automatic dependency. No step runs until its dependencies are resolved.
## Running
```python theme={"dark"}
result = await pipeline().collect()
print(result.output)
```
The output will be similar to:
```
# Example Domain
- The page serves as an illustrative example for documentation purposes
- It can be used freely without permission or coordination
- More information is available through IANA at the referenced link
```
# Workflow Composition
Source: https://docs.timbal.ai/examples/workflows/workflow-composition
Nest workflows inside other workflows for modular pipelines
A reporting pipeline that uses an inner data workflow as a step. The inner workflow handles data fetching and cleaning, while the outer workflow generates and delivers the report.
## Workflow
```python pipeline.py theme={"dark"}
from timbal import Agent, Workflow
from timbal.state import get_run_context
import json
def fetch_sales() -> list[dict]:
"""Fetch raw sales data."""
return [
{"product": "Widget A", "amount": 120.00, "region": "EU"},
{"product": "Widget B", "amount": 85.50, "region": "US"},
{"product": "Widget A", "amount": 200.00, "region": "US"},
{"product": "Widget C", "amount": 45.00, "region": "EU"},
]
def aggregate(sales: list) -> dict:
"""Aggregate sales by region."""
by_region = {}
for sale in sales:
region = sale["region"]
by_region[region] = by_region.get(region, 0) + sale["amount"]
return {
"by_region": by_region,
"total": sum(sale["amount"] for sale in sales),
}
# Inner workflow: data preparation
data_pipeline = (
Workflow(name="data_pipeline")
.step(fetch_sales)
.step(aggregate,
sales=lambda: get_run_context().step_span("fetch_sales").output)
)
analyst = Agent(
name="analyst",
model="openai/gpt-5-mini",
system_prompt="You are a data analyst. Given sales data, write a short summary with key insights."
)
def format_email(analysis: str, data: dict) -> str:
"""Format the final email."""
return f"Subject: Weekly Sales Report\n\n{analysis}\n\nRaw data:\n{json.dumps(data, indent=2)}"
# Outer workflow: uses data_pipeline as a step
report_pipeline = (
Workflow(name="report_pipeline")
.step(data_pipeline)
.step(analyst,
prompt=lambda: json.dumps(get_run_context().step_span("data_pipeline").output))
.step(format_email,
analysis=lambda: get_run_context().step_span("analyst").output.collect_text(),
data=lambda: get_run_context().step_span("data_pipeline").output)
)
```
## How It Works
```
┌─ data_pipeline ─────────┐
│ fetch_sales → aggregate │
└────────────────────┘
│
├─→ analyst
│ │
└────┴─→ format_email
```
1. **`data_pipeline`** (inner workflow) — fetches and aggregates sales data as a single step
2. **`analyst`** — LLM analyzes the aggregated data (waits for `data_pipeline`)
3. **`format_email`** — combines the LLM analysis and raw data into a report (waits for both `data_pipeline` and `analyst`)
The inner workflow's final output (`aggregate`'s result) becomes the step output accessible by the outer workflow.
## Running
```python theme={"dark"}
result = await report_pipeline().collect()
print(result.output)
```
The output will be similar to:
```
Subject: Weekly Sales Report
The total sales for this period amount to $450.50. The US region leads with
$285.50 (63%), while the EU region accounts for $165.00 (37%). Widget A is the
top performer with $320.00 across both regions.
Raw data:
{
"by_region": {
"EU": 165.0,
"US": 285.5
},
"total": 450.5
}
```
# Approval gates
Source: https://docs.timbal.ai/human-in-the-loop/approval-gates
Declarative permission checks that fire before a runnable executes — gate irreversible actions for human approval
Approval gates (`requires_approval`) are declarative permission checks that fire **before** a runnable executes. Use them for irreversible actions: refunds, deploys, account deletions, outbound emails, anything that costs money or moves data. When a gate fires, zero handler code runs until a human decides.
## Quick start
Mark a tool as approval-required, stream events, and resume with a decision keyed by `approval_id`.
```python theme={"dark"}
from timbal import Agent, Tool
from timbal.types.events import ApprovalEvent
def refund_customer(amount: int) -> str:
return f"refunded ${amount}"
refund = Tool(
handler=refund_customer,
requires_approval=lambda amount: amount > 100,
approval_prompt=lambda amount: f"Approve refunding ${amount}?",
)
agent = Agent(
name="support_agent",
model="openai/gpt-5",
tools=[refund],
)
approval_id = None
async for event in agent(prompt="Refund $250"):
if isinstance(event, ApprovalEvent):
approval_id = event.approval_id
result = await agent(
prompt="Refund $250",
resume={approval_id: True},
).collect()
```
When a gate fires, the run ends with `status.code == "cancelled"` and `status.reason == "approval_required"`. The `ApprovalEvent` carries:
* `approval_id` — stable id used to resolve the gate
* `runnable_path`, `runnable_name`, `runnable_type` — what was about to execute
* `tool_call_id` — when the gate fired inside an agent tool call, the LLM `tool_use` id that triggered it, so the frontend can pin the approval card to the exact tool\_use block in the transcript. `None` for direct calls.
* `input` — validated handler input (after redaction, if configured)
* `input_schema` — JSON Schema of the handler params, so the UI can render a typed form
* `prompt`, `description` — human-facing strings
* `t0` — Unix-ms timestamp of when approval was requested (useful for SLA timers)
## Configuring approval gates
`requires_approval` accepts `True`, `False`, or a callable that receives the same kwargs as the handler and returns `bool`. `approval_prompt` and `approval_description` accept strings or callables and surface in the `ApprovalEvent` so the human reviewer has context.
```python theme={"dark"}
high_risk_deploy = Tool(
handler=deploy_handler,
requires_approval=lambda env, **_: env == "production",
approval_prompt=lambda env, **_: f"Deploy to {env}?",
approval_description="Deploys ship traffic to the listed environment.",
)
```
If `requires_approval` or `approval_prompt` raises, the runnable does **not** silently approve nor execute. It ends with `status.code == "error"` and `status.reason == "approval_policy_error"`, distinct from handler errors so dashboards can surface policy bugs separately.
## Resolutions: approve, deny, audit
Pass either a bare `bool` (`True`/`False`) or an `ApprovalResolution` for richer audit fields:
```python theme={"dark"}
from timbal.types.approval import ApprovalResolution
result = await agent(
prompt="Refund $250",
resume={
approval_id: ApprovalResolution(
approved=False,
reason="Refund exceeds policy limit.",
approver_id="user_42",
comment="Customer is outside the refund window.",
)
},
).collect()
```
Audit fields are first-class (not free-form metadata) and persist under `span.metadata["approval"]["resolution"]`:
* `approver_id` — who decided
* `comment` — free-form reasoning
* `decided_at` — Unix-ms timestamp; defaults to construction time. Pass an explicit value if you replay decisions and need idempotency
* `metadata` — org-specific extras
### Edit on approve (`override_input`)
A reviewer often wants to approve *with a tweak* (fix a typo'd recipient, lower an amount) rather than reject and round-trip back to the model. Set `override_input` on the resolution: the keys are merged over the originally-proposed input (override wins), **re-validated through the handler's params model**, and the handler runs with the corrected values.
```python theme={"dark"}
from timbal.types.approval import ApprovalResolution
result = await agent(
prompt="email the customer",
parent_id=paused_run_id,
resume={
approval_id: ApprovalResolution(
approved=True,
override_input={"to": "correct@example.com"}, # fix just this field
)
},
).collect()
```
Only the listed keys change; everything else from the proposal is kept. The edit is audited under `span.metadata["approval"]["resolution"]["override_input"]`, and the effective (redacted) input the handler ran with is recorded at `span.metadata["approval"]["effective_input"]`. `override_input` is ignored on denial. Re-validation means a bad override (wrong type, missing required field) fails the run with a normal validation error instead of silently running garbage.
### Tool denial vs Agent denial
Behavior differs based on who initiated the call:
* **Direct tool call** — denial returns `status.code == "cancelled"` and `status.reason == "approval_denied"`. The handler does not run.
* **Tool called by an Agent** — denial is converted into a `ToolResultContent` so the model can see "this tool was denied" and choose another path (apologize, escalate, try an alternative). The agent does not crash.
## Time-limited decisions
Bound decisions with `expires_at` (Unix-ms). Expired resolutions are ignored at gate time and the gate emits a fresh `ApprovalEvent` with `metadata["approval"]["expired"] == True`:
```python theme={"dark"}
import time
from timbal.types.approval import ApprovalResolution
decision = ApprovalResolution(
approved=True,
approver_id="user_42",
comment="Approved from the support console.",
expires_at=int(time.time() * 1000) + 60_000, # valid for 60s
)
```
This is useful when an operator stamps a decision in a UI but a worker doesn't pick it up immediately. Stale decisions force a fresh re-review instead of silently going through.
## Redacting approval input
Approval input is shown to humans and written to traces. If a gated runnable receives secrets or PII, redact the public approval snapshot.
The simple form lists keys to mask with `"***"`:
```python theme={"dark"}
rotate_key = Tool(
handler=rotate_key_impl,
requires_approval=True,
approval_prompt="Rotate this API key?",
approval_redact_keys=["api_key", "password"],
)
```
For custom logic, use `approval_redactor`. It receives a copy of the validated input dict and returns the public snapshot:
```python theme={"dark"}
rotate_key = Tool(
handler=rotate_key_impl,
requires_approval=True,
approval_redactor=lambda input: {
**input,
"api_key": "***",
"customer_email": input["customer_email"].split("@")[0] + "@***",
},
)
```
The redacted snapshot is used everywhere the input would otherwise be visible: `ApprovalEvent.input`, `span.input` while the gate is pending, `span.metadata["approval"]["input"]`, `OutputEvent.metadata["pending_approvals"]`, and any exporter (OTel, Langfuse, etc.). The handler still receives the **original unredacted input** when the approval is resumed.
A redactor that raises or returns a non-dict falls back to a placeholder so secrets never leak through a buggy redactor.
## `approval_id` semantics
The `approval_id` is derived from `(runnable_path, validated_input)`. The same path + input shares one decision, so a single resolution survives retries of the same call (stream resumes, transient failures, agent loops re-asking for the same tool). Treat the id as **opaque**: the derivation is an internal contract and may change across SDK versions, so don't persist ids across deploys.
For irreversible operations (money movement, destructive deletes) where every call must require a fresh decision, include a unique value in the input so each call derives a distinct `approval_id`, typically an `idempotency_key: str` parameter with `default_params={"idempotency_key": lambda: str(uuid4())}`. Timbal evaluates the callable per-invocation.
## Approvals in workflows
Workflow steps follow the same rules as tools. `requires_approval` is a `Runnable` config, so wrap the function in a `Tool` (or use any `Runnable`) before adding it as a step. When a gated step fires, the workflow run cancels with `approval_required` and emits one `ApprovalEvent` per pending gate. **Independent gates fire in parallel**, so you don't need to ping-pong one approval at a time.
```python theme={"dark"}
from timbal import Tool, Workflow
deploy_prod = Tool(
name="deploy_prod",
handler=deploy_prod_impl,
requires_approval=True,
approval_prompt="Promote to prod?",
)
workflow = (
Workflow(name="release_pipeline")
.step(deploy_staging)
.step(deploy_prod)
.step(announce, depends_on=["deploy_prod"])
)
```
When you resume with `resume={...}`, only the steps you decided on advance. Other pending gates remain pending and re-emit on the next call. This means you can approve a subset, observe what runs, and decide on the rest later.
## See also
* [Suspend & interaction tools](/human-in-the-loop/suspend) — ask the user for arbitrary input mid-run
* [Resuming a paused run](/human-in-the-loop/resuming) — durable cross-process resume and cancellation
* [Observability](/human-in-the-loop/observability) — status reasons and usage counters
# Client integration (HTTP)
Source: https://docs.timbal.ai/human-in-the-loop/client-integration
The full contract for a frontend talking to the /stream SSE endpoint: pause, render, resume
This is the full contract for a frontend talking to `timbal serve` (the `/stream` SSE endpoint). Two phases: **pause** (server → client) and **resume** (client → server). The example uses `ask_user`; an approval gate is identical except the pause event is an `APPROVAL` event and you resume with `true`/`false`.
## 1. Start the run
`POST /stream` with the runnable's params:
```json theme={"dark"}
{ "prompt": "set up my database" }
```
The response is an SSE stream (`text/event-stream`), one `data:` line per event. When the agent calls `ask_user`, the client receives an **`INTERACTION`** event:
```json theme={"dark"}
{
"type": "INTERACTION",
"run_id": "06a22e62b85d7b558000db9caf2790b2",
"path": "assistant.ask_user",
"t0": 1780672043531,
"interaction_id": "bfd231bf5760d201b31382e168c339d4",
"kind": "ask_user",
"runnable_name": "ask_user",
"runnable_type": "Tool",
"tool_call_id": "toolu_01abc...",
"payload": { "question": "Which database should I use?", "options": ["postgres", "mysql", "sqlite"] },
"response_schema": { "type": "string", "enum": ["postgres", "mysql", "sqlite"] }
}
```
`tool_call_id` (when present) is the LLM tool\_use id, so you can pin the prompt next to the matching message in the transcript. `response_schema` (when the tool declared one) is the JSON Schema the answer must satisfy; validate the user's input against it before resuming. An `APPROVAL` event additionally carries `input_schema` (the handler's params schema, for rendering a typed form).
Immediately followed by the final **`OUTPUT`** event for the run, which marks it as paused:
```json theme={"dark"}
{
"type": "OUTPUT",
"run_id": "06a22e62b85d7b558000db9caf2790b2",
"path": "assistant",
"status": { "code": "cancelled", "reason": "input_required", "message": "Input required to resume." },
"output": {
"suspension_id": "bfd231bf5760d201b31382e168c339d4",
"status": "input_required",
"kind": "ask_user",
"payload": { "question": "Which database should I use?", "options": ["postgres", "mysql", "sqlite"] }
}
}
```
**What the frontend does:**
* Render UI from the `INTERACTION` event's `kind` + `payload` (here: a question with three option buttons). For an `APPROVAL` event, render `prompt` + `input` and offer Approve/Deny.
* Stash `run_id` and `interaction_id` (or `approval_id`).
* A run is paused (not finished) whenever the terminal `OUTPUT` has `status.reason` of `input_required` or `approval_required`.
## 2. Resume with the answer
`POST /stream` again, echoing the original params plus two keys: `parent_id` (the paused `run_id`) and `resume` (a map of `id → value`):
```json theme={"dark"}
{
"prompt": "set up my database",
"parent_id": "06a22e62b85d7b558000db9caf2790b2",
"resume": { "bfd231bf5760d201b31382e168c339d4": "postgres" }
}
```
The run replays, `ask_user` returns `"postgres"`, and the stream ends with a normal success `OUTPUT`:
```json theme={"dark"}
{
"type": "OUTPUT",
"path": "assistant",
"status": { "code": "success", "reason": "end_turn" },
"output": { "role": "assistant", "content": [{ "type": "text", "text": "Great, using postgres." }] }
}
```
That's the whole loop. If a turn opens **multiple** pauses (parallel tools/steps, even a mix of approvals and interactions), the client receives one event per pause and sends every answer in a single `resume` map: `{ "": ..., "": true }`. The id you send back is always the one you received: approval ids resume with `true`/`false`, interaction ids with the value the handler asked for.
## Cancelling over HTTP
To abort instead of answering (user closed the dialog, navigated away), resume the id with the tagged cancel object, the JSON equivalent of `Cancel`:
```json theme={"dark"}
{
"prompt": "set up my database",
"parent_id": "06a22e62b85d7b558000db9caf2790b2",
"resume": { "bfd231bf5760d201b31382e168c339d4": { "type": "timbal.cancel", "reason": "user closed the dialog" } }
}
```
The run ends with a terminal `OUTPUT` of `status.reason == "cancelled"` (not `input_required`/`approval_denied`), and nothing is sent back to the model. An edit-on-approve over HTTP is the same idea on an approval id: `{ "": { "approved": true, "override_input": { "to": "fixed@example.com" } } }`.
## See also
* [Resuming a paused run](/human-in-the-loop/resuming) — the in-process and durable-provider story
* [Suspend & interaction tools](/human-in-the-loop/suspend) — `InteractionEvent` and response schemas
* [Approval gates](/human-in-the-loop/approval-gates) — the `APPROVAL` event and its fields
# Overview
Source: https://docs.timbal.ai/human-in-the-loop/index
Pause any run for a human: gate irreversible actions for approval, or stop mid-handler to ask the user a question, then resume on the same durable rails
These patterns apply to any **Runnable**: [Agents](/agents), [Workflows](/workflows) steps, and standalone [Tools](/agents/tools). Timbal pauses a run for a human in two ways, and resumes both through one channel:
* **[Approval gates](/human-in-the-loop/approval-gates)** (`requires_approval`) are declarative permission checks that fire **before** a runnable executes. Use them for irreversible actions: refunds, deploys, account deletions, outbound emails, anything that costs money or moves data.
* **[`suspend()`](/human-in-the-loop/suspend)** is a control-flow primitive a handler calls from the **inside** to ask the user for arbitrary input (a clarifying question, a picked option, a confirmation). Use it to build interaction tools like `ask_user` or `confirm`.
Both pause the run, persist it, emit a structured event, and resume when you call the runnable again with `resume={...}`. They share the same durable rails (`parent_id` + tracing provider). The only real difference is *when* they pause and *what* they resume with.
| | `requires_approval` (gate) | `suspend()` (interaction) |
| ------------- | ------------------------------------------------------- | --------------------------------------------------- |
| Intent | "may I do this?" | "tell me / give me" |
| Pauses | before the handler runs | inside the handler |
| Handler ran? | no (zero code) | yes, up to the `suspend()` call (re-runs on resume) |
| Resumes with | `bool` / `ApprovalResolution` | arbitrary value |
| Applied | declaratively on any runnable (even ones you don't own) | by calling `suspend()` in the handler |
| Event | `ApprovalEvent` | `InteractionEvent` |
| Cancel reason | `approval_required` | `input_required` |
`resume={id: value}` is the single channel for continuing **any** paused run. For an approval gate the value is a decision (`True`/`False` or an `ApprovalResolution`); for a `suspend()` call it's the arbitrary value the handler asked for. A [`Cancel`](/human-in-the-loop/resuming#cancelling-instead-of-answering) value works for either and aborts the whole run. The id-spaces are disjoint, so one `resume` dict can settle a mix of pending approvals and suspensions in a single call. Unrecognized ids are ignored with a warning.
## On this page group
* **[Approval gates](/human-in-the-loop/approval-gates)** — `requires_approval`, resolutions, edit-on-approve, redaction, time limits, workflows
* **[Suspend & interaction tools](/human-in-the-loop/suspend)** — `suspend()`, writing `ask_user`/`ask_user_multi`/`confirm`, `InteractionEvent`, response schemas
* **[Resuming a paused run](/human-in-the-loop/resuming)** — cancel, durable cross-process resume, duplicate-worker protection, enumerating pending
* **[Client integration (HTTP)](/human-in-the-loop/client-integration)** — the `/stream` wire contract a frontend talks to
* **[Observability](/human-in-the-loop/observability)** — status reasons, usage counters, common patterns
# Observability
Source: https://docs.timbal.ai/human-in-the-loop/observability
Status reasons, usage counters, and common patterns for paused runs
## Status reasons
When the run cancels, `OutputEvent.status.reason` carries one of:
* `approval_required` — a gate emitted an `ApprovalEvent` and is waiting on a decision
* `approval_denied` — a denying resolution was consumed (direct call only; agents convert this to a tool result)
* `approval_already_claimed` — durable claim said another worker already resumed this gate
* `approval_policy_error` — a `requires_approval` / `approval_prompt` callable raised
* `input_required` — a `suspend()` call emitted an `InteractionEvent` and is waiting on a value
* `cancelled` — a `Cancel` was supplied on resume (approval or suspension); the whole run aborted and nothing was fed back to the model
## Usage counters
`OutputEvent.usage` records pause-lifecycle counters so you can plot them in dashboards:
* `approvals:required` — a gate emitted an `ApprovalEvent`
* `approvals:approved` — a valid approved resolution was consumed
* `approvals:denied` — a valid denied resolution was consumed
* `approvals:expired` — an expired resolution was ignored and the gate re-emitted
* `approvals:cancelled` — a `Cancel` aborted the run at a gate
* `suspends:required` — a `suspend()` call paused the run
These propagate through the usage merge tree just like token counts, so a parent agent run aggregates pause counts from every nested tool/workflow gate or suspension.
## Common patterns
1. Configure a durable provider (`JsonlTracingProvider` / `SqliteTracingProvider` / `PlatformTracingProvider`).
2. UI calls `agent(prompt=...)`, captures `ApprovalEvent` (or polls the trace for `pending_approvals()`), shows reviewer the prompt + redacted input.
3. Reviewer clicks Approve/Deny. UI persists `(approval_id, ApprovalResolution)` to a queue.
4. Worker pulls the message and calls `agent(prompt=..., parent_id=run_id, resume={approval_id: resolution})`.
5. If `result.status.reason == "approval_already_claimed"`, no-op. Otherwise, the handler executed exactly once.
Give the agent the `ask_user` tool. When it's blocked it calls `ask_user`, the run ends `input_required` with an `InteractionEvent`, your UI renders the question, and you resume with `resume={interaction_id: answer}`. Keep everything before the `suspend()` call idempotent: the handler re-runs from the top on resume.
The agent may call multiple gated/suspending tools in parallel. Each pause emits its own event and the run cancels once they all settle. Collect every id, present them as a checklist, and resume with the full dict in one call: approvals and interactions can be mixed freely.
The default `approval_id` is stable across retries of the same `(path, input)`. To require a fresh decision per invocation, include a per-call unique value in the input, typically an `idempotency_key=str(uuid4())`, so each call derives a distinct id.
Use `approval_redact_keys=["api_key", "password"]` for the simple case. Use `approval_redactor=lambda input: {...}` for partial masking (e.g. mask the local-part of an email but keep the domain). The handler still receives the unredacted input.
The model proposed a slightly-wrong argument (a typo'd email, an amount that should be lower). Instead of denying and round-tripping, approve with `ApprovalResolution(approved=True, override_input={"to": "fixed@example.com"})`. The override merges over the proposal, re-validates through the params model, and the handler runs with the corrected input. Over HTTP send `{"approved": true, "override_input": {...}}`.
When the user closes the dialog or navigates away rather than deciding, resume with `Cancel(reason=...)` (HTTP: `{"type": "timbal.cancel", "reason": "..."}`) keyed to any pending id. The run ends `status.reason == "cancelled"` and nothing is fed back to the model, distinct from a denial, which the agent would see and react to.
## See also
* [Approval-Required Tools example](/examples/agents/approval-required-tools) — runnable end-to-end snippet
* [Events & Streaming](/core-concepts/events) — the event stream
* [Tracing](/core-concepts/tracing) — how runs are persisted and replayed
* [Context & State Management](/core-concepts/context) — `RunContext` internals
* [Tools](/agents/tools) — tool configuration reference
# Resuming a paused run
Source: https://docs.timbal.ai/human-in-the-loop/resuming
Resume any paused run (gate or suspension) with parent_id and resume={...}, in-process or across workers
You resume any paused run, gate or suspension, by calling the runnable again with `parent_id` (the paused run id) and `resume={...}`. In-process this works out of the box; across processes you need a durable provider; and you can enumerate what's pending either way.
## Cancelling instead of answering
A human won't always say yes or no. Sometimes they close the dialog, hit Escape, or abandon the task. That's a **cancel**, and it's different from a deny:
* **Deny / decline** continues the run. An approval `approved=False` is fed back to the model as a tool result so the agent can apologize or try another path; a `suspend()` decline is just a value your handler interprets.
* **Cancel** aborts the *entire* run. Nothing is fed back to the model.
Resume with `Cancel` (works on either an approval or a suspension id) to abort:
```python theme={"dark"}
from timbal.types.approval import Cancel
result = await agent(
prompt="...",
parent_id=paused_run_id,
resume={pending_id: Cancel(reason="user closed the dialog")},
).collect()
# result.status.code == "cancelled"
# result.status.reason == "cancelled" # distinct from approval_denied / input_required
# result.status.message == "user closed the dialog"
```
The handler never runs. The cancel reason lands on the cancelled span/status so it's queryable in traces, and a `Cancel` keyed to any pending id in a batch tears down the whole run.
## Across processes (durable providers)
Everything above works in-process by default. For "pause in a UI now, resume in a worker later" (the real human-in-the-loop shape) switch off the default `InMemoryTracingProvider` (which only resumes within the same Python process) to a durable provider. This applies identically to approval gates and `suspend()`.
```python theme={"dark"}
from pathlib import Path
from timbal import Agent
from timbal.state.tracing.providers import JsonlTracingProvider
provider = JsonlTracingProvider.configured(_path=Path("traces.jsonl"))
agent = Agent(
name="support_agent",
model="openai/gpt-5",
tools=[refund],
tracing_provider=provider,
)
```
`JsonlTracingProvider` writes one record per run and uses a sidecar lock file (`traces.jsonl.approval_claims.json` + `.lock` via `fcntl`) for cross-process approval claims. Good for local dev and single-host deployments. Not recommended for high-throughput production: `_store()` rewrites the file on each run.
```python theme={"dark"}
from pathlib import Path
from timbal.state.tracing.providers import SqliteTracingProvider
provider = SqliteTracingProvider.configured(_path=Path("traces.db"))
```
Same API as JSONL but uses a SQLite database with row-level locking. Better for higher write rates on a single host.
When `TIMBAL_API_KEY` and a project subject are set, runs auto-select `PlatformTracingProvider`. Paused runs replicate across workers without extra config.
To resume from a different process, pass the original run id as `parent_id`:
```python theme={"dark"}
result = await agent(
prompt="Refund $250",
parent_id=paused_run_id,
resume={approval_id: True},
).collect()
```
Timbal loads the parent trace (input messages, pending gates/suspensions, prior tool calls) before executing the resolution, so the runnable sees exactly the state it was paused at.
## Memory compaction on resume
If the agent has `memory_compaction` configured, a resume turn is treated like a continuation of the paused turn, not a fresh one. The loaded memory ends with the gated/suspended `tool_use` that has **no `tool_result` yet** — that trailing block is exactly what Timbal re-executes to settle the pause without re-calling the model.
[Tool result offloading](/agents/memory-compaction#tool-result-offloading) applies on resume the same way it does on a fresh turn: when the gated tool finally runs after approval, an oversized result is reduced once before it enters memory.
Compaction is structure-aware about this:
* The pending `tool_use` is **never** compacted away. (A naive pass would treat it as an orphaned tool call, strip it, and force the model to re-plan — which is nondeterministic and would silently drop the human's decision.)
* The **history before** the pending call is still compacted, so resuming a long paused thread doesn't carry the full uncompacted transcript into the continuation LLM call and overflow the context window.
This holds for every built-in strategy (`compact_tool_results`, `keep_last_n_messages`, `keep_last_n_turns`, `summarize`) and any custom compactor. You don't configure anything extra — it just works on the same `resume=` call.
Compaction runs at turn boundaries **and** between iterations within a turn (mid-loop), so a single turn that makes many or large tool calls is compacted as it grows once utilization crosses `memory_compaction_ratio`. Mid-loop compaction always protects the most recent (unconsumed) assistant tool batch — the results the next model call must read — so it bounds context without sending the agent back to re-plan the same step.
In a `Workflow`, compaction is a per-step concern: a `Workflow` has no LLM context of its own, and an `Agent` used as a step runs with isolated context (no cross-turn memory) but still mid-loop-compacts a long tool loop inside that step. Step outputs passed between steps are not "context" and are never compacted.
## Duplicate worker protection
When multiple workers consume the same queue, two of them might race to resume the same `(parent_id, approval_id)`. Timbal **claims** the pair before executing the resolution. The first claimer wins; later duplicates stop before handler execution with `status.reason == "approval_already_claimed"`.
```python theme={"dark"}
result = await agent(
prompt="Refund $250",
parent_id=paused_run_id,
resume={approval_id: True},
).collect()
if result.status.reason == "approval_already_claimed":
# Another worker already executed this approval. Safe to no-op.
return
```
This protection is implemented by `JsonlTracingProvider` and `SqliteTracingProvider` out of the box. **Custom providers must override `claim_approval(parent_id, approval_id, run_id)`** to get the same durable-lock behavior; the base class default is a no-op.
## Enumerating what's pending
When a run cancels and the runnable had multiple concurrent calls, each pause emits its own event. There are two ergonomic ways to enumerate them, and the same shape applies to both approvals and interactions.
**During the stream**, capture every event:
```python theme={"dark"}
pending_approvals, pending_interactions = [], []
async for event in agent(prompt="..."):
if isinstance(event, ApprovalEvent):
pending_approvals.append(event)
if isinstance(event, InteractionEvent):
pending_interactions.append(event)
```
**After `.collect()`**, read the lists the collector attaches to `OutputEvent.metadata` (the `status` only references the first pause):
```python theme={"dark"}
result = await agent(prompt="...").collect()
if result.status.reason == "approval_required":
for entry in result.metadata["pending_approvals"]:
print(entry["approval_id"], entry["runnable_path"], entry["prompt"], entry["input"])
if result.status.reason == "input_required":
for entry in result.metadata["pending_interactions"]:
print(entry["interaction_id"], entry["kind"], entry["payload"])
```
Resume by passing **all** the decisions/answers you want to settle in one call:
```python theme={"dark"}
resume = {entry["approval_id"]: True for entry in result.metadata["pending_approvals"]}
result = await agent(prompt="...", parent_id=paused_run_id, resume=resume).collect()
```
For server-side traversal of a loaded trace (e.g. building a review queue from durable storage), `RunContext.pending_approvals()` and `RunContext.pending_interactions()` walk `RunContext._trace` directly. They tolerate both live `RunStatus` and dict-after-reload shapes, so they work against in-memory, JSONL, SQLite, and platform traces. Approval entries use the **redacted** input snapshot, never the raw secrets.
`metadata["pending_*"]` is added by `.collect()`. Over the HTTP server (which streams **raw** events) the frontend reads the `APPROVAL` / `INTERACTION` events directly instead. See [Client integration (HTTP)](/human-in-the-loop/client-integration).
## See also
* [Approval gates](/human-in-the-loop/approval-gates) — declarative gates for irreversible actions
* [Suspend & interaction tools](/human-in-the-loop/suspend) — ask the user for arbitrary input
* [Client integration (HTTP)](/human-in-the-loop/client-integration) — the `/stream` wire contract
# Suspend & interaction tools
Source: https://docs.timbal.ai/human-in-the-loop/suspend
Pause a run from inside a handler to ask the user for arbitrary input, then resume with their answer
`suspend()` pauses a run from inside a handler and waits for an externally-supplied value. It's the general form of the [approval gate](/human-in-the-loop/approval-gates) (an approval resumes with a `bool`, a suspension resumes with anything) and it's how you build tools like `ask_user`, `confirm`, or `pick_option` that let an agent explicitly stop and hand control back to the user mid-loop.
## How it works
1. A handler calls `suspend(payload, kind=...)`.
2. The run ends with status `cancelled` / reason `input_required` and emits an `InteractionEvent` carrying `payload`.
3. The frontend renders the payload (keyed by `kind`) and collects a response.
4. You resume by calling the runnable again with the original run as `parent_id` and `resume={interaction_id: value}`. The handler re-executes and `suspend()` returns `value`.
```python theme={"dark"}
from timbal import Agent
from timbal.tools import ask_user
from timbal.types.events import InteractionEvent, OutputEvent
agent = Agent(name="assistant", model="openai/gpt-5", tools=[ask_user])
# First call — the model asks a question and the run suspends.
pending = []
final = None
async for event in agent(prompt="set up my database"):
if isinstance(event, InteractionEvent):
pending.append(event)
if isinstance(event, OutputEvent) and event.path == "assistant":
final = event
# pending[0].kind == "ask_user"
# pending[0].payload == {"question": "Which database?", "options": [...]}
# Resume with the user's answer.
result = await agent(
prompt="set up my database",
parent_id=final.run_id,
resume={pending[0].interaction_id: "postgres"},
).collect()
```
## Writing an interaction tool
A tool is any handler that calls `suspend()`:
```python theme={"dark"}
from timbal import suspend
def ask_user(question: str, options: list[str] | None = None) -> str:
"""Ask the user a clarifying question. Use ONLY when blocked."""
return suspend({"question": question, "options": options}, kind="ask_user")
def confirm(action: str) -> bool:
return bool(suspend({"action": action}, kind="confirm"))
```
`suspend` is exported at the top level (`from timbal import suspend`), and the ready-made `ask_user` / `ask_user_multi` / `confirm` tools ship in `timbal.tools` (`from timbal.tools import ask_user, ask_user_multi, confirm`).
Because the handler **re-executes from the top** on resume, `suspend()` must come before any non-idempotent side-effect in the handler (same constraint as LangGraph's `interrupt()`). Put the `suspend()` call first, or make everything before it idempotent. This is exactly why irreversible actions belong behind an [approval gate](/human-in-the-loop/approval-gates) instead: the gate guarantees zero handler code runs until approved.
## `InteractionEvent`
```python theme={"dark"}
class InteractionEvent(BaseEvent):
type: Literal["INTERACTION"]
t0: int # Unix-ms timestamp at which the run suspended
interaction_id: str # pass back as the resume key
kind: str # frontend renderer discriminator (e.g. "ask_user")
runnable_path: str
runnable_name: str
runnable_type: str
tool_call_id: str | None # originating LLM tool_use id (None for direct calls)
payload: dict # what the frontend renders
response_schema: dict | None # optional JSON Schema for the resume value
```
The `interaction_id` is a deterministic hash of the handler path + payload, stable across the pause/resume round-trip, so the client always sends back the id it received. `tool_call_id` lets the client correlate the interaction with the exact tool\_use block in the chat transcript when the agent triggered it.
### Declaring a response schema
Pass `response_schema` to `suspend()` to tell the client what shape the resume value must be. It rides on the `InteractionEvent` so the frontend can validate the user's input *before* resuming, with no extra round-trip to discover a bad value.
```python theme={"dark"}
from timbal import suspend
def ask_age() -> int:
"""Ask the user for their age."""
return suspend(
{"question": "How old are you?"},
kind="ask_user",
response_schema={"type": "integer", "minimum": 0, "maximum": 130},
)
```
The schema is advisory metadata for the client. `None` (the default) means any value is accepted.
`response_schema` is **not enforced server-side**. Timbal hands the resume value to the handler verbatim and does not validate it against the schema. Treat the schema as a contract for *your* client to validate against before resuming; if you need server-side guarantees, validate inside the handler after `suspend()` returns.
## See also
* [Approval gates](/human-in-the-loop/approval-gates) — gate irreversible actions before any handler code runs
* [Resuming a paused run](/human-in-the-loop/resuming) — durable cross-process resume and cancellation
* [Client integration (HTTP)](/human-in-the-loop/client-integration) — the `/stream` wire contract
# Introduction
Source: https://docs.timbal.ai/index
Simple, performant, battle-tested framework for building reliable AI applications
## What is Timbal?
Timbal is an open-source Python framework for building reliable AI applications. It gives you two primitives, **Agents** for autonomous reasoning and **Workflows** for explicit pipelines, plus everything you need to take them to production: a stable multi-provider model layer, memory, tracing, and evals.
There's no hidden magic. Under the hood it's async functions, Pydantic validation, and event-driven streaming. If you know `async`/`await`, you already know how Timbal works.
## Agents vs Workflows
**[Agents](/agents)** are autonomous execution units that orchestrate LLM calls and tool use. The model decides what to do.
Best for:
* Open-ended problem solving and multi-step reasoning
* Dynamic tool selection based on context
* Multi-turn conversations with memory
**[Workflows](/workflows)** are explicit, step-by-step pipelines. You define the steps, and the framework infers dependencies and runs independent steps concurrently. See the [workflows guide](/workflows) for control flow, branching, and composition.
Best for:
* Multi-stage data processing
* Predictable, deterministic execution paths
* Performance-critical work that benefits from parallelism
Both share the same interface and emit the same event stream, so you can compose them freely. An Agent can be a Workflow step, and a Workflow can be an Agent's tool.
## Why Timbal
* **The most performant agent framework.** In overhead benchmarks against LangGraph, CrewAI, the OpenAI Agents SDK, PydanticAI, and Agno (observability on both sides, faked LLMs to isolate framework cost), Timbal runs agent loops several times faster while using a fraction of the memory.
* **Simple and hackable.** The core framework is under 10k lines with no hidden magic, just async functions and Pydantic. You can read it, modify it, and fork it to fit your needs. The others are bloated with legacy, indirection, and abstraction that make doing the same nearly impossible.
* **One interface.** Agents, Workflows, and Tools share the same calling convention and event stream. Learn it once.
* **Provider-agnostic.** Swap models by changing a string. Built-in `FallbackModel` chains providers for automatic failover.
* **Production-shaped.** Refined in production before open-sourcing. Fast failure, clear errors, stable interfaces.
* **Stable in a chaotic ecosystem.** Providers ship breaking changes monthly. Timbal absorbs the churn so your code doesn't have to.
## Key Features
Persistent context with tool-result offloading and compaction for long conversations.
One interface across every provider, with fallback chains for failover.
A built-in tool library, your own functions, and any MCP server.
Get typed Pydantic models back instead of raw text.
Pause for approval or input, then resume across process restarts.
Reusable tool packages the agent loads on demand.
Every run produces a full span trace, exportable over OTLP.
Declarative YAML evaluation suite with built-in validators.
Run the full stack locally with `timbal start`, then ship it.
New here? Head to the [Quickstart](/quickstart) to build and run your first agent.
# Installation
Source: https://docs.timbal.ai/installation
Step-by-step instructions for setting up Timbal CLI and Python package
## Installation methods
The **Timbal CLI** provides a command-line interface for managing Timbal projects, including initialization, building, deploying, and other project management tasks.
Use `curl` to download and execute the installation script:
```bash theme={"dark"}
curl -LsSf https://timbal.ai/install.sh | sh
```
If your system doesn't have `curl`, you can use `wget`:
```bash theme={"dark"}
wget -qO- https://timbal.ai/install.sh | sh
```
Use PowerShell to download and execute the installation script:
```powershell theme={"dark"}
powershell -ExecutionPolicy ByPass -c "irm https://timbal.ai/install.ps1 | iex"
```
The installation script may be inspected before use:
```bash theme={"dark"}
curl -LsSf https://timbal.ai/install.sh | less
```
```powershell theme={"dark"}
irm https://timbal.ai/install.ps1 | less
```
Alternatively, the installer or binaries can be downloaded directly from [GitHub](https://github.com/timbal-ai/timbal/releases).
## Upgrading
To upgrade the CLI to the latest version, you can either run:
```bash theme={"dark"}
timbal upgrade
```
Or re-run the installation script:
```bash theme={"dark"}
curl -LsSf https://timbal.ai/install.sh | sh
```
```powershell theme={"dark"}
powershell -ExecutionPolicy ByPass -c "irm https://timbal.ai/install.ps1 | iex"
```
## Uninstallation
To remove the CLI, delete the binary and the `~/.timbal` data directory:
```bash theme={"dark"}
rm ~/.local/bin/timbal
rm -rf ~/.timbal
```
```powershell theme={"dark"}
Remove-Item "$env:USERPROFILE\.local\bin\timbal.exe"
Remove-Item -Recurse -Force "$env:USERPROFILE\.timbal"
```
***
## Python Package
If you want to use Timbal directly in your Python projects or integrate it into an existing codebase, you can install the Python package directly.
Before adding the package, ensure you have `python>=3.11` installed.
Using `pip`:
```bash theme={"dark"}
pip install timbal
```
Or with `uv`:
```bash theme={"dark"}
uv add timbal
```
Alternatively, if you want the latest changes and want to experiment with beta features, you can clone the repository and install from source:
```bash theme={"dark"}
git clone https://github.com/timbal-ai/timbal.git
cd timbal
pip install -e .
```
**Next Steps:** You're ready to create your first Timbal project! Continue to the [quickstart guide](./quickstart) to build your first AI agent.
# Claude
Source: https://docs.timbal.ai/mcp-integration/claude
Learn how to connect Timbal to Claude in the browser with OAuth or in Claude Code with a Timbal API Key via MCP.
Use **[OAuth on Claude.ai](#mcp-oauth)** in your browser, or a **[Timbal API Key with Claude Code](#mcp-api-key)** (the `claude` CLI).
## Claude.ai: OAuth
> ### Step 1: Open Claude.ai
1. In your browser, open [claude.ai](https://claude.ai) and sign in.
> ### Step 2: Access Connectors
1. In the left sidebar, open **Customize** → **Connectors**, or go directly to [claude.ai/customize/connectors](https://claude.ai/customize/connectors).
> ### Step 3: Add Timbal as a new MCP server
1. Click the **+** button (next to the search control), then choose **Add custom connector**.
2. Add the Timbal MCP URL (`https://api.timbal.ai/mcp`). For example, you can name the connector **`timbal-mcp`**.
3. When **timbal-mcp** appears in the list as a custom connector, click **Connect**.
4. A browser window will open to complete **Timbal authentication**.
5. Log in using your preferred method: GitHub, Google, Microsoft, or Timbal OAuth.
6. Accept the OAuth permissions when prompted and continue.
7. Return to Claude — you can use **Timbal tools** in chat.
Make sure to log in with your active Timbal account — the account registered on the [Timbal Platform](https://app.timbal.ai). Using a different account may prevent access to Timbal tools.
## Claude Code: API Key
> ### Step 1: Open Claude Code
1. Install [Claude Code](https://docs.claude.com/en/docs/claude-code/setup) (the `claude` CLI) on your machine if you have not already.
2. Open a terminal where you want to use Timbal tools with Claude.
> ### Step 2: Add Timbal as a new MCP server
1. Run the following, replacing `` with your Timbal API Key:
```bash theme={"dark"}
claude mcp add --transport http timbal-mcp https://api.timbal.ai/mcp \
--header "Authorization: Bearer "
```
2. If you already added `timbal-mcp` without a header, run `claude mcp remove timbal-mcp`, then run the command again.
You can choose another server name instead of `timbal-mcp`; it is only a label in your local Claude Code configuration.
See the [Authentication documentation](https://docs.timbal.ai/api-reference/authentication#authentication) for instructions on generating an API Key.
> ### Step 3: Start using Timbal in Claude Code
1. Run `claude` and work in your repository as usual.
2. Start a chat.
3. You can now use **Timbal tools** in your conversation. Claude Code will automatically leverage the MCP integration.
4. Approve tool runs when Claude Code prompts you, if your security settings require confirmation.
# Cursor
Source: https://docs.timbal.ai/mcp-integration/cursor
Learn how to connect Timbal to Cursor via MCP to access its features.
> ## Step 1: Open the Cursor interface
1. Launch the **Cursor** app on your system.
2. Log in with your account if required.
> ## Step 2: Access MCP settings
1. In the top menu, click **Settings** ( on the top-right) -> **Cursor Settings**.
2. Navigate to the **Tools & MCP** section.
3. This section allows you to **add, edit, or remove MCP connections**.
> ## Step 3: Add Timbal as a new MCP server
1. Click **Add Custom MCP**.
2. Add the Timbal MCP URL (`https://api.timbal.ai/mcp`) in `mcp.json`. For example, you can name it `"timbal-mcp"`:
```json title=mcp.json theme={"dark"}
{
"mcpServers": {
"timbal-mcp": {
"url": "https://api.timbal.ai/mcp",
// Use this to authenticate with a Timbal API Key instead of OAuth
// "headers": {
// "Authorization": "Bearer "
// }
}
}
}
```
3. Save the configuration.
### Option 1: Connect via OAuth
4. In the **Tools & MCP** section, check the **Installed MCP Servers** list — Timbal should appear there.
5. Click **Connect**.
6. A browser window will open to complete **Timbal authentication**.
7. Log in using your preferred method: GitHub, Google, Microsoft, or Timbal OAuth.
8. Accept the OAuth permissions when prompted and continue.
9. Return to Cursor — Timbal should now show a green indicator, and its tools will be listed.
Make sure to log in with your active Timbal account — the account registered on the [Timbal Platform](https://app.timbal.ai). Using a different account may prevent access to Timbal tools.
### Option 2: Connect using a Timbal Api Key
5. If you added a `"Authorization"` with `"TIMBAL_API_KEY"` in `mcp.json`, Cursor will automatically use it — no browser login is needed.
6. Check the **Installed MCPs** list — Timbal should appear in green, with its tools ready to use.
> ## Step 4: Start using Timbal in Cursor
1. Open the **Toggle Agents Sidebar** ( on top-right).
2. Click **New Agent** and start a chat.
3. You can now use **Timbal tools** in your conversation. Cursor will automatically leverage the MCP integration.
The first time you try to use a **Timbal tool** in Cursor, you’ll need to **accept the button in the chat** asking if you want to run Timbal MCP.
# Overview
Source: https://docs.timbal.ai/mcp-integration/index
Learn how to integrate Timbal with MCP on supported servers.
## What is MCP?
[Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) is an open standard that enables communication between AI models and external tools and services.
By exposing **Timbal as an MCP server**, AI-powered editors and applications can interact with Timbal’s functionality directly.
Looking for the other direction — connecting MCP servers **to your Timbal agents** as tool sources? See [MCP Servers](/agents/mcp).
## Connecting Timbal via MCP
To use Timbal with MCP, you first need to configure your editor or client to connect to Timbal’s MCP endpoint:
```
https://api.timbal.ai/mcp
```
Each editor or client handles MCP connections differently. The following sections inside **Connecting Timbal** provide detailed, step-by-step instructions for connecting Timbal to specific editors and tools.
Once the connection is configured, authentication is required to access Timbal’s features. An **active Timbal account** on the [Timbal Platform](https://app.timbal.ai) is needed.
**Authentication** can be performed using one of the following methods:
* **OAuth** (Google, Microsoft, GitHub, or Timbal OAuth)
* **API Key**
See the [Authentication documentation](https://docs.timbal.ai/api-reference/authentication#authentication) for instructions on generating an API Key.
# VS Code
Source: https://docs.timbal.ai/mcp-integration/vscode
Learn how to connect Timbal to VS Code via MCP and use its functionality inside the editor.
> ## Step 1: Open the VS Code App
1. Launch the **VS Code** app on your system.
2. Log in with your account if required.
> ## Step 2: Access MCP settings
1. Go to **Manage** button () at the bottom-left corner of the VS Code window.
2. Select **Profiles**.
3. In the Profiles window, go to Contents → **MCP Servers**.
4. Open the `mcp.json` configuration file.
> ## Step 3: Add Timbal as a new MCP server
1. Add the Timbal MCP URL (`https://api.timbal.ai/mcp`). For example, you can set the server name `"timbal-mcp"`:
```json title="mcp.json" theme={"dark"}
{
"servers": {
// After saving, the Start button and configuration will appear here
"timbal-mcp": {
"url": "https://api.timbal.ai/mcp",
"type": "http",
// Use this to authenticate with a Timbal API Key instead of OAuth
// "headers": {
// "Authorization": "Bearer "
// }
}
}
}
```
2. Save the file.
3. The **Start** button appears above the server entry (in this example, "timbal-mcp"). **Click** it to start the server.
### Option 1: Connect via OAuth
4. A browser window will open for authentication.
If the browser does not open automatically, you can start the connection manually from the MCP Servers panel.
5. Log in using your preferred OAuth method (GitHub, Google, Microsoft, or Timbal OAuth).
6. Accept the requested permissions.
7. Return to VS Code. In the `mcp.json` file, the status should change from **Start** to **Running**.
Make sure you log in with your active Timbal account (the one registered on the Timbal platform). Using a different account may prevent access to Timbal tools.
### Option 2: Connect using a Timbal API Key
4. If you add the `"Authorization"` header with your `"TIMBAL_API_KEY"` in `mcp.json`, VS Code will authenticate automatically.
5. No browser login is required.
6. In the `mcp.json` file, the status should display **Running** instead of **Start**.
> ## Step 4: Verify the MCP server is running
1. Open the left sidebar.
2. Go to **Extensions**.
3. At the bottom, under **MCP Servers – Installed**, you should see **Timbal MCP** listed.
4. Click the **Manage** button () to view and verify the configuration details.
> ## Step 5: Test the integration
1. Open a new chat.
2. You can now use **Timbal tools** in your conversation. VS Code will automatically leverage the MCP integration.
# Windsurf
Source: https://docs.timbal.ai/mcp-integration/windsurf
Step-by-step instructions to integrate Timbal with Windsurf through MCP.
> ## Step 1: Open the Windsurf interface
1. Launch the Windsurf app on your system.
2. Log in with your account if required.
> ## Step 2: Access MCP settings
1. In the top menu, click **Settings** ( on the top-right) -> **Windsurf Setting**s.
2. Navigate to the **Cascade** section → **MCP Servers**.
3. Click the **Open MCP Marketplace** link.
> ## Step 3: Add Timbal as a new MCP server
1. In the **Installed MCPs** list, click the **Settings** button ( positioned to the right of the title).
2. The file `mcp_config.json` will open.
3. Add the Timbal MCP URL (`https://api.timbal.ai/mcp`). For example, you can set the server name `"timbal-mcp"`:
```json title=mcp_config.json theme={"dark"}
{
"mcpServers": {
"timbal-mcp": {
"serverUrl": "https://api.timbal.ai/mcp",
// Use this to authenticate with a Timbal API Key instead of OAuth
// "headers": {
// "Authorization": ""
// }
}
}
}
```
4. Save the configuration.
### Option 1: Connect via OAuth
5. A browser window will open for authentication.
If the pop-up does not appear, you can manually click the **Connect** button next to Timbal MCP in the **MCP Marketplace**.
6. Log in using your preferred OAuth method: GitHub, Google, Microsoft, or Timbal OAuth.
Make sure to log in with your active Timbal account — the account registered on the [Timbal Platform](https://app.timbal.ai). Using a different account may prevent access to Timbal tools.
7. Accept the OAuth permissions.
8. Return to Windsurf — Timbal should now appear in green, with available tools listed inside.
### Option 2: Connect using a Timbal Api Key
5. If you added a `"Authorization"` with `"TIMBAL_API_KEY"` in `mcp_config.json`, Windsurf will automatically use it — no browser login is needed.
6. Check the **Installed MCPs** list — Timbal should appear in green, with its tools ready to use.
> ## Step 4: Start using Timbal in Windsurf
1. Open the **Toggle Cascade Side Bar** ( on top-right).
2. Start a chat.
3. You can now use **Timbal tools** in your conversation. Windsurf will leverage the MCP integration automatically.
# Zed
Source: https://docs.timbal.ai/mcp-integration/zed
Connect Timbal to Zed via MCP
> ## Step 1: Open the Zed interface
1. Launch the Zed app on your system.
2. Log in with your account if required.
> ## Step 2: Access MCP settings
1. Open the **Agent panel** ().
2. In the **Toggle Agent Menu** (the three points ...), click **Settings**.
3. In section Model Context Protocol (MCP) Servers click `+ Add Server`.
4. Select **Add Custom Server**.
> ## Step 3: Add Timbal as a new MCP server
1. A popup titled **Add MCP Server** will appear.
2. In the bottom-left corner, click **Configure Remote**.
3. Add the Timbal MCP URL (`https://api.timbal.ai/mcp`). For example, you can name the server `"timbal-mcp"`:
```json title="Add MCP Server" theme={"dark"}
{
"timbal-mcp": { // The name of Timbal MCP server
"url": "https://api.timbal.ai/mcp",
"headers": {
"Authorization": "Bearer "
}
}
}
```
Zed currently supports authentication using API Keys only. OAuth authentication is not supported for MCP connections in Zed.
4. Click Add Server to save the configuration.
> ## Step 4: Verify the connection
1. You can see in the right the Settings the MCP Servers the tools available in green.
2. Click the gear icon next to the server to view the list of available tools.
> ## Step 5: Start using Timbal in Zed
1. Open the **Agent panel**.
2. Start a new chat.
3. You can now use **Timbal tools** in your conversation. Zed will automatically leverage the MCP integration.
You will be prompted in the chat to confirm before running a Timbal MCP tool. Make sure to accept the request to allow the tool to execute.
# Anthropic
Source: https://docs.timbal.ai/models/anthropic
Claude Fable 5, Opus 5, Sonnet 5, and Haiku models with specs, pricing, and capabilities
Source: [Anthropic model docs](https://docs.anthropic.com/en/docs/about-claude/models) · [What's new in Opus 5](https://platform.claude.com/docs/en/about-claude/models/whats-new-opus-5) · [Pricing](https://docs.anthropic.com/en/about-claude/pricing)
## Latest
Reasoning · Speed
`anthropic/claude-fable-5`
Anthropic's most capable widely released model for long-horizon agentic work, complex reasoning, and ambitious multi-day coding tasks.
* \$10 / \$50 per 1M tokens (input / output)
* 1M context
* 128K max output
* Text, Image input
* Adaptive thinking (always on)
* Web search
* Knowledge cutoff Jan 2026
Reasoning · Speed
`anthropic/claude-opus-5`
Latest Opus and the recommended default for complex agentic coding and enterprise work — near-Fable-5 intelligence at half the price, with large gains in deep reasoning, long-horizon tool loops, and test-time compute scaling.
* \$5 / \$25 per 1M tokens (input / output; Fast mode \$10 / \$50 at \~2.5x speed)
* 1M context (default and maximum)
* 128K max output
* Text, Image input
* Adaptive thinking (on by default)
* Web search
* Knowledge cutoff May 2026
Reasoning · Speed
`anthropic/claude-sonnet-5`
Current-generation Sonnet — near-Opus intelligence at Sonnet pricing for coding, agents, and everyday professional work. Drop-in replacement for Sonnet 4.6.
* \$3 / \$15 per 1M tokens (input / output; intro \$2 / \$10 through Aug 31, 2026)
* 1M context
* 128K max output
* Text, Image input
* Adaptive thinking
* Web search
* Knowledge cutoff Jan 2026
Reasoning · Speed
`anthropic/claude-opus-4-8`
Previous Opus generation for agentic coding, long-running tasks, and complex reasoning. Same price as Opus 5 — migrate unless you depend on thinking being off by default.
* \$5 / \$25 per 1M tokens (input / output)
* 1M context
* 128K max output
* Text, Image input
* Adaptive thinking
* Web search
* Knowledge cutoff Jan 2026
Reasoning · Speed
`anthropic/claude-opus-4-7`
Frontier Opus for agentic coding and complex reasoning: stronger software engineering, verification, and high-resolution vision than 4.6, with adaptive thinking and a 1M-token context window at standard per-token rates.
* \$5 / \$25 per 1M tokens (input / output)
* 1M context
* 128K max output
* Text, Image input
* Adaptive thinking
* Web search
* Knowledge cutoff Jan 2026
Reasoning · Speed
`anthropic/claude-opus-4-6`
Anthropic's most intelligent model for building agents and coding. Exceptional at planning, code review, debugging, and operating reliably within large codebases.
* \$5 / \$25
* 1M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
* Knowledge cutoff May 2025
Reasoning · Speed
`anthropic/claude-haiku-4-5`
The fastest Claude model with near-frontier intelligence for office files, strategy planning, and business analysis.
* \$1 / \$5
* 200K context
* 64K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Feb 2025
***
## Legacy
Reasoning · Speed
`anthropic/claude-opus-4-5`
Previous flagship Opus model with top-tier reasoning and coding capabilities.
* \$5 / \$25
* 200K context
* 64K max output
* Text, Image input
* Extended thinking
* Knowledge cutoff May 2025
Reasoning · Speed
`anthropic/claude-opus-4-1`
Earlier Opus variant with strong reasoning, higher pricing tier.
* \$15 / \$75
* 200K context
* 32K max output
* Text, Image input
* Extended thinking
* Knowledge cutoff Jan 2025
* Deprecated — retires August 5, 2026
Reasoning · Speed
`anthropic/claude-sonnet-4-6`
Previous Sonnet generation with strong speed/intelligence balance and computer use skills. Superseded by Sonnet 5.
* \$3 / \$15
* 1M context
* 64K max output
* Text, Image input
* Thinking
* Knowledge cutoff Aug 2025
Reasoning · Speed
`anthropic/claude-sonnet-4-5`
Previous Sonnet generation with strong all-around performance and 1M beta context support.
* \$3 / \$15
* 200K context (1M beta)
* 64K max output
* Text, Image input
* Thinking
* Knowledge cutoff Jan 2025
# BytePlus
Source: https://docs.timbal.ai/models/byteplus
Seed 2.0 and Seed 1.8 models with specs, pricing, and capabilities
Source: [BytePlus ModelArk docs](https://docs.byteplus.com/en/docs/ModelArk). All models support tool/function calling. Models with an activation warning must be enabled in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before use.
## Seed 2.0
Reasoning · Speed
`byteplus/seed-2-0-pro-260328`
BytePlus's flagship Seed 2.0 model. Excels at complex reasoning, coding, and agentic tasks with deep thinking capability and multimodal understanding across text, images, and video.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.47 / \~\$2.37
* 256K context
* Text, Image, Video input
* Deep thinking
Reasoning · Speed
`byteplus/seed-2-0-lite-260228`
Cost-efficient Seed 2.0 variant optimised for high-concurrency and latency-sensitive workloads. Supports multimodal input and tool calling at a fraction of the Pro cost.
* \$0.25 / \$1
* 256K context
* Text, Image, Video input
Reasoning · Speed
`byteplus/seed-2-0-mini-260215`
Lightweight Seed 2.0 model designed for edge deployments and cost-sensitive scenarios requiring fast responses with minimal resource usage.
* \~\$0.10 / \~\$0.40
* 256K context
* Text input
***
## Seed 1.x
Reasoning · Speed
`byteplus/seed-1-8-251228`
Advanced multimodal model with deep thinking and strong agent capabilities. Supports text, image, video, and document understanding with structured output (beta) and context caching.
* \$0.25 / \$2
* 256K context
* 64K max output
* Text, Image, Video input
* Deep thinking
Reasoning · Speed
`byteplus/seed-1-6-250915`
Stable text generation model from the Seed 1.x series, suited for general-purpose NLP tasks, summarisation, and instruction following.
* \~\$0.25 / \~\$2
* 262K context
* Text input
***
## Third-party (Ark)
Reasoning · Speed
`byteplus/kimi-k2-250905`
Moonshot Kimi K2 agentic model on BytePlus ModelArk for tool calling and autonomous tasks.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.60 / \~\$2.50
* 256K context
* Text input
Reasoning · Speed
`byteplus/kimi-k2-thinking-251104`
Moonshot Kimi K2 thinking variant on BytePlus ModelArk with chain-of-thought reasoning and tool use.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.60 / \~\$2.50
* 256K context
* Text input
* Thinking
Reasoning · Speed
`byteplus/deepseek-v4-pro-260425`
DeepSeek V4 Pro on BytePlus ModelArk: frontier open MoE for coding, reasoning, and long-context agent workloads.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.47 / \~\$2.37
* 256K context
* Text input
Reasoning · Speed
`byteplus/deepseek-v4-flash-260425`
Cost-efficient DeepSeek V4 Flash on BytePlus ModelArk for fast reasoning and agent workloads.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.25 / \~\$1
* 256K context
* Text input
Reasoning · Speed
`byteplus/deepseek-v3-2-251201`
DeepSeek V3.2 on BytePlus ModelArk with improved reasoning and tool use over V3.1.
* \~\$0.25 / \~\$1
* 256K context
* Text input
Reasoning · Speed
`byteplus/deepseek-r1-250528`
DeepSeek R1 reasoning model on BytePlus ModelArk (May 2025 checkpoint).
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.50 / \~\$2
* 256K context
* Text input
* Thinking
Reasoning · Speed
`byteplus/gpt-oss-120b-250805`
OpenAI GPT OSS 120B on BytePlus ModelArk. Apache 2.0.
* \~\$0.15 / \~\$0.60
* 128K context
* Text input
Reasoning · Speed
`byteplus/glm-4-7-251222`
Zhipu GLM-4.7 on BytePlus ModelArk for coding, reasoning, and agentic tasks.
* \~\$0.25 / \~\$1
* 128K context
* Text input
Reasoning · Speed
`byteplus/seed-2-0-code-preview-260328`
Seed 2.0 code-specialized preview on BytePlus ModelArk for agentic coding workloads.
Activation required — enable in the [Ark Console](https://console.byteplus.com/ark/region:ark+ap-southeast-1/models) before first use.
* \~\$0.25 / \~\$1
* 256K context
* Text input
# Cerebras
Source: https://docs.timbal.ai/models/cerebras
Wafer-scale inference at world-record token speeds with specs, pricing, and capabilities
Source: [Cerebras model docs](https://inference-docs.cerebras.ai/introduction). All model IDs use the prefix `cerebras/`. Powered by Cerebras wafer-scale chips — the world's largest AI accelerator — delivering up to 3000+ tokens/second.
## All Models
Reasoning · Speed
`cerebras/gpt-oss-120b`
OpenAI's open-weight MoE model with 120B total parameters (5.1B active per token), running at up to 3000 tokens/s on Cerebras wafer-scale hardware. Near-parity with o4-mini on reasoning benchmarks. Supports extended thinking. Apache 2.0.
* \$0.35 / \$0.75
* 128K context
* Text input
* Thinking
* Knowledge cutoff Jun 2024
Reasoning · Speed
`cerebras/zai-glm-4.7`
ZAI GLM 4.7 with 355B parameters, running at \~1000 tokens/s on Cerebras hardware. Strong multilingual reasoning and instruction-following capabilities.
* \$2.25 / \$2.75
* 128K context
* Text input
* Knowledge cutoff \~early 2025
# Fireworks
Source: https://docs.timbal.ai/models/fireworks
Open-source models via Fireworks serverless inference with specs, pricing, and capabilities
Source: [Fireworks model docs](https://fireworks.ai/models). All model IDs use the prefix `fireworks/accounts/fireworks/models/`. Only serverless models are listed here — other Fireworks models require a dedicated deployment.
## All Models
Reasoning · Speed
`fireworks/accounts/fireworks/models/deepseek-v4-pro`
DeepSeek V4 Pro on Fireworks serverless: frontier open MoE for coding, reasoning, and up to \~1M token context with function calling.
* \$1.74 / \$3.48
* 1M context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/deepseek-v4-flash`
DeepSeek V4 Flash on Fireworks serverless: fast, cost-efficient MoE with near-Pro reasoning at 1M context.
* \$0.14 / \$0.28
* 1M context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/qwen3p7-plus`
Alibaba Qwen 3.7 Plus on Fireworks serverless: multimodal flagship with strong agentic and coding benchmarks.
* \$0.40 / \$1.60
* 262K context
* Text, Image input
Reasoning · Speed
`fireworks/accounts/fireworks/models/qwen3p6-plus`
Qwen 3.6 multimodal plus tier on Fireworks for vision-language and general agent tasks.
* \$0.50 / \$3.00
* 262K context
* Text, Image input
* Serverless deprecated — prefer qwen3p7-plus
Reasoning · Speed
`fireworks/accounts/fireworks/models/kimi-k2p6`
Moonshot Kimi K2.6 on Fireworks: multimodal MoE for high-quality tool use and long-context workloads.
* \$0.95 / \$4.00
* 262K context
* Text, Image input
Reasoning · Speed
`fireworks/accounts/fireworks/models/kimi-k2p7-code`
Moonshot Kimi K2.7 Code on Fireworks serverless for long-context coding agents with thinking mode.
* \$0.95 / \$4.00
* 262K context
* Text, Image input
* Thinking
Reasoning · Speed
`fireworks/accounts/fireworks/models/kimi-k2p5`
Moonshot Kimi K2.5 on Fireworks serverless: multimodal 1T-parameter MoE with strong agentic tool use.
* \$0.60 / \$3.00
* 256K context
* Text, Image input
* Serverless deprecated — prefer kimi-k2p6
Reasoning · Speed
`fireworks/accounts/fireworks/models/glm-5p1`
Z.ai GLM-5.1 on Fireworks serverless: post-training upgrade with stronger coding, reasoning, and agentic tool use.
* \$1.40 / \$4.40
* 203K context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/glm-5p2`
Z.ai GLM-5.2 on Fireworks serverless: 1M-token context, strongest open coding model with prompt caching (cached input \$0.14 / 1M).
* \$1.40 / \$4.40
* 1M context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/minimax-m2p5`
MiniMax M2.5 MoE (230B total, 10B active) with SOTA coding and agentic tool use on Fireworks serverless.
* \$0.30 / \$1.20
* 200K context
* Text input
* Serverless deprecated — prefer minimax-m2p7 or minimax-m3
Reasoning · Speed
`fireworks/accounts/fireworks/models/minimax-m2p7`
MiniMax M2.7 MoE on Fireworks serverless: improved agent harnesses, complex skills, and dynamic tool search.
* \$0.30 / \$1.20
* 196K context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/minimax-m3`
MiniMax M3 multimodal MoE on Fireworks serverless for chat, agents, and long-context workloads.
* \$0.30 / \$1.20
* 512K context
* Text, Image input
Reasoning · Speed
`fireworks/accounts/fireworks/models/gpt-oss-120b`
OpenAI's open-weight 120B MoE achieving near-parity with o4-mini on reasoning benchmarks. Apache 2.0.
* \$0.15 / \$0.60
* 128K context
* Text input
Reasoning · Speed
`fireworks/accounts/fireworks/models/gpt-oss-20b`
OpenAI's compact 20B MoE similar to o3-mini, running on edge devices with 16GB memory. Apache 2.0.
* \$0.07 / \$0.30
* 128K context
* Text input
# Google
Source: https://docs.timbal.ai/models/google
Gemini 3.6, 3.5, and 2.5 series models with specs, pricing, and capabilities
Source: [Google AI model docs](https://ai.google.dev/gemini-api/docs/models)
## Gemini 3.6
Reasoning · Speed
`google/gemini-3.6-flash`
Workhorse Flash model with better token efficiency, code generation, and multimodal reasoning for multi-step orchestration and agentic workflows.
* \$1.50 / \$7.50
* 1M context
* 64K max output
* Text, Image, Video, Audio, PDF input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
***
## Gemini 3.5
Reasoning · Speed
`google/gemini-3.5-flash`
Sustained frontier-level intelligence for agentic workflows, sub-agent deployment, and long-horizon tasks at scale — optimized for speed and cost.
* \$1.50 / \$9
* 1M context
* 64K max output
* Text, Image, Video, Audio, PDF input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-3.5-flash-lite`
Fastest, most cost-effective 3.5-class model for high-throughput agentic search, document processing, and lightweight subagent workflows.
* \$0.30 / \$2.50
* 1M context
* 64K max output
* Text, Image, Video, Audio, PDF input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
***
## Gemini 3
Reasoning · Speed
`google/gemini-3.1-pro-preview`
The upgraded core intelligence model representing a step forward in reasoning, a smarter and more capable baseline for complex problem-solving across consumer and developer products.
* \$2 / \$12
* 1M context
* 64K max output
* Text, Image, Video, Audio, PDF input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-3.1-flash-lite`
The fastest and most cost-efficient Gemini 3 model, built for high-volume developer workloads at scale with 2.5x faster time to first token versus 2.5 Flash.
* \$0.25 / \$1.50
* 1M context
* 64K max output
* Text, Image, Video, Audio input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-3-flash-preview`
Combines Gemini 3 Pro's reasoning capabilities with the Flash line's latency, efficiency, and cost, designed for the most complex agentic workflows.
* \$0.50 / \$3
* 1M context
* 64K max output
* Text, Image, Video, Audio input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
***
## Gemini 2.5
\$2.50 / \$15 above 200K input tokens. All Gemini models support Google Search grounding.
Reasoning · Speed
`google/gemini-2.5-pro`
The most advanced reasoning Gemini model, capable of solving complex problems across text, audio, images, video, and code repositories.
* \$1.25 / \$10
* 1M context
* 65K max output
* Text, Image, Video, Audio, PDF input
* Extended thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-2.5-pro-preview-tts`
Text-to-speech variant of Gemini 2.5 Pro for generating audio output from text input.
* \$1 / \$20
* 1M context
* 66K max output
* Text input
* Audio output
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-2.5-flash`
The best model in terms of price and performance with well-rounded capabilities, the first Flash model featuring thinking capabilities.
* \$0.30 / \$2.50
* 1M context
* 65K max output
* Text, Image, Video, Audio input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-2.5-flash-lite`
The fastest and most budget-friendly multimodal reasoning model in the 2.5 family, ideal for classification, translation, and high-scale operations.
* \$0.10 / \$0.40
* 1M context
* 65K max output
* Text, Image, Video, Audio input
* Thinking
* Web search
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-2.5-flash-image`
Specialized model for image generation and editing from text and image prompts. \~\$0.039 per image output.
* \$0.30 / \$30
* 32K context
* 32K max output
* Text, Image input
* Text, Image output
* Knowledge cutoff Jan 2025
Reasoning · Speed
`google/gemini-2.5-flash-preview-tts`
Text-to-speech variant of Gemini 2.5 Flash for generating audio output at lower cost.
* \$0.50 / \$10
* 1M context
* 66K max output
* Text input
* Audio output
* Knowledge cutoff Jan 2025
# Groq
Source: https://docs.timbal.ai/models/groq
Ultra-low latency inference via Groq LPU with specs, pricing, and capabilities
Source: [Groq model docs](https://console.groq.com/docs/models). All model IDs use the prefix `groq/`. Ultra-low latency inference via custom LPU hardware.
## All Models
Reasoning · Speed
`groq/qwen/qwen3.6-27b`
Qwen3.6 27B with hybrid thinking mode, delivered at Groq's ultra-low latency. Preview tier replacing retired Qwen3 32B / Llama 4 Scout.
* \$0.60 / \$3.00
* 131K context
* Text input
* Hybrid thinking
Reasoning · Speed
`groq/openai/gpt-oss-120b`
OpenAI's open-weight MoE model with 120B total parameters (5.1B active per token), running at Groq speeds. Near-parity with o4-mini on reasoning benchmarks. Apache 2.0.
* \$0.15 / \$0.60
* 128K context
* Text input
* Thinking
Reasoning · Speed
`groq/openai/gpt-oss-20b`
OpenAI's compact open-weight MoE model with 20B total parameters (3.6B active), delivering results similar to o3-mini at Groq's ultra-low latency. Apache 2.0.
* \$0.075 / \$0.30
* 128K context
* Text input
* Thinking
Reasoning · Speed
`groq/llama-3.3-70b-versatile`
Multilingual instruction-tuned model with 70B parameters, optimized for versatile tasks with Groq's ultra-low latency inference.
* \$0.59 / \$0.79
* 131K context
* Text input
* Deprecated — shutdown August 16, 2026
Reasoning · Speed
`groq/llama-3.1-8b-instant`
The most compact Llama 3.1 model with 8B parameters, optimized for instant responses on Groq's LPU hardware.
* \$0.05 / \$0.08
* 131K context
* Text input
* Deprecated — shutdown August 16, 2026
# Moonshot (Kimi)
Source: https://docs.timbal.ai/models/moonshot
Kimi K3 and K2.x models via Moonshot's OpenAI-compatible API
Source: [Kimi API Platform](https://platform.kimi.ai). All model IDs use the prefix `moonshot/`. Base URL: `https://api.moonshot.ai/v1`.
## Flagship
Reasoning · Speed
`moonshot/kimi-k3`
Moonshot's 2.8T-parameter flagship MoE with a 1M-token context window, native vision/video, and always-on reasoning for long-horizon coding and knowledge work. Pass `reasoning_effort` via `model_params` (currently `"max"` only). Do not set `temperature` — the API fixes it at `1.0`.
* \$3 / \$15 (cache-hit input \$0.30)
* 1M context
* Text, Image, Video input
* Thinking (`reasoning_content` / `reasoning_effort`)
## Coding
Reasoning · Speed
`moonshot/kimi-k2.7-code`
Coding-focused multimodal model with thinking mode for long-context programming agents.
* \$0.95 / \$4 (cache-hit input \$0.19)
* 256K context
* Text, Image, Video input
* Thinking
Reasoning · Speed
`moonshot/kimi-k2.7-code-highspeed`
Same K2.7 Code weights with higher output throughput (\~180 tok/s) for interactive coding.
* \$1.90 / \$8 (cache-hit input \$0.38)
* 256K context
* Text, Image, Video input
* Thinking
## General
Reasoning · Speed
`moonshot/kimi-k2.6`
General-purpose multimodal MoE with thinking and non-thinking modes for chat, agents, and vision.
* \$0.95 / \$4 (cache-hit input \$0.16)
* 256K context
* Text, Image, Video input
* Thinking
Reasoning · Speed
`moonshot/kimi-k2.5`
Previous multimodal MoE generation. Prefer K2.6 or K3 for new workloads. Soft sunset for new users; full sunset August 31, 2026.
* \$0.60 / \$3
* 256K context
* Text, Image input
* Thinking
## Usage
```python theme={"dark"}
from timbal import Agent
agent = Agent(
name="kimi",
model="moonshot/kimi-k3",
model_params={"reasoning_effort": "max"},
tools=[],
)
result = await agent.collect(prompt="Explain Kimi K3 in one sentence.")
print(result.output.collect_text())
```
# OpenAI
Source: https://docs.timbal.ai/models/openai
GPT-5, GPT-4, and o-series models with specs, pricing, and capabilities
Source: [OpenAI model docs](https://platform.openai.com/docs/models) · [API pricing](https://openai.com/api/pricing/) (cached input \$0.50 / 1M for GPT-5.5)
## GPT-5 Series
Reasoning · Speed
`openai/gpt-5.5`
OpenAI's latest flagship for coding, knowledge work, and research—positioned above GPT-5.4 with a 1.05M-token API context (per OpenAI model table) and higher per-token pricing.
* \$5 / \$30 per 1M tokens (input / output); cached input \$0.50 / 1M
* 1.05M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
Reasoning · Speed
`openai/gpt-5.5-pro`
GPT-5.5 Pro tier for the hardest coding, agentic, and research workloads via the Responses API.
* \$30 / \$180 per 1M tokens (input / output)
* 1.05M context
* Text, Image input
* Extended thinking
Reasoning · Speed
`openai/gpt-5.6-sol`
GPT-5.6 flagship tier for the hardest coding, agentic, and long-horizon tasks. Supports programmatic tool calling and multi-agent flows in the Responses API.
* \$5 / \$30 per 1M tokens (input / output); cached input \$0.50 / 1M
* 1.05M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
Reasoning · Speed
`openai/gpt-5.6-terra`
GPT-5.6 balanced tier for everyday production work—competitive with GPT-5.5 at roughly half the cost of Sol.
* \$2.50 / \$15 per 1M tokens (input / output); cached input \$0.25 / 1M
* 1.05M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
Reasoning · Speed
`openai/gpt-5.6-luna`
GPT-5.6 fast, cost-efficient tier for high-volume, latency-sensitive, and budget-conscious workloads.
* \$1 / \$6 per 1M tokens (input / output); cached input \$0.10 / 1M
* 1.05M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
Reasoning · Speed
`openai/gpt-5.4`
OpenAI's most capable and efficient frontier model for professional work, combining industry-leading coding, reasoning, and agentic workflows with native computer-use capabilities.
* \$2.50 / \$15
* 1.05M context
* 128K max output
* Text, Image input
* Extended thinking
* Web search
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.4-pro`
GPT-5.4 Pro tier for maximum capability coding and agent workloads via the Responses API.
* \$30 / \$180 per 1M tokens (input / output)
* 1.05M context
* Text, Image input
* Extended thinking
Reasoning · Speed
`openai/gpt-5.4-mini`
OpenAI's most capable small model, significantly improving over GPT-5 mini across coding, reasoning, multimodal understanding, and tool use while running more than 2x faster.
* \$0.75 / \$4.50
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.4-nano`
The smallest, cheapest version of GPT-5.4 for tasks where speed and cost matter most. Ideal for classification, data extraction, ranking, and coding subagents.
* \$0.20 / \$1.25
* 400K context
* 128K max output
* Text, Image input
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.3-chat-latest`
Tuned for safe, useful answers more directly, designed for everyday usability across drafting, brainstorming, summarizing, and general writing tasks.
* \~\$1.75 / \~\$14
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.2`
The most capable model series for professional knowledge work, with significant improvements in general intelligence, long-context understanding, agentic tool-calling, and vision.
* \$1.75 / \$14
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.2-pro`
A version of GPT-5.2 that uses more compute to think harder and provide consistently better answers for complex tasks.
* \$21 / \$168
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Aug 2025
Reasoning · Speed
`openai/gpt-5.1`
Improves meaningfully on both intelligence and communication style; the first model to use adaptive reasoning to decide when to think before responding.
* \$1.25 / \$10
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Sep 2024
Reasoning · Speed
`openai/gpt-5.1-codex`
A faster, more intelligent agentic coding model designed for long-running, project-scale work with enhanced reasoning and token efficiency.
* \$1.25 / \$10
* 400K context
* 128K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Sep 2024
Reasoning · Speed
`openai/gpt-5`
OpenAI's unified AI system representing a significant leap in intelligence, with a smart efficient model for most questions and a deeper reasoning model for harder problems.
* \$1.25 / \$10
* 400K context
* 128K max output
* Text, Image, Audio, Video input
* Thinking
* Web search
* Knowledge cutoff Sep 2024
Reasoning · Speed
`openai/gpt-5-mini`
A smaller, faster variant of GPT-5's thinking model optimized for developer use, balancing strong reasoning with lower cost and latency.
* \$0.25 / \$2
* 400K context
* 128K max output
* Text, Image input
* Web search
* Knowledge cutoff May 2024
Reasoning · Speed
`openai/gpt-5-nano`
The smallest and fastest variant in the GPT-5 family, made for developers needing maximum speed at minimal cost.
* \$0.05 / \$0.40
* 400K context
* 128K max output
* Text, Image input
* Knowledge cutoff May 2024
***
## GPT-4 Series
Reasoning · Speed
`openai/gpt-4.1`
Excels at instruction following and tool calling with broad knowledge, featuring a 1M token context window and low latency without a reasoning step.
* \$2 / \$8
* 1.05M context
* 32K max output
* Text, Image input
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/gpt-4.1-mini`
A significant leap in small model performance that matches or exceeds GPT-4o in intelligence evals while reducing latency by nearly half and cost by 83%.
* \$0.40 / \$1.60
* 1M context
* 32K max output
* Text, Image input
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/gpt-4.1-nano`
OpenAI's fastest and cheapest model with a 1M token context window, ideal for classification and autocompletion tasks.
* \$0.10 / \$0.40
* 1M context
* 32K max output
* Text, Image input
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/gpt-4o`
An autoregressive omni model that accepts any combination of text, audio, image, and video inputs, trained end-to-end across modalities.
* \$2.50 / \$10
* 128K context
* 16K max output
* Text, Image, Audio input
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/gpt-4o-mini`
A fast, affordable small model for focused tasks that accepts text and image inputs at a fraction of the cost of frontier models.
* \$0.15 / \$0.60
* 128K context
* 16K max output
* Text, Image input
* Knowledge cutoff Oct 2023
***
## o-Series (Reasoning)
Reasoning · Speed
`openai/o4-mini`
A smaller reasoning model optimized for fast, cost-efficient reasoning with exceptional performance in math, coding, and visual tasks.
* \$1.10 / \$4.40
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/o4-mini-deep-research`
Deep research variant of o4-mini designed for extended multi-step research tasks with mandatory search.
* \$2 / \$8
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Web search (required)
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/o3`
OpenAI's most powerful reasoning model that pushes the frontier across coding, math, science, and visual perception, ideal for complex queries requiring multi-faceted analysis.
* \$2 / \$8
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/o3-mini`
The most cost-efficient model in OpenAI's reasoning series, delivering o1-level STEM performance with lower cost and faster speed.
* \$1.10 / \$4.40
* 200K context
* 100K max output
* Text input
* Thinking
* Knowledge cutoff Oct 2023
Reasoning · Speed
`openai/o3-pro`
A version of o3 designed to think longer and provide the most reliable, thoroughly reasoned responses for maximum performance on complex tasks.
* \$20 / \$80
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Web search
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/o3-deep-research`
Deep research variant of o3 for extended multi-step research tasks with mandatory search and maximum reasoning depth.
* \$10 / \$40
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Web search (required)
* Knowledge cutoff Jun 2024
Reasoning · Speed
`openai/o1`
OpenAI's first reasoning model, trained to think before answering by producing a long internal chain of thought, with strong performance in science, coding, and math.
* \$15 / \$60
* 200K context
* 100K max output
* Text, Image input
* Thinking
* Knowledge cutoff Oct 2023
Reasoning tokens are billed as output tokens on all o-series models.
# Overview
Source: https://docs.timbal.ai/models/overview
Specs, pricing, and capabilities for every supported model
Every model supported by Timbal with full specs, pricing, capability scores, and short descriptions. **Price** = per 1M tokens (input / output). All models support tool/function calling unless noted.
Claude Fable 5, Opus 5, Sonnet 5, and Haiku
Seed 2.0 and Seed 1.8 models
Wafer-scale inference at world-record token speeds
Open-source models via Fireworks
Gemini 3.6, 3.5, 3.1 and 2.5 series
Ultra-low latency via Groq LPU
Kimi K3 / K2.x via Moonshot's OpenAI-compatible API
GPT-5, GPT-4, and o-series reasoning models
High-throughput inference on custom RDU hardware
Open-source models via TogetherAI
Grok 4 and Grok 4 Fast
MiMo V2 Pro, Omni, and Flash
## Scoring
Each model is rated on two axes using a **1-5 scale**:
* **Reasoning** — depth of analytical and chain-of-thought capability
* **Speed** — relative latency and throughput for its class
Scores are **relative within the full set of models on this page**, not within a single provider. A reasoning score of 5 means frontier-class reasoning (e.g. o3-pro, Claude Opus 4.7, GPT-5.5, Gemini 2.5 Pro). A speed score of 5 means the fastest tier (nano/mini/flash-lite models, or Groq-hosted inference).
# SambaNova
Source: https://docs.timbal.ai/models/sambanova
High-throughput inference on custom RDU hardware with specs, pricing, and capabilities
Source: [SambaNova model docs](https://docs.sambanova.ai/docs/en/get-started/overview). All model IDs use the prefix `sambanova/`. Powered by SambaNova RDU (Reconfigurable Dataflow Unit) chips designed for large-scale AI inference.
## DeepSeek
Reasoning · Speed
`sambanova/DeepSeek-V3.2`
DeepSeek V3.2 running on SambaNova RDU hardware.
* \$3.00 / \$4.50
* 8K context
* Text input
Reasoning · Speed
`sambanova/DeepSeek-V3.1`
DeepSeek V3.1 running on SambaNova RDU hardware.
* \$3.00 / \$4.50
* 128K context
* Text input
## Meta Llama
Reasoning · Speed
`sambanova/Llama-4-Maverick-17B-128E-Instruct`
Meta Llama 4 Maverick with 17B active parameters across 128 experts. Supports image input.
* \$0.63 / \$1.80
* 128K context
* Text, Image input
Reasoning · Speed
`sambanova/Meta-Llama-3.3-70B-Instruct`
Meta Llama 3.3 70B instruction-tuned model running on SambaNova RDU hardware.
* \$0.60 / \$1.20
* 128K context
* Text input
## Google Gemma
Reasoning · Speed
`sambanova/gemma-4-31B-it`
Google Gemma 4 31B instruction-tuned model on SambaNova RDU hardware.
* \$0.22 / \$0.59
* 128K context
* Text, Image input
Reasoning · Speed
`sambanova/gemma-3-12b-it`
Google Gemma 3 12B instruction-tuned model on SambaNova RDU hardware.
* \$0.10 / \$0.20
* 128K context
* Text input
## OpenAI OSS / MiniMax
Reasoning · Speed
`sambanova/gpt-oss-120b`
OpenAI's open-weight 120B MoE on SambaNova RDU hardware. Near-parity with o4-mini on reasoning benchmarks. Apache 2.0.
* \$0.22 / \$0.59
* 128K context
* Text input
Reasoning · Speed
`sambanova/MiniMax-M2.7`
MiniMax M2.7 large-scale MoE running on SambaNova RDU hardware.
* \$0.60 / \$2.40
* 128K context
* Text input
# TogetherAI
Source: https://docs.timbal.ai/models/togetherai
Open-source models via TogetherAI inference with specs, pricing, and capabilities
Source: [TogetherAI model docs](https://docs.together.ai/docs/chat-models). All model IDs use the prefix `togetherai/`. Models with a dedicated-only warning are not serverless — you must create and start a dedicated endpoint first.
## Meta LLaMA
Reasoning · Speed
`togetherai/meta-llama/Llama-3.3-70B-Instruct-Turbo`
Multilingual instruction-tuned model with 70B parameters, delivering enhanced performance relative to Llama 3.1 70B and matching Llama 3.2 90B on text-only tasks.
* \$0.88 / \$0.88
* 128K context
* Text input
* Knowledge cutoff Dec 2023
***
## Qwen
Reasoning · Speed
`togetherai/Qwen/Qwen3.5-397B-A17B`
Multimodal foundation model with 397B total parameters (17B active) featuring a Hybrid MoE architecture with early fusion vision-language training. State-of-the-art across chat, RAG, vision-language, and agentic workflows.
* \$0.30 / \$1.20
* 262K context
* Text, Image input
* Hybrid thinking
* Knowledge cutoff \~2025
Reasoning · Speed
`togetherai/Qwen/Qwen3-235B-A22B-Instruct-2507-tput`
MoE model with 235B total parameters (22B active) in non-thinking mode, optimized for throughput. Supports multilingual dialogue across 100+ languages.
* \$0.20 / \$0.60
* 262K context
* Text input
* Knowledge cutoff \~early 2025
Reasoning · Speed
`togetherai/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8`
Qwen's most agentic code model, a 480B-parameter MoE (35B active) achieving results comparable to Claude Sonnet on agentic coding, browser-use, and repository-scale tasks.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8) before use.
* \$0.22 / \$1
* 262K context
* Text input
Reasoning · Speed
`togetherai/Qwen/Qwen3-Coder-Next-FP8`
Next-generation coding model with hybrid thinking mode for adaptive reasoning depth.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/Qwen/Qwen3-Coder-Next-FP8) before use.
* \$0.50 / \$1.20
* 256K context
* Text input
* Hybrid thinking
Reasoning · Speed
`togetherai/Qwen/Qwen3-Next-80B-A3B-Instruct`
First model in the Qwen3-Next series with 80B total parameters (3.9B active), featuring hybrid attention. Matches Qwen3-235B performance while using less than 10% training cost.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/Qwen/Qwen3-Next-80B-A3B-Instruct) before use.
* \$0.15 / \$1.50
* 262K context
* Text input
* Hybrid thinking
Reasoning · Speed
`togetherai/Qwen/Qwen2.5-7B-Instruct-Turbo`
Part of the Qwen2.5 family with 7B parameters, featuring improvements in coding, mathematics, instruction following, and structured data understanding.
* \$0.30 / \$1.20
* 128K context
* Text input
* Knowledge cutoff \~Oct 2023
***
## DeepSeek
Reasoning · Speed
`togetherai/deepseek-ai/DeepSeek-V3.1`
Hybrid model supporting both thinking and non-thinking modes. Features significantly improved tool usage and agent task performance, with quality comparable to DeepSeek-R1-0528 in thinking mode.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/deepseek-ai/DeepSeek-V3.1) before use.
* \$0.60 / \$1.70
* 128K context
* Text input
* Hybrid thinking
* Knowledge cutoff \~mid 2025
Reasoning · Speed
`togetherai/deepseek-ai/DeepSeek-V4-Pro`
DeepSeek V4 Pro on Together serverless: hybrid attention, up to 512K context, strong coding and agent benchmarks.
* \$1.74 / \$3.48
* 512K context
* Text input
* Hybrid thinking
* Knowledge cutoff \~2025
***
## Kimi / MiniMax / GLM / Other
Reasoning · Speed
`togetherai/moonshotai/Kimi-K2.6`
Moonshot Kimi K2.6 on Together serverless: 1T-scale MoE with tool calling and JSON mode for agentic and multimodal workloads.
* \$1.20 / \$4.50
* 262K context
* Text, Image input
* Knowledge cutoff \~2025
Reasoning · Speed
`togetherai/moonshotai/Kimi-K2.7-Code`
Moonshot Kimi K2.7 Code on Together for long-context programming agents with thinking mode.
* \$0.95 / \$4.00
* 262K context
* Text, Image input
* Thinking
Reasoning · Speed
`togetherai/MiniMaxAI/MiniMax-M2.7`
MiniMax successor MoE (\~229B) with improved coding and agentic tool use, JSON mode, and prompt caching on Together serverless.
* \$0.30 / \$1.20
* 203K context
* Text input
Reasoning · Speed
`togetherai/MiniMaxAI/MiniMax-M3`
MiniMax M3 multimodal model on Together for chat, agents, and long-context workloads.
* \$0.30 / \$1.20
* 512K context
* Text, Image input
Reasoning · Speed
`togetherai/zai-org/GLM-5`
Zhipu AI's fifth-generation model with \~745B parameters in a MoE architecture (44B active), designed for complex system engineering and long-range agent tasks. Trained entirely on Huawei Ascend chips.
* \$1 / \$3.20
* 200K context
* Text input
* Thinking
* Knowledge cutoff late 2025
Reasoning · Speed
`togetherai/zai-org/GLM-5.1`
Z.ai post-training upgrade to GLM-5: 754B MoE (40B active), 200K context, thinking mode, tool calling, and stronger coding via RL.
* \$1.40 / \$4.40
* 200K context
* Text input
* Thinking
* Knowledge cutoff late 2025
Reasoning · Speed
`togetherai/zai-org/GLM-5.2`
Z.ai GLM-5.2 flagship on Together for coding, reasoning, and agentic tool use.
* \$1.40 / \$4.40
* 200K context
* Text input
* Thinking
Reasoning · Speed
`togetherai/zai-org/GLM-4.7`
Zhipu AI's foundation model with \~400B parameters and 200K context, designed for real-world development environments with strong coding, reasoning, and agentic capabilities.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/zai-org/GLM-4.7) before use.
* \$0.45 / \$2
* 200K context
* Text input
* Thinking
* Knowledge cutoff \~mid 2024
Reasoning · Speed
`togetherai/openai/gpt-oss-120b`
OpenAI's open-weight MoE model with 120B total parameters (5.1B active per token). Achieves near-parity with o4-mini on core reasoning benchmarks while running on a single 80GB GPU. Apache 2.0.
* \$0.15 / \$0.60
* 128K context
* Text input
* Thinking
* Knowledge cutoff Jun 2024
Reasoning · Speed
`togetherai/openai/gpt-oss-20b`
OpenAI's compact 20B MoE delivering o3-mini-level results on Together serverless. Apache 2.0.
* \$0.05 / \$0.20
* 128K context
* Text input
* Thinking
* Knowledge cutoff Jun 2024
Reasoning · Speed
`togetherai/google/gemma-3n-E4B-it`
Google's on-device multimodal model with 8B raw parameters but an effective 4B memory footprint. First sub-10B model to exceed 1300 on LMArena, running with as little as 3GB of memory.
* \$0.02 / \$0.04
* 32K context
* Text, Image, Audio, Video input
Reasoning · Speed
`togetherai/google/gemma-3-27b-it`
Google's multimodal open model with 27B parameters, built from the same technology as Gemini 2.0. Supports 128K context, 140+ languages, and runs on a single GPU/TPU.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/google/gemma-3-27b-it) before use.
* \~\$0.10 / \~\$0.10
* 128K context
* Text, Image input
* Knowledge cutoff Aug 2024
Reasoning · Speed
`togetherai/deepcogito/cogito-v2-1-671b`
DeepCogito's MoE model with 671B total parameters (37B active), trained via a novel process supervision approach that guides reasoning chains. Competitive with frontier closed models while using fewer tokens.
* \$1.25 / \$1.25
* 128K context
* Text input
* Thinking
Reasoning · Speed
`togetherai/mistralai/Mistral-Small-24B-Instruct-2501`
A 24B-parameter dense model setting new benchmarks in the sub-70B category, with native function calling, JSON output, and support for dozens of languages. Fits on a single RTX 4090.
Dedicated only — create and start a [dedicated endpoint](https://api.together.ai/models/mistralai/Mistral-Small-24B-Instruct-2501) before use.
* \$0.10 / \$0.30
* 32K context
* Text input
* Knowledge cutoff Oct 2023
# xAI
Source: https://docs.timbal.ai/models/xai
Grok 4.5 and Grok 4.3 models with specs, pricing, and capabilities
Source: [xAI model docs](https://docs.x.ai/docs/models). Search includes Web Search and X Search, priced at \$2.50-\$5/1K calls. Cached input: \$0.05/1M.
## Grok 4.5
Reasoning · Speed
`xai/grok-4.5`
xAI's July 2026 flagship for coding, agentic tool use, and knowledge work. Configurable reasoning effort (low / medium / high).
* \$2 / \$6 (\$4 / \$12 above 200K prompt tokens)
* 500K context
* Text, Image input
* Configurable reasoning
* Web search
***
## Grok 4.3
Reasoning · Speed
`xai/grok-4.3`
Production mid-tier for coding, agents, and general workloads. Replacement for the retired Grok 4 Fast / 4.1 Fast slugs (removed May 15, 2026).
* \$1.25 / \$2.50
* 1M context
* Text, Image input
* Reasoning
* Web search
# Xiaomi MiMo
Source: https://docs.timbal.ai/models/xiaomi
MiMo V2.5 models with specs, pricing, and capabilities
Source: [Xiaomi MiMo platform](https://platform.xiaomimimo.com). All models support tool/function calling. MiMo v2 (pro/omni/flash) was fully deprecated June 30, 2026.
## MiMo V2.5
Reasoning · Speed
`xiaomi/mimo-v2.5-pro`
MiMo v2.5 Pro flagship on Xiaomi serverless for coding, agents, and extended reasoning workloads.
* \$0.435 / \$0.87
* 1M context
* Text input
Reasoning · Speed
`xiaomi/mimo-v2.5`
MiMo v2.5 base model on Xiaomi serverless: improved reasoning and agentic tool use over the retired v2 line.
* \$0.14 / \$0.28
* 262K context
* Text input
# Quickstart
Source: https://docs.timbal.ai/quickstart
Build your first AI Agent with Timbal in a few lines of code
In this quickstart guide we're going to see how to build an AI agent with Timbal. We'll start from scratch and gradually enhance it with advanced features. Let's dive in!
Before moving forward, ensure you've completed the [installation steps](/installation).
## Create a Timbal Project
Run the following Timbal CLI command to create a new project:
```bash theme={"dark"}
timbal create my-project
```
This will prompt you interactively to choose what you want to build (Agent or Workflow) and what UI you want for your project. After making your selections, it creates a project structure with all the necessary files and dependencies to get started.
### Project Structure
After running the command, your project will look like this:
```
my-project/
├── api/ - Elysia API server (Bun)
├── workforce/ - Your timbal components (Python)
│ └── /
│ ├── agent.py - Main app logic (or workflow.py for workflows)
│ ├── timbal.yaml - Timbal configuration
│ └── pyproject.toml - Python dependencies
├── ui/ - React + Vite + TypeScript + shadcn (Bun) [optional]
├── .gitignore - Git ignore file
└── README.md - Project documentation
```
| Directory/File | Purpose |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| **api/** | Elysia API server running on Bun |
| **workforce/** | Your timbal components (Python). Contains subdirectories for each agent or workflow |
| **workforce/\/agent.py** | Main application file where you define your AI agents.
If you create a workflow, you'll have a `workflow.py` file instead |
| **workforce/\/pyproject.toml** | Python project configuration and dependency management |
| **workforce/\/timbal.yaml** | Timbal deployment configuration and settings |
| **ui/** | React + Vite + TypeScript + shadcn running on Bun (only if UI is selected) |
## Run locally
Run `timbal configure` before your first `timbal start` — it stores your Timbal credentials locally. For model calls, platform auth from configure is enough; otherwise set `OPENAI_API_KEY` in `/.env` (or the key for whichever provider your model uses).
From the project directory, start the full stack — API, workforce (your Python agents/workflows), and UI if you included one:
```bash theme={"dark"}
cd my-project
timbal start
```
`timbal start` prints the local URLs and keeps services running until you quit (`q`). While it's running you can press `o` to open the UI in your browser, or `h` for other commands.
If `/.env` exists, `timbal start` loads it automatically and injects those variables into every service (UI, API, and all workforce members). Per-member overrides can live in `workforce//.env`. See [Environment variables](/deployment#environment-variables) for scoping, precedence, and overrides.
To test a workforce member in isolation, you can also run its Python file directly: `uv run python workforce//agent.py` (the scaffold includes a small terminal REPL).
## Customize your agent
`timbal create` scaffolds a starter `agent.py` (or `workflow.py`) under `workforce//`. Open that file and edit the `Agent` definition.
A minimal agent needs only a **name** and **model**:
```python agent.py icon="python" theme={"dark"}
from timbal import Agent
agent = Agent(
name="my_agent",
model="openai/gpt-5-mini"
)
```
If you use your own provider key instead of platform auth, set it in a `.env` file at the project root (e.g. `OPENAI_API_KEY` for OpenAI models) — `timbal start` picks it up automatically. Never commit `.env` to version control.
The scaffold already includes a terminal REPL (`main()` at the bottom of `agent.py`). After editing, restart with `timbal start` to pick up changes in the full app, or run the file directly for a quick loop.
You can learn more about the Agent class and all the available options in the [Agents](/agents) section.
## Adding Tools
Agents become powerful with tools.
In Timbal, you can use both **baked-in tools** (pre-built and ready to use) and **custom tools** (functions you write yourself).
### Baked-in Tools
Timbal includes several ready-to-use tools. For example, the `WebSearch` tool allows your agent to search the internet for real-time information:
```python agent.py icon="python" theme={"dark"}
from timbal import Agent
from timbal.tools import WebSearch
agent = Agent(
name="my_agent",
model="openai/gpt-5-mini",
tools=[WebSearch()]
)
```
### Custom Tools
Creating custom tools is straightforward—just write a function. Here's a simple example that returns the current date and time:
```python agent.py icon="python" theme={"dark"}
from datetime import datetime
from timbal import Agent
from timbal.tools import WebSearch
def get_datetime() -> str:
return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
agent = Agent(
name="my_agent",
model="openai/gpt-5-mini",
tools=[WebSearch(), get_datetime]
)
```
You can combine any number of tools to create powerful, specialized agents.
Mix and match baked-in and custom tools to fit your specific use case.
You can even use agents and workflows as tools!
**Congratulations!** You've built a fully functional AI agent with tools in just a few lines of code.
Ready to explore more? Check out our [examples](/examples), or browse the latest code samples on GitHub.
## Moving to Production
Ready to take your agent to production? Timbal provides enterprise-grade deployment options designed for real-world applications.
Whether you need fully managed hosting or complete infrastructure control, we've got you covered. Check out our comprehensive [deployment guide](/deployment) to explore your options and choose the best approach for your needs.
# Agents & LLMs
Source: https://docs.timbal.ai/workflows/agents-and-llms
Use Agents as workflow steps and build LLM routing pipelines
Any Runnable works as a workflow step, including **Agents**. Use agents inside workflows when you want LLM reasoning at specific points in an otherwise deterministic pipeline, or when a classifier agent routes to specialist agents.
## Agent as a single step
```python theme={"dark"}
import asyncio
from timbal import Agent, Workflow
summarizer = Agent(
name="summarizer",
model="anthropic/claude-sonnet-4-6",
max_tokens=1024,
)
workflow = Workflow(name="summarize").step(
summarizer,
prompt="Summarize the quarterly report in three bullet points.",
)
async def main():
result = await workflow().collect()
print(result.output.collect_text())
asyncio.run(main())
```
The agent's output (typically a `Message`) is the workflow output.
## Passing inputs through the workflow
Expose agent params as workflow-level inputs by leaving them unbound on the step:
```python theme={"dark"}
workflow = (
Workflow(name="support")
.step(support_agent, prompt="Handle this ticket") # fixed prompt
)
# Or pass prompt at call time:
result = await workflow(prompt="Customer cannot log in").collect()
```
Steps with fixed kwargs in `.step(...)` hide those params from the workflow schema. Steps without defaults for a param surface it on the workflow.
## LLM routing (classify → specialist)
Use a lightweight classifier agent, then branch with `when`:
```python theme={"dark"}
import asyncio
from timbal import Agent, Workflow
from timbal.state import get_run_context
classifier = Agent(
name="classifier",
model="openai/gpt-5-mini",
system_prompt="Classify the message as 'technical' or 'billing'. One word only.",
max_tokens=64,
)
technical_agent = Agent(
name="technical_agent",
model="anthropic/claude-sonnet-4-6",
system_prompt="You are a technical support specialist.",
max_tokens=1024,
)
billing_agent = Agent(
name="billing_agent",
model="openai/gpt-5-mini",
system_prompt="You are a billing support specialist.",
max_tokens=1024,
)
workflow = (
Workflow(name="support_router")
.step(classifier)
.step(
technical_agent,
when=lambda: "technical"
in get_run_context().step_span("classifier").output.collect_text().lower(),
)
.step(
billing_agent,
when=lambda: "billing"
in get_run_context().step_span("classifier").output.collect_text().lower(),
)
)
async def main():
result = await workflow(prompt="I can't access my account after the upgrade").collect()
print(result.output.collect_text())
asyncio.run(main())
```
Only the matching specialist runs. See [Conditional Routing](/examples/workflows/conditional-routing) for a function-based variant.
## Agents + plain functions
Mix deterministic steps with agents in one pipeline:
```python theme={"dark"}
workflow = (
Workflow(name="doc_pipeline")
.step(fetch_content, url="https://example.com/report") # plain function
.step(extract_metadata, html=lambda: get_run_context().step_span("fetch_content").output)
.step(summarizer, prompt=lambda: f"Summarize: {get_run_context().step_span('extract_metadata').output['text']}")
.step(format_markdown, summary=lambda: get_run_context().step_span("summarizer").output.collect_text())
)
```
See [Sequential Steps](/examples/workflows/sequential-steps) for a full document pipeline.
## Nested workflows with agents
An inner workflow can contain agents; the outer workflow treats it as one step:
```python theme={"dark"}
analysis = (
Workflow(name="analysis")
.step(fetch_sales)
.step(summarizer, prompt=lambda: f"Summarize: {get_run_context().step_span('fetch_sales').output}")
)
delivery = (
Workflow(name="delivery")
.step(analysis)
.step(send_email, body=lambda: get_run_context().step_span("analysis").output.collect_text())
)
```
## When to use an agent vs a workflow
* **One agent with tools** — the model picks tools dynamically; good for open-ended tasks.
* **Workflow with agent steps** — you control which LLM runs when; good for staged pipelines, cost control, and auditability.
* **Workflow without agents** — pure data/code; fastest and cheapest.
## See also
* [Branching](/workflows/branching) — `when` conditions and skipped dependents
* [Agents](/agents) — agent configuration, tools, memory
* [Sequential Steps example](/examples/workflows/sequential-steps)
# Branching
Source: https://docs.timbal.ai/workflows/branching
Conditional step execution with the `when` parameter
## Conditional Execution
The `when` parameter controls whether a step runs based on runtime conditions:
```python theme={"dark"}
workflow = (
Workflow(name="router")
.step(classify, text="Urgent: server down")
.step(handle_urgent,
text="Server down",
when=lambda: get_run_context().step_span("classify").output == "urgent")
.step(handle_normal,
text="Server down",
when=lambda: get_run_context().step_span("classify").output == "normal")
)
```
Both `handle_urgent` and `handle_normal` wait for `classify` to complete. Only the one whose condition is met will execute.
## Skipped Steps
If a step's condition is not met, it is skipped along with all its dependents:
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(validate_input, data="...")
.step(process,
when=lambda: get_run_context().step_span("validate_input").output == "valid")
.step(save_results,
data=lambda: get_run_context().step_span("process").output)
)
```
If `validate_input` returns `"invalid"`, both `process` and `save_results` are skipped. `save_results` depends on `process`, which never runs.
Not all dependents of a skipped step are skipped. When a step uses `depends_on`, it only waits for the referenced steps to **resolve** (either complete or be skipped) — it doesn't need their data. This is useful when you don't know which branch will run:
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(classify, text="new customer signup")
.step(handle_new,
data="signup data",
when=lambda: get_run_context().step_span("classify").output == "new")
.step(handle_existing,
data="update data",
when=lambda: get_run_context().step_span("classify").output == "existing")
.step(finalize, depends_on=["handle_new", "handle_existing"])
)
```
`handle_new` and `handle_existing` have inverse conditions, so only one runs. `finalize` depends on both via `depends_on`, so it waits for both to resolve (one completes, the other is skipped) and then executes regardless.
## Conditional with Agents
Use an LLM to make routing decisions:
```python theme={"dark"}
classifier = Agent(
name="classifier",
model="openai/gpt-5-mini",
system_prompt="Classify the message as 'technical' or 'billing'. Respond with one word only.",
max_tokens=64,
)
technical_agent = Agent(
name="technical_agent",
model="anthropic/claude-sonnet-4-6",
system_prompt="You are a technical support specialist.",
max_tokens=1024,
)
billing_agent = Agent(
name="billing_agent",
model="openai/gpt-5-mini",
system_prompt="You are a billing support specialist.",
max_tokens=1024,
)
workflow = (
Workflow(name="support_router")
.step(classifier)
.step(technical_agent,
when=lambda: "technical" in get_run_context().step_span("classifier").output.collect_text().lower())
.step(billing_agent,
when=lambda: "billing" in get_run_context().step_span("classifier").output.collect_text().lower())
)
result = await workflow(prompt="I can't access my account").collect()
```
The classifier agent decides which handler runs. Only the matching branch executes.
# Control Flow
Source: https://docs.timbal.ai/workflows/control-flow
Manage step execution order with dependencies and data passing
Workflows execute steps based on their dependencies. By understanding how Timbal resolves these dependencies, you can build efficient pipelines that run steps in parallel when possible and sequentially when needed.
## Parallel by Default
When steps have no relationship between them, they run concurrently. Timbal doesn't wait for one to finish before starting the next:
```python theme={"dark"}
async def fetch_users():
return ["Alice", "Bob"]
async def fetch_orders():
return [{"id": 1}, {"id": 2}]
workflow = (
Workflow(name="data_loader")
.step(fetch_users)
.step(fetch_orders)
)
# fetch_users and fetch_orders run in parallel
```
## Context Access
The input and output from each step is stored in its [Run Context](/core-concepts/context), accessible via `get_run_context()`. Each step exposes two built-in variables:
* **`.input`**: Contains a dictionary of all the parameters passed to the step. They are accessed through their name.
* **`.output`**: Contains the value(s) returned by the step. Can be a single value, dictionary, array, or custom class.
Steps can access data from their own context, the parent workflow, or any sibling step:
| Method | Description |
| ------------------- | ----------------------- |
| `current_span()` | Current step's data |
| `parent_span()` | Parent workflow's data |
| `step_span("name")` | Any sibling step's data |
### Custom Variables
You can also create your own custom variables to share data between steps:
```python theme={"dark"}
async def process_user(user_id: str):
# Store custom data on this step's context
get_run_context().current_span().user_status = "active"
return f"Processed user: {user_id}"
async def check_status():
# Access the custom variable from another step
status = get_run_context().step_span("process_user").user_status
return f"User status: {status}"
workflow = (
Workflow(name="user_pipeline")
.step(process_user, user_id="user_123")
.step(check_status)
)
```
After the workflow runs, each step has the following variables in their context:
**`process_user`:**
```python theme={"dark"}
.input["user_id"] = "user_123"
.output = "Processed user: user_123"
.user_status = "active"
```
**`check_status`:**
```python theme={"dark"}
.input = {}
.output = "User status: active"
```
## Data Dependencies
When a step parameter references another step's output via a lambda, Timbal automatically creates a dependency and enforces sequential execution:
```python theme={"dark"}
async def merge_results(users: list, orders: list) -> dict:
return {"users": users, "orders": orders}
workflow = (
Workflow(name="data_loader")
.step(fetch_users)
.step(fetch_orders)
.step(merge_results,
users=lambda: get_run_context().step_span("fetch_users").output,
orders=lambda: get_run_context().step_span("fetch_orders").output,
)
)
```
Here:
* `fetch_users` and `fetch_orders` run in parallel (no dependencies between them)
* `merge_results` waits for **both** to complete (its parameters reference their outputs)
The dependency is resolved automatically. You don't need to declare it explicitly. Even though `fetch_users` and `fetch_orders` haven't executed when `merge_results` is defined, Timbal detects the `step_span()` references and knows to wait.
### Handling Optional Steps
When a step might be skipped (e.g., due to a `when` condition), you need to safely handle cases where that step didn't execute. Use `default=None` with `step_span()` to check if a step ran before accessing its output.
#### Execution Flow
Consider a workflow that validates data and then processes it differently based on the validation result:
```
1. validate_data (always runs)
↓
├─→ If valid: process_valid runs
└─→ If invalid: process_invalid runs
2. merge_results (always runs)
└─→ Needs to access either process_valid OR process_invalid
(but not both, since only one will have executed)
```
**The Problem**: When `merge_results` tries to access `process_valid` or `process_invalid`, one of them won't exist because it was skipped. Without `default=None`, accessing a skipped step would raise an error.
**The Solution**: Use `step_span("name", default=None)` which returns `None` if the step was skipped, allowing you to check which step actually ran.
#### Example
```python {20-29} theme={"dark"}
workflow = (
Workflow(name="pipeline")
# Step 1: Always runs - validates the input data
.step(validate_data, data={"email": "user@example.com"})
# Step 2: Only runs if validation passes
.step(process_valid,
data=lambda: get_run_context().step_span("validate_data").input.get("data"),
when=lambda: get_run_context().step_span("validate_data").output.get("valid", False))
# Step 3: Only runs if validation fails
.step(process_invalid,
data=lambda: get_run_context().step_span("validate_data").input.get("data"),
when=lambda: not get_run_context().step_span("validate_data").output.get("valid", False))
# Step 4: Always runs - merges results from validation and processing
.step(merge_results,
order=lambda: get_run_context().step_span("validate_data").input.get("data"),
validated=lambda: get_run_context().step_span("validate_data").output,
processed=lambda: (
# Try to get process_valid output (returns None if step was skipped)
get_run_context().step_span("process_valid", default=None).output
if get_run_context().step_span("process_valid", default=None) is not None
# Otherwise, try process_invalid output
else (
get_run_context().step_span("process_invalid", default=None).output
if get_run_context().step_span("process_invalid", default=None) is not None
else None
)
)
)
)
```
**How it works**:
1. `validate_data` always executes and returns `{"valid": True}` or `{"valid": False}`
2. Based on the validation result, either `process_valid` or `process_invalid` runs (but never both)
3. `merge_results` uses `step_span("name", default=None)` to safely check which processing step ran:
* If `step_span("process_valid", default=None)` returns a span (not `None`), that step ran
* If it returns `None`, the step was skipped, so we check `process_invalid` instead
**Key Point**: `step_span("name", default=None)` returns `None` if the step was skipped, allowing you to handle optional dependencies gracefully. Without `default=None`, accessing a skipped step would raise an error.
## Loops
Use `while_` to repeat a step. It accepts an int (run exactly N times) or a parameterless callable evaluated after each iteration (do-while — the step always runs at least once):
```python theme={"dark"}
def fetch_page() -> dict:
# The step owns its cursor state (module global, closure, or reading
# its own previous span) — params are NOT re-resolved per iteration.
span = get_run_context().step_span("fetch_page", default=None)
cursor = span.output["next_cursor"] if span else None
return api.fetch(cursor=cursor)
workflow = (
Workflow(name="paginate")
.step(fetch_page,
while_=lambda: get_run_context().step_span("fetch_page").output["next_cursor"] is not None)
.step(summarize,
pages=lambda: get_run_context().step_span("fetch_page").output)
)
```
Each iteration produces its own span. `step_span("name")` returns the **latest** one — both inside the `while_` condition and in downstream steps. `Trace.get_path()` returns all iterations.
A `while_` condition that reads its own step's span does not create a dependency edge (the loop is not a cycle in the DAG). Conditions reading *other* steps' spans create dependencies as usual. `while_` combines with `when`: if `when` returns `False`, the step (and the whole loop) is skipped.
Things to keep in mind with `while_`:
* **Params resolve once**, before the first iteration. Lambdas are not re-evaluated per iteration — a looping step must own its cursor/accumulator state (e.g. by reading its own previous span, as above).
* **No built-in iteration cap** for callable conditions. A condition that never returns falsy loops forever; prefer an int count or make the condition provably terminating.
* **Pausing mid-loop restarts the loop.** If the step hits an approval gate or calls `suspend()`, loop progress is not persisted — on resume the loop restarts from iteration 1, re-running side effects of completed iterations. Also, because approval ids derive from `(path, input)` and the input is fixed across iterations, a single approval covers every iteration in the same resume run. Avoid combining `while_` with approval-gated or suspending steps unless that behavior is acceptable.
* An error in any iteration stops the loop and fails the step (and the workflow).
## Explicit Dependencies
Use `depends_on` when you need ordering without a data dependency:
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(init_database)
.step(process_data, depends_on=["init_database"])
)
```
`process_data` waits for `init_database` even though it doesn't use its output.
## Summary
* Steps run in **parallel** by default
* Access sibling step data via `step_span()`, custom variables via `current_span()`
* Lambda parameters **automatically create dependencies** — even if the referenced step hasn't executed yet
* Use `depends_on` for explicit ordering without data dependency
* Use `while_` (int count or callable, do-while) to repeat a step; `step_span()` returns the latest iteration
# Errors & Failure
Source: https://docs.timbal.ai/workflows/errors-and-failure
How workflow steps fail, skip, and propagate errors
Workflows run steps concurrently but respect dependencies. When something goes wrong, behavior depends on whether the step **failed** (raised an error) or was **skipped** (`when` returned false, or a dependency was missing).
## Skipped vs failed
| State | Cause | Dependents |
| ----------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| **Skipped** | `when` returned `False`, or `step_span()` hit a skipped upstream step (`SpanNotFound`) | Steps that need that step's data are skipped; steps with only `depends_on` wait and may still run |
| **Failed** | Handler raised, step returned `OutputEvent` with error, or param/`when` evaluation failed | Steps that try to read the failed step's output fail during evaluation; workflow ends with `status.code == "error"` |
Skipped is intentional (branch not taken). Failed is an error you should handle or fix.
## Step failure
When a step handler raises, Timbal records the error on that step's span and marks the workflow failed:
```python theme={"dark"}
result = await workflow().collect()
# result.status.code == "error"
# result.status.reason == "step_failed"
# result.error == {"type": "ValueError", "message": "...", "traceback": "..."}
```
If multiple steps run in parallel and one fails, the workflow still reports error even if other steps succeeded.
## Dependent steps
A downstream step that reads a failed step's output via `step_span()` fails during parameter resolution (before its handler runs):
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(raises_error) # fails
.step(
process,
data=lambda: get_run_context().step_span("raises_error").output,
)
)
```
Use `step_span("name", default=None)` when a step might have been skipped (see [Control flow](/workflows/control-flow#handling-optional-steps)), not when it failed. A failed upstream step does not produce output to read.
## Skipped branches
When `when` is false, the step is skipped and dependents that **require its output** are skipped too:
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(validate_input, data="...")
.step(process, when=lambda: get_run_context().step_span("validate_input").output == "valid")
.step(save_results, data=lambda: get_run_context().step_span("process").output)
)
```
If validation fails, both `process` and `save_results` are skipped.
Steps that only need ordering (not data) can use `depends_on` to run after a branch resolves:
```python theme={"dark"}
.step(finalize, depends_on=["handle_new", "handle_existing"])
```
See [Branching](/workflows/branching#skipped-steps) for the full pattern.
## Pauses (approval / suspend)
When a step hits an [approval gate](/human-in-the-loop/approval-gates) or [`suspend()`](/human-in-the-loop/suspend), the workflow pauses with `status.code == "cancelled"` and a reason like `approval_required` or `input_required`. Parallel steps that also pause emit their events before the workflow stops, so you can resume multiple pending ids in one call.
Pauses are not failures. Resume with `parent_id` and `resume={...}` as documented in [Resuming a paused run](/human-in-the-loop/resuming).
## Cycle detection
Linking steps that would create a cycle raises immediately when you call `.step()`:
```python theme={"dark"}
# ValueError: Linking step_a -> step_b would create a cycle in the workflow.
```
Dependencies come from `depends_on`, `when` callables, and lambda parameters that call `step_span()`.
## Tracing
Every step gets its own span under the workflow span. Inspect per-step input, output, and errors via `get_run_context()._trace` or your [tracing provider](/core-concepts/tracing). Failed steps record `error` on their span even when the workflow's final status is `error`.
## See also
* [Control flow](/workflows/control-flow) — optional steps with `default=None`
* [Branching](/workflows/branching) — conditional execution
* [Human in the loop](/workflows/human-in-the-loop) — pauses on workflow steps
# Human in the Loop
Source: https://docs.timbal.ai/workflows/human-in-the-loop
Pause workflow steps for approval or user input, then resume
Human-in-the-loop works on **workflow steps** the same way it works on agents and tools. Any step wrapped in a `Tool` (or any Runnable) can use `requires_approval`, and any handler can call `suspend()`.
## Approval gates on steps
Wrap a step function in a `Tool` with `requires_approval`:
```python theme={"dark"}
from timbal import Tool, Workflow
deploy_prod = Tool(
name="deploy_prod",
handler=deploy_prod_impl,
requires_approval=True,
approval_prompt="Promote to production?",
)
workflow = (
Workflow(name="release_pipeline")
.step(deploy_staging)
.step(deploy_prod)
.step(announce, depends_on=["deploy_prod"])
)
```
When the gate fires, the workflow pauses with `status.reason == "approval_required"`. Independent gates in parallel all emit before the workflow stops, so you can approve multiple steps in one `resume={...}` call.
Full reference: [Approval gates](/human-in-the-loop/approval-gates), including edit-on-approve, redaction, time limits, and [approvals in workflows](/human-in-the-loop/approval-gates#approvals-in-workflows).
## Suspend inside a step
Handlers that call `suspend()` pause the workflow mid-step. The handler re-runs from the top on resume, so keep side effects after the `suspend()` call or make them idempotent.
Use [Suspend & interaction tools](/human-in-the-loop/suspend) for `ask_user`, `ask_user_multi`, `confirm`, and custom interaction tools.
## Resuming
Call the workflow again with `parent_id` (the paused run id) and `resume={approval_id: True}` or `resume={interaction_id: value}`. Cross-process resume needs a durable [tracing provider](/human-in-the-loop/resuming).
## See also
* [Errors & failure](/workflows/errors-and-failure) — pauses vs failures
* [Human in the Loop overview](/human-in-the-loop)
# Overview
Source: https://docs.timbal.ai/workflows/index
Build multi-step AI pipelines with explicit control flow
Workflows orchestrate **steps** in a DAG: you define what runs, Timbal infers dependencies and runs independent steps in parallel. Use them when the path is predictable (ETL, gated deploys, fan-out/fan-in, LLM routing) rather than leaving every decision to the model.
## Workflow vs Agent
| | **Workflow** | **Agent** |
| ----------- | -------------------------------------------- | ---------------------------------- |
| Control | You define steps and order | The model decides what to do |
| Best for | Pipelines, batch jobs, deterministic routing | Open-ended reasoning, tool picking |
| Parallelism | Built-in across independent steps | Sequential tool loop |
| Composition | Nest workflows, mix agents + functions | Tools and sub-agents |
They share the same interface: call a Runnable to get an event stream, or chain `.collect()` for the final `OutputEvent`. A common pattern is a **workflow that routes to agents** (classify → specialist agent). See [Agents & LLMs](/workflows/agents-and-llms).
## Reading guide
1. **[Control flow](/workflows/control-flow)** — parallel execution, `step_span()`, `depends_on`, wiring data between steps
2. **[Branching](/workflows/branching)** — conditional steps with `when`
3. **[Errors & failure](/workflows/errors-and-failure)** — what happens when a step fails or is skipped
4. **[Agents & LLMs](/workflows/agents-and-llms)** — agents as steps, LLM routing patterns
5. **[Human in the loop](/workflows/human-in-the-loop)** — approval gates on workflow steps
6. **Examples** below — copy-paste pipelines for common shapes
## Quickstart
```python theme={"dark"}
import asyncio
from timbal import Workflow
from timbal.state import get_run_context
def celsius_to_fahrenheit(celsius: float) -> float:
return (celsius * 9 / 5) + 32
def format_result(temperature: float) -> str:
return f"Temperature: {temperature}°F"
workflow = (
Workflow(name="temperature_converter")
.step(celsius_to_fahrenheit, celsius=35)
.step(
format_result,
temperature=lambda: get_run_context().step_span("celsius_to_fahrenheit").output,
)
)
async def main():
result = await workflow().collect()
print(result.output)
asyncio.run(main())
```
Functions used as steps must accept and return Pydantic-serializable types (`str`, `int`, `float`, `bool`, `dict`, `list`, `BaseModel`). Custom classes that aren't Pydantic models cannot be passed between steps.
## Adding steps
Use `.step()` to add a Runnable (function, Tool, Agent, or nested Workflow):
```python theme={"dark"}
from timbal import Tool, Workflow
# Same handler twice? Give each Tool a unique name.
threshold_high = Tool(name="threshold_high", handler=check_threshold)
threshold_low = Tool(name="threshold_low", handler=check_threshold)
workflow = (
Workflow(name="monitoring")
.step(threshold_high, value=80, limit=100)
.step(threshold_low, value=80, limit=50)
)
```
Each step name must be unique within the workflow. Timbal detects cycles when you link steps and raises if the graph would loop.
## Workflow inputs
Keyword arguments passed to the workflow are forwarded to steps. Parameters without a default on a step become **workflow-level inputs** (merged into the workflow's params schema):
```python theme={"dark"}
workflow = (
Workflow(name="scraper")
.step(fetch) # fetch(url: str) → url is a workflow input
.step(process, raw=lambda: get_run_context().step_span("fetch").output)
)
result = await workflow(url="https://example.com").collect()
```
Fixed values passed in `.step(name, key=value)` are defaults for that step only and are not exposed as workflow inputs.
## Output
The workflow returns the **last executed step's output**. If you need multiple step results, add a final merge step:
```python theme={"dark"}
workflow = (
Workflow(name="pipeline")
.step(fetch_users)
.step(fetch_orders)
.step(
build_report,
users=lambda: get_run_context().step_span("fetch_users").output,
orders=lambda: get_run_context().step_span("fetch_orders").output,
)
)
# build_report's return value is the workflow output
```
## Running
```python theme={"dark"}
# Collect final OutputEvent
result = await workflow(url="https://example.com").collect()
print(result.output)
print(result.status.code) # "success" | "error" | "cancelled"
print(result.usage)
# Or stream events from every step
async for event in workflow(url="https://example.com"):
print(event)
```
## Composition
Nest workflows as steps. The inner workflow's final output becomes that step's output:
```python theme={"dark"}
data_pipeline = (
Workflow(name="data_pipeline")
.step(fetch_data, source="api")
.step(clean_data, raw=lambda: get_run_context().step_span("fetch_data").output)
)
report_pipeline = (
Workflow(name="report_pipeline")
.step(data_pipeline)
.step(generate_report, data=lambda: get_run_context().step_span("data_pipeline").output)
)
```
See [Workflow Composition](/examples/workflows/workflow-composition) for a full example.
## Examples
Chain steps with data passing between them
Fetch from multiple sources concurrently, then merge
Route to different handlers based on validation
Nest an inner workflow as a step