> ## Documentation Index > Fetch the complete documentation index at: https://docs.timbal.ai/llms.txt > Use this file to discover all available pages before exploring further. # List Eval Definitions > List eval definitions (test cases from evals/*.yaml) per workforce component. Reads the env branch's worktree by default; pass ?rev= to read from git. ## OpenAPI ````yaml GET /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/definitions openapi: 3.1.0 info: title: Timbal Platform API description: Public API documentation for the Timbal platform license: name: '' version: 2.0.0 servers: - url: https://api.timbal.ai description: Production - url: https://api.dev.timbal.ai description: Development security: [] tags: - name: ace description: Action Control Engine (ACE) operations - name: analytics description: Org and project analytics - name: billing description: Subscriptions, usage, and payment-related operations - name: content description: Re-sign stored content URLs - name: files description: File operations - name: iam description: IAM — actions, roles, users, and authorization introspection - name: k2 description: Knowledge Bases v2 operations - name: orgs description: Organization operations - name: projects description: Project operations - name: runs description: Execution operations - name: templates description: Public project template catalog - name: users description: Authenticated user profile paths: /orgs/{org_id}/projects/{project_id}/envs/{env_id}/evals/definitions: get: tags: - projects description: >- List eval definitions (test cases from evals/*.yaml) per workforce component. Reads the env branch's worktree by default; pass ?rev= to read from git. operationId: handler parameters: - name: org_id in: path required: true schema: type: string - name: project_id in: path required: true schema: type: string - name: env_id in: path required: true schema: type: string - name: rev in: query description: |- Git rev to read definitions from (branch, tag, or SHA). When set, definitions come from git at that rev instead of the worktree. required: false schema: type: - string - 'null' - name: component_id in: query description: |- Filter to a single workforce component by its numeric app id (returned as `component_id`). Combines with `component_uid`: when both are set, components must match both. required: false schema: type: - string - 'null' - name: component_uid in: query description: |- Filter to a single workforce component by its manifest uid (the `_id` field in `timbal.yaml`, returned as `component_uid`). required: false schema: type: - string - 'null' responses: '200': description: Eval definitions listed successfully content: application/json: schema: $ref: '#/components/schemas/ListEvalDefinitionsResBody' '400': description: Unknown rev '403': description: Insufficient permissions security: - bearer_auth: [] components: schemas: ListEvalDefinitionsResBody: type: object required: - source - rev - commit_hash - components properties: commit_hash: type: string description: |- Head commit of that rev. With `source = "worktree"`, uncommitted file changes are NOT part of this commit — check per-file `committed` flags. components: type: array items: $ref: '#/components/schemas/ComponentEvals' rev: type: string description: Branch (worktree) or rev (git) the definitions were read from. source: type: string description: 'Where definitions were read from: `worktree` or `git`.' ComponentEvals: type: object description: A workforce component and the evals defined in its `evals/` dir. required: - component_dir - evals properties: component_dir: type: string description: Component directory name under `workforce/`. component_id: type: - string - 'null' description: |- `OrgsApps.id`, when the repo component matches a registered app. Use as `component_id` for `POST .../evals/runs`. component_name: type: - string - 'null' description: App name in the platform, when registered. component_uid: type: - string - 'null' description: |- `_id` from the component's `timbal.yaml` (`component_uid` for `POST .../evals/runs`). evalconf: description: |- Parsed `evalconf.yaml` (from `evals/` or the component root), when present. Its `runnable` key is the per-component default. evals: type: array items: $ref: '#/components/schemas/EvalDefinition' parse_errors: type: array items: type: string description: Files that exist but failed to parse as eval YAML (path → error). warnings: type: array items: type: string description: |- Non-fatal issues the CLI would hit: duplicate eval names (hard error, fails the gate), missing names, unknown validator keywords, missing runnable. EvalDefinition: type: object description: |- One eval test case parsed from a YAML file. Field-for-field mirror of `timbal.evals.models.Eval` plus provenance. required: - tags - validators - file - committed - spec properties: committed: type: boolean description: |- Whether the file content is committed at the branch head. Always true when reading from git; from the worktree, false means the deploy gate won't see this version until it's committed (manual runs default to the worktree and will). description: type: - string - 'null' effective_runnable: type: - string - 'null' description: |- `runnable` with the `evalconf.yaml` fallback resolved — what the CLI will actually load. Null means the CLI has nothing to run and the eval will error. env: description: Env var overrides applied to the eval subprocess (`env` key). file: type: string description: Repo-relative path of the YAML file this eval lives in. name: type: - string - 'null' description: Eval name. Required by the CLI — a missing name fails the run. params: description: Input params passed to the runnable (`params` key), e.g. `prompt`. runnable: type: - string - 'null' description: |- FQN of the runnable under test as written in this entry, e.g. `../agent.py::agent`. Null when the entry relies on the `evalconf.yaml` default. spec: description: The full raw eval entry as parsed YAML (lossless ground truth). tags: type: array items: type: string timeout: type: - number - 'null' format: double description: Per-eval timeout in ms, when set. validators: type: array items: $ref: '#/components/schemas/ValidatorSummary' description: |- Flattened assertions (value + flow validators) for display. The authoritative form stays in `spec`. ValidatorSummary: type: object description: One assertion extracted from an eval entry, flattened for display. required: - target - name - value properties: name: type: string description: |- Validator keyword as written, e.g. `contains!`, `seq!`, `not_semantic!`. target: type: string description: |- Dot-path of what's being validated, relative to the eval root — e.g. `output`, `get_datetime.input.timezone`. Empty for validators attached directly to the root (e.g. a top-level `seq!`). value: description: The expected value / steps list, as written in the YAML. securitySchemes: bearer_auth: type: http scheme: bearer description: >- Timbal API key. Obtain your API key from the Timbal platform settings. See [Authentication](/api-reference/authentication) for more information. ````