Your First Voice Agent

Install

Use Python 3.11 or newer and install the server and voice extras:

pip install "timbal[server,voice]"

Or add them to a uv project:

uv add "timbal[server,voice]"

The server extra provides HTTP and WebSocket support. The voice extra adds WebRTC and local turn detection. Local detection models are downloaded on first use; the first session can take longer while they load.

Configure credentials

Create a .env beside agent.py:

.env

OPENAI_API_KEY=your_openai_key
DEEPGRAM_API_KEY=your_deepgram_key
ELEVENLABS_API_KEY=your_elevenlabs_key
ELEVENLABS_VOICE_ID=your_accessible_voice_id

The example uses OpenAI for the text agent, Deepgram for transcription, and ElevenLabs for speech. Without DEEPGRAM_API_KEY, the server transcribes with ElevenLabs instead. Choose a voice accessible to your ElevenLabs account. Credentials remain on the server.

Create the agent

agent.py

from timbal import Agent


agent = Agent(
    name="voice_assistant",
    model="openai/gpt-4.1-mini",
    system_prompt=(
        "You are a helpful voice assistant. Reply in one or two short spoken "
        "sentences. Ask one question at a time."
    ),
    voice_config={"language": "en"},
)

voice_config configures the server’s speech session. The defaults are Deepgram Flux (flux-general-multi) for transcription and for deciding when your turn ends, and ElevenLabs eleven_v4_turbo for speech. It can be a dict, a VoiceConfig, or a callable returning configuration. Unknown keys fail validation at server startup.

Start and speak

python -m timbal.server --import_spec agent.py::agent --host 127.0.0.1 --port 4444

With uv, use uv run python -m timbal.server with the same arguments. The server loads .env from the working directory. TIMBAL_RUNNABLE=agent.py::agent can supply the import spec instead of the flag. Open http://127.0.0.1:4444/voice, allow microphone access, and start a session. Say “Hello, what can you help me with?” The agent should transcribe your words and speak a short reply. Ask a follow-up question to try a conversation. A browser microphone requires localhost or HTTPS. The agent waits for you to speak first. This example has no greeting, tools, fillers, or background audio. Leave the playground’s thinking sound and ambience controls off for this first conversation. The agent is also available at POST /run and POST /stream. Voice routes require an Agent runnable; a Workflow or standalone Tool cannot be used as the voice conversation runnable.

Next: let the agent say hello

Keep this agent.py, its .env, and the same server command for the following lessons. After changing the file, restart the server and start a new browser session. Continue to Greetings to make the agent speak first, then Using tools to give it a useful action. You can change speech providers later in Providers and configuration.

Getting started

Core Concepts

Agents

Voice Agents

Workflows

Human in the Loop

Evals

Deployment

Your First Voice Agent

Install

Configure credentials

Create the agent

Start and speak

Next: let the agent say hello

​Install

​Configure credentials

​Create the agent

​Start and speak

​Next: let the agent say hello

Install

Configure credentials

Create the agent

Start and speak

Next: let the agent say hello