Skip to main content
All built-in transports create the same transcription → agent → speech session. Speech credentials stay on the server. Use an Agent runnable and install timbal[server,voice]; LiveKit also needs timbal[voice-livekit]. First get a conversation working in the quickstart. Then choose one transport below. Browser-local thinking sounds and background audio are not automatically included in a carrier’s audio stream; spoken fillers are part of the session’s speech output.

Concurrent calls

Voice admission is unbounded by default, including per-request LiveKit calls. Configure TIMBAL_VOICE_MAX_CONCURRENT_SESSIONS at server startup to opt into a per-process limit:
  • Unset, empty, or 0: no admission cap.
  • A positive integer, such as 16: admit up to that many concurrent sessions.
  • auto: derive the cap from available CPU, worker count, the server’s voice configuration, and recording cost.
With a cap enabled, excess calls are rejected; a LiveKit dial returns HTTP 503. Sessions are still counted when admission is unbounded. Custom hosts that bypass timbal.server.http.lifespan should call configure_capacity(voice_config) and log_capacity() from timbal.server.capacity once at startup with the server’s resolved voice configuration. active_sessions() and max_concurrent_sessions() expose the process’s active count and effective cap (0 means uncapped).

Browser playground

Open /voice on the running server. The playground exposes provider and model selection, microphone controls, transcripts, and turn metrics. GET /voice/meta describes the loaded runnable and its declared configuration. GET /voice_config returns {"voice_config": ...} with portable declared settings; it is not the fully resolved environment configuration.

WebSocket clients

Connect to /voice/ws and send a JSON configuration hello without a type field:
The server waits up to two seconds for the hello. A binary first frame starts immediately with defaults. Send mono PCM16 little-endian chunks as binary frames, or as base64 JSON:
The server sends speech as {"type": "audio", "data": "..."} in the same PCM format. Decode and queue it for playback. Report cumulative milliseconds of output audio actually played:
Count playback, not bytes received or merely scheduled. On interrupted, stop playback and discard queued audio. The session uses playback acknowledgements to align the caller’s heard reply with conversation memory; without them it estimates the position from the buffered playback schedule. Closing the socket closes the session. Other JSON messages include session_started, transcript_partial, transcript_committed, agent_status, agent_text_delta, agent_text_done, agent_approval, agent_interaction, filler, metrics, voice_usage, interrupted, error, event_turn_started, notify_ack, session_transcript, and session_ended. agent_status reports tool activity. A committed transcript with replace=true replaces the previous user transcript bubble rather than appending another. session_started includes session_id, stt_provider, stt_model, tts_provider, model (the text model), turn_detector (the resolved class name, such as LocalAudioTurnDetector), direction, ambient, and parent_run_id when the call continues a text run. The transport adds transport and playback_acks. A hello may also send parent_id to continue a text run in this call. The server waits up to two seconds for that hello on the WebSocket (hello_wait_secs). A second WebSocket on a single-session server is closed with code 1008. A second WebRTC offer gets HTTP 409. Voice on a non-Agent runnable is 1008 on the socket and 400 on /voice/rtc. WebRTC without the voice extra returns 501. At the concurrency cap the dial returns 503. A missing DEEPGRAM_API_KEY, or an unknown provider id, falls back to ElevenLabs for that session instead of failing the handshake. mic_change is accepted on the socket and is not required for a working client.

WebRTC

Create a peer connection with a microphone audio track and a data channel before creating the offer. Complete ICE gathering, then send:
Apply the returned SDP answer. The server completes its ICE gathering before replying; this route does not use trickle ICE. Speech arrives on an audio track. Session JSON events arrive on the data channel, with no base64 audio messages and no client playback acknowledgements. Playback tracking uses the server’s paced media clock. Configure TIMBAL_STUN_URL and, when needed, TIMBAL_TURN_URL, TIMBAL_TURN_USERNAME, and TIMBAL_TURN_PASSWORD for your network. TIMBAL_VOICE_RTC_FORCE_RELAY=1 filters to relay candidates only when TURN is configured. The route returns 501 if WebRTC dependencies are unavailable.

LiveKit

Install the LiveKit extra:
Create a room and an agent join token through your LiveKit integration, then ask a long-lived Timbal server to join it:
Set TIMBAL_VOICE_DIAL_SECRET to validate this header and TIMBAL_LIVEKIT_URL to restrict the destination. Clients join and publish microphone audio using LiveKit’s client SDK. Audio uses room tracks; session events use the timbal.events data topic. hello_wait_secs controls the browser configuration-hello window (default two seconds), and sip_hello_wait_secs controls the SIP caller window (default zero). Phone callers normally have no data channel to send a hello. A session holds its capacity slot and room until it ends. On a long-lived server it ends when:
  • the room is deleted or the LiveKit connection is lost, before or after the session starts;
  • the caller leaves and does not rejoin within TIMBAL_VOICE_ABANDON_SECS (default 45; SIP hangups end immediately);
  • the caller’s microphone does not arrive within TIMBAL_VOICE_CALLER_WAIT_SECS of the agent joining (default 120; 0 waits indefinitely).
The separate boot-environment path uses TIMBAL_VOICE_TRANSPORT=livekit, TIMBAL_LIVEKIT_URL, and TIMBAL_LIVEKIT_TOKEN to join one room at startup. Reserve that path for a process created for a single call; long-lived deployments should dial per request.

Twilio and Telnyx

Expose the server through HTTPS with WebSocket upgrades and configure the carrier’s voice webhook:
  • Twilio: POST https://YOUR_HOST/voice/twilio/incoming.
  • Telnyx: POST https://YOUR_HOST/voice/telnyx/incoming.
The webhook returns TwiML or TeXML connecting a bidirectional media stream to /voice/twilio/stream or /voice/telnyx/stream. The bridge decodes carrier 8 kHz G.711 μ-law audio into the session format and converts speech back to the carrier format. Interruption clears carrier playback; mark echoes track playback position. Set TWILIO_AUTH_TOKEN or TELNYX_PUBLIC_KEY to enable incoming-webhook signature verification. Without these variables, the webhook accepts requests with a development-mode warning. Configure your reverse proxy so the externally visible URL used by the carrier is the URL validated by the webhook. The routes answer and bridge calls. Placing an outbound call requires your carrier integration to originate it and connect the media stream. A host that knows the call direction can select outbound_greeting; setting the greeting alone does not originate a call.

Approval and input requests

Voice sessions forward agent_approval and agent_interaction events when the agent pauses. Render the request and resume through POST /stream with the event’s run_id as parent_id and the decision in resume. Speaking into the microphone is not an approval resolution. See Client integration. An agent_text_done event also carries the agent run_id, allowing an authorized application to continue that conversation through the text endpoint. Identity and prior-run continuation require host-controlled authorization; they are separate from speech tuning in the hello.

Reporting events

On WebRTC and LiveKit, a client can tell the agent that something happened, such as a background task finishing, so it speaks up without waiting for the caller. Send this on the data channel (LiveKit: the room data channel):
key is optional; a newer event with the same key replaces one that has not been delivered. The agent waits for a quiet line. An optional announce picks how it speaks up: "when_idle" (default), "silent" (only on the caller’s next turn), or "interrupt" (cut in now), which only your backend may send; see Choosing what speaks up. The server answers at once with the event’s receipt, or a rejection:
Rejection reasons: not_enabled (client events are off), blocked (a guardrail refused the text), not_allowed (announce: "interrupt" from the caller’s device), invalid (no text, too long, or an unknown announce), and closed (the call has ended). Text from the caller’s device is untrusted. The server accepts it only when the agent’s voice_config sets "events": {"client_notify": true} (the hello cannot), runs it through the agent’s input guardrails, and labels it for the model as information rather than instructions. On LiveKit, frames from other participants are ignored, and frames the room server API sends (SendData, no participant) come from your backend and need no opt-in. Server-side code calls VoiceSession.notify directly; see Using tools. When the agent takes a turn of its own, for an event or an announced background result, the client receives this before the turn’s text and audio: