STT, TTS, and VAD settings are stored in the flow and applied to calls. Providers are selected from the configured provider directories. Provider keys are stored in the environment configuration, while self-hosted endpoints do not require a provider key.
STT (speech-to-text) converts the caller’s voice into text for the agent.
ASR model from directory — the recognition model is picked by reference to an entry of the ASR model directory, with its Provider shown beside it. A model marked deprecated is labelled as such right in the field.
language (empty = flow language; kk → multi) — the recognition language. Empty inherits the agent’s language; kk turns on the multilingual mode.
interim_results — live interim transcript — the transcript arrives as the caller speaks rather than in one piece at the end. Batch models have no interim results: the utterance is recognized whole once the caller stops.
keywords (comma-separated, weight — “word:1.5”) — words the recognizer should hear more accurately; the weight goes after a colon.
Provider-specific parameters (JSON) — everything else a given provider supports: stability and similarity on ElevenLabs, emotion on Cartesia, instructions on OpenAI.
Emergency fallback TTS → Enable fallback provider — used only when the primary synthesis fails or hits a limit. Switching provider changes how the agent sounds, so the switch is marked on the call.
TTS audio cache — repeated phrases are served from the cache without calling the provider. TTS cache — advanced settings below it holds Entry TTL, sec and Reset the cache on publish.
VAD (voice activity detection) controls when speech starts and ends, while interruption settings control whether the caller can cut the agent off. The settings are read at call start; nothing needs rebuilding.
prefix_padding_duration, sec — audio pre-buffer — how much audio is kept ahead of the detected speech.
sample_rate (8000/16000) — the audio sample rate.
max_buffered_speech, sec — the ceiling on buffered speech.
allow_interruptions — allow interrupting the agent.
Loudness gate (noise does not count as speech) — a second check: a quiet frame does not start an utterance even when the model calls it speech. While the gate is off, the loudness thresholds in the cards below do not apply.
min_interruption_duration, sec — min. speech to interrupt.
min_interruption_words — min. recognized words to interrupt (0 — no word gate) — how many words the caller must say before the agent stops talking. The default is 2.
min_duration_all_branches — the duration threshold applies to every interruption, not only mid-speech.
min_endpointing_delay, sec and max_endpointing_delay, sec — the wait before a turn is committed.
VAD runtime configuration unfolds below it — the same thresholds, but per agent state: a When the agent is speaking or thinking card and an Other states card. Each holds activation_threshold, deactivation_threshold, min_speech_duration, min_silence_duration, activation_volume and deactivation_volume. An empty field in the first card means “as in the other states”; the set of rules itself is fixed in code, only the numbers change.
The word gate has a cost, and it is worth knowing before you tune it. At the default of 2, one- and two-word replies do not interrupt the agent — and that includes “stop” and “no”. Those words are not lost: a short reply below the threshold is kept and joined with the caller’s next phrase, so the agent reacts a beat later rather than mid-sentence. Set the value to 0 if your scenario needs a single word to cut the agent off immediately, and accept that a cough, an “uh-huh” or a “yeah” will do the same.With a speech recognizer that does not stream partial results, the word count can only be checked at the caller’s first pause — so on such a setup the gate reacts later than it would with streaming recognition.
Two more sections sit beside it in the voice block: Turn detection (end of turn), where the Mode is chosen (auto, VAD pause only, the audio detector or the STT provider), and Backchannel, with its trigger frequency and word list.
An ambient recording — office noise, say — is mixed under the agent’s voice and makes the conversation feel more natural. It loops seamlessly for the length of the call and does not affect recognition: the background is never analyzed as speech, so neither voice detection nor the transcript change because of it.
Enable background audio.
Audio asset (background library) — pick a recording from the library; Upload audio… (wav/mp3/ogg/opus) adds a new one. If the selected asset has been deleted from the library, the dashboard says so — the background will not play.
Background volume — how loud it is mixed against the agent’s voice.
If the selected asset fails to load, the call continues normally without background audio.
Detects the language of each turn and routes it to speech synthesis and analytics. Useful for bilingual and multilingual agents.
Detect turn language — off by default.
Source — where the language comes from:
LLM (dynamic_language variable — no extra calls) — the language comes from a dynamic variable of the flow itself. The recommended option: it makes no additional requests.
Classifier service (URL) — an external service decides on every turn; its address goes in the same field.
Candidate languages (comma-separated) — what the detector is allowed to choose from.
The detected language is used for text-to-speech, so the voice speaks in the right language, and is recorded on the call as a turn_language event for later review.
