Speech events
Agents can respond to the user's speech activity during a voice conversation:
- Barge-in – whether the user can interrupt the agent while it is speaking.
- No user input – reacting when the user stays silent.
Barge-in
Barge-in lets the user interrupt the agent while it is speaking. When enabled, the agent stops playing its current response as soon as the user starts talking and begins processing the new input. When disabled, the agent finishes playing its response before it starts listening to the user.
Barge-in applies to all models. For regular (non-speech-to-speech) models, it is controlled by the Barge-in toggle in the Settings tab of the Bot connection's configuration screen, or in the Speech and telephony tab in the Agent's configuration screen (similar to the DTMF toggle). For speech-to-speech models, barge-in is handled by the model itself.
Barge-in can also be controlled at a finer granularity:
- Override it for a specific Conversation node, or set it globally in Global flow settings.
- Configure it during welcome message playback via the
welcome_message_barge_inparameter, or per part viawelcome_message_parts– see Call settings during welcome message. - Change it dynamically during the call via the
bargeInsetting inactivity_paramsorsession_params– see Call settings parameters configuration. - Let the agent turn it on or off through a tool, using
session_params_tools– see Tools for modifying call settings.
Interrupted responses
When the user interrupts a response, the Large Language Model can be told how much of it the user actually heard. The response in the conversation history is cut down to the part that was played, followed by a note that the rest was not heard – so the model does not assume the user heard an option it never got to, or the question its response ended with.
For example, if the agent says "Which area suits you? North, center or south." and the user interrupts after the question, the next LLM call sees:
Which area suits you? [interrupted - user did not hear the rest]
Enable it by adding the interrupted_response parameter to the Advanced tab of the Agent's or Flow's dialog:
{
"interrupted_response": {
"mode": "enabled"
}
}
This applies to Agents and Flows on regular (non-speech-to-speech) models. Speech-to-speech models handle interruptions themselves.
How precisely the cut follows what was played depends on the text-to-speech provider: with providers that report word timings it falls on the last word played; with others it is estimated from how long the response played. When the Bot connection cannot tell how much of a response was played – for example, a response that was still waiting to be played when the user interrupted – the response is kept whole.
Each cut is recorded as an interrupted_response entry in the conversation log, showing what the user heard.
| Parameter | Type | Description |
|---|---|---|
interrupted_response
|
InterruptedResponse | What the model is told of a response the user interrupted. |
InterruptedResponse
| Parameter | Type | Description |
|---|---|---|
mode
|
enum | Supported values: • disabled – the response is kept whole (default)• enabled – the response is cut down to the part the user heard |
note
|
string | Text that follows the part the user heard. An empty string adds no note. Default = [interrupted - user did not hear the rest] |
In a multi-agent topology, a sub-agent without its own interrupted_response follows the root agent's.
No user input
Note: This event applies only to regular (non-speech-to-speech) models. For user-silence handling with speech-to-speech models, use Inactivity reminder instead.
When no user input is received within a configurable amount of time, the Bot connection associated with the AI Agent generates a NoUserInput event. This is typically used to prevent awkward silences in the conversation, or to detect a connectivity problem in the voice path.
To enable the event, add the following to your Bot connection's Advanced configuration. Adjust the userNoInputRetries parameter value to your needs.
{
"userNoInputTimeoutMS": 5000,
"userNoInputRetries": 2
}
AI Agents convert the NoUserInput event into a "fake" NO-USER-INPUT-[X] user utterance sent to the LLM, where [X] is the retry number starting from 1. The LLM responds according to the instructions in the agent's prompt.
Include explicit instructions in your prompt on how to react to NO-USER-INPUT-[X] messages.
Example
Respond to "NO-USER-INPUT-1" with "Sorry, are you still there?".
Respond to "NO-USER-INPUT-2" with calling `end_call` tool and saying "Sorry, I can't hear you".
Testing in Chat
Because the no-user-input event is delivered as a "fake" user utterance, you can test how your agent reacts to it directly in Chat – simply type the utterance (for example, NO-USER-INPUT-1) and inspect the agent's response.