Speech events

Agents can respond to the user's speech activity during a voice conversation:

Barge-in – whether the user can interrupt the agent while it is speaking.
No user input – reacting when the user stays silent.

Barge-in

Barge-in lets the user interrupt the agent while it is speaking. When enabled, the agent stops playing its current response as soon as the user starts talking and begins processing the new input. When disabled, the agent finishes playing its response before it starts listening to the user.

Barge-in applies to all models. For regular (non-speech-to-speech) models, it is controlled by the Barge-in toggle in the Settings tab of the Bot connection's configuration screen, or in the Speech and Telephony tab in the Agent's configuration screen (similar to the DTMF toggle). For speech-to-speech models, barge-in is handled by the model itself.

Barge-in can also be controlled at a finer granularity:

Override it for a specific Conversation node, or set it globally in Global flow settings.
Configure it during welcome message playback via the welcome_message_barge_in parameter, or per part via welcome_message_parts – see Call settings during welcome message.
Change it dynamically during the call via the bargeIn setting in activity_params or session_params – see Call settings parameters configuration.
Let the agent turn it on or off through a tool, using session_params_tools – see Tools for modifying call settings.

No user input

Note: This event applies only to regular (non-speech-to-speech) models. For user-silence handling with speech-to-speech models, use Inactivity reminder instead.

When no user input is received within a configurable amount of time, the Bot connection associated with the AI Agent generates a NoUserInput event. This is typically used to prevent awkward silences in the conversation, or to detect a connectivity problem in the voice path.

To enable the event, add the following to your Bot connection's Advanced configuration. Adjust the userNoInputRetries parameter value to your needs.

{
    "userNoInputTimeoutMS": 5000,
    "userNoInputRetries": 2
}

AI Agents convert the NoUserInput event into a "fake" NO-USER-INPUT-[X] user utterance sent to the LLM, where [X] is the retry number starting from 1. The LLM responds according to the instructions in the agent's prompt.

Include explicit instructions in your prompt on how to react to NO-USER-INPUT-[X] messages.

Example

Respond to "NO-USER-INPUT-1" with "Sorry, are you still there?".

Respond to "NO-USER-INPUT-2" with calling `end_call` tool and saying "Sorry, I can't hear you".

Testing in Chat

Because the no-user-input event is delivered as a "fake" user utterance, you can test how your agent reacts to it directly in Chat – simply type the utterance (for example, NO-USER-INPUT-1) and inspect the agent's response.