Events during voice conversation

Agents support handling of the following events during voice conversations:

• No user input

• DTMF digits

• Answering machine detection

Events are generated by a Bot connection associated with AI Agent and typically require some configuration in the Bot connection’s configuration screen, or in the Speech and Telephony tab in Agent’s configuration screen.

AI Agents convert received events into “fake” user utterances, as described below, and pass them to the LLM. The latter responds to them according to the instructions in agent’s prompt.

This mechanism makes it very simple to check your agent’s behavior via Chat, simply type the “fake” user utterance (e.g. “NO-USER-INPUT-1”) and inspect your agent’s response.

No user input

Bot connection can generate a NoUserInput event when no user input is received within specific amount of time. It is typically used to prevent awkward silences in the conversation or detect connectivity problem in the voice path.

To enable the event, add the following to your Bot connection's Advanced configuration. Adjust userNoInputRetries parameter value to your needs - see below.

{
    "sendEventsToBot": [
        "noUserInput"
    ],
    "userNoInputTimeoutMS": 5000,
    "userNoInputRetries": 2
}

AI Agents convert the NoUserInput event to NO-USER-INPUT-[X] message sent to LLM, where [X]is retry number starting from 1.

Include explicit instructions in your prompt on how to react to NO-USER-INPUT-[X] messages.

Example

Respond to "NO-USER-INPUT-1" with "Sorry, are you still there?".

Respond to "NO-USER-INPUT-2" with calling `end_call` tool and saying "Sorry, I can't hear you".

DTMF digits

Bot connections can generate DTMFDigits events when DTMF digits are detected. This may be used to create legacy IVR-like bots or to implement a shortcut for transfer to human agent.

To enable a DTMFDigits event, turn on the DTMF toggle in Settings tab in Bot connection’s configuration screen, or in Speech and Telephony tab in Agent’s configuration screen.

By default, the Bot connection sends each digit as a separate DTMFDigits events. If you want Bot connection to collect multiple digits and send them in a single DTMFDigits event, see Voice AI connet > Bot integration > Receiving notifications > DTMF digits for configuration details.

AI Agent converts DTMFDigits event to DTMF-[digits] message sent to LLM, where [digits] are collected digits.

Include explicit instructions in your prompt on how to react to DTMF-[digits] messages.

Example

If user says "DTMF-1" pass the call to helpdesk agent.
If user says "DTMF-2" pass the call to escalation agent.

Answering machine detection

A Bot connection can generate a MachineDetection event when it detects an answering machine. This is primarily relevant for outbound calls and can be used to prevent the agent from wasting time and outbound calling credits.

To enable a MachineDetection event, add the following to your Bot connection's Advanced configuration.

{
    "machineDetection": "detect"
}

AI Agent converts MachineDetection event to VOICEMAIL-DETECTED message sent to LLM.

Include explicit instructions in your prompt on how to react to VOICEMAIL-DETECTED messages.

Example

Respond to "VOICEMAIL-DETECTED" with calling `end_call` tool and saying "Hi, this is Adelle from City Hall. I just wanted to let you know that your application for the pool renovation has been approved! Give me a call back when you get a chance, and I'll go over the details with you. Thanks!"

To disconnect the call upon answering machine detection, set machineDetection property to disconnect. The disconnect will be handled by the Bot Connection, with no AI Agent / LLM involvement.

For more information, see Voice AI Connect > Bot integration > Outbound calling.