Active listening mode

Active listening mode refers to the AI Agent's ability to process input while the user is speaking instead of waiting for final speech recognition. This can be used to improve agent responsiveness when using standard textual (non-speech-to-speech) Large Language Models.

When active listening mode is active, the AI Agent uses hypotheses from the STT engine to generate LLM responses while the user speaks. In most cases the last hypothesis is very close to the final recognition – thus allowing the AI Agent to use a pre-generated LLM response and shortening the end-to-end response latency.

Note that active listening mode consumes additional LLM tokens – therefore you are essentially trading tokens for lower latency.

Enable it by adding the active_listening parameter to the Agent's or Flow's advanced configuration screen, setting mode to the value that matches your STT engine (see the ActiveListening table below):

{
    "active_listening": {
        "mode": "every_hypothesis"
    }
}

Where it applies:

• Agents – on any regular (textual) model. The one exception is the experimental responses_stateful setting, which keeps conversation state on OpenAI's servers and therefore cannot generate speculative responses.
• Flows – on conversation nodes, generated with that node's own model (so if a node uses a custom LLM, that model is used).
• Speech-to-speech (realtime) models – not supported. These models consume the caller's audio directly, so there are no STT hypotheses to work with.

In addition to mode you may customize the behavior with the parameters below.

Parameter	Type	Description
active_listening	ActiveListening	Improve agent responsiveness by generating LLM responses based on STT hypotheses.

ActiveListening

Parameter	Type	Description
mode	enum	Active listening mode. Supported values: • disabled – active listening is off (default) • every_hypothesis – process every hypothesis received from STT; recommended for STTs that do not include punctuation in hypotheses (e.g. Azure STT) • end_of_sentence – process hypotheses ending with sentence punctuation (.!?) and "eager end of turn" events; recommended for STTs that include punctuation in hypotheses (e.g. Deepgram Nova 3) • eager_end_of_turn – process only "eager end of turn" events (currently supported by Deepgram Flux only)
max_parallel	int	Maximum number of parallel response generations. Default = 3
hypothesis_interval_ms	int	Minimum time between hypotheses in milliseconds that trigger response generation. Default = 100 msec
similarity_threshold	float	Similarity threshold between final recognition and last hypothesis (0.0 to 1.0) Default = 0.9
logs	bool	Write an active_listening entry to the conversation log when a pre-generated response is consumed (see Monitoring active listening below). Default = true

Monitoring active listening

Whenever the AI Agent successfully reuses a pre-generated response, it records an active_listening entry in the conversation log. The entry reports:

• the exact STT hypothesis text that was used to generate the response;
• how much response latency was saved (in milliseconds);
• the similarity score between the last hypothesis and the final recognition.

These entries let you measure how often active listening shortens the response latency and by how much, helping you decide whether the additional token cost is worthwhile. Set logs to false to suppress them.