Gender detection

An agent or flow can learn whether it is talking to a man or a woman and address the caller accordingly. This matters in languages where the words used to address someone depend on their gender, such as Hebrew or Arabic.

The gender is detected from the caller's voice a few seconds into the call. Each detection is written to the conversation log as a gender_detected entry.

Gender detection is currently supported for agents and flows that use regular (not speech-to-speech) models.

Enabling gender detection

Gender detection requires a special audio processor on the Bot connection, which currently only AudioCodes support can enable. Contact us as described on the Support page, describe your use case, and provide the ID of the Bot connection that needs gender detection.

Then add the following to your agent's (or flow's) Advanced configuration:

{
    "user_gender_detection": {
        "enable": true
    }
}

Adapting to the detected gender

An agent or flow can adapt to the detected gender in two ways:

• In the text the LLM generates. The LLM is told the detected gender and phrases its replies accordingly. In Hebrew, masculine and feminine forms are often spelled the same without diacritics, so you may need to instruct the agent to add them (nikkud) for the text-to-speech engine to pronounce the right form.
• In how the text-to-speech engine speaks. The engine is told the gender and pronounces the text accordingly, as described in Passing the gender to the text-to-speech engine.

Passing the gender to the text-to-speech engine

Some text-to-speech engines accept the caller's gender and adjust their speech synthesis accordingly. The following vendors currently support this:

• Deepdub

If you use one of these engines, enable configure_tts in your configuration:

{
    "user_gender_detection": {
        "enable": true,
        "configure_tts": true
    }
}

Learning the gender from the conversation

An agent can also learn the caller's gender from what they say — for example, when they state it or ask to be addressed differently. With configure_tts enabled, the agent then updates the text-to-speech engine too, so the synthesized speech stays correct. Flows do not support this: they learn the gender from the voice only.

When the voice and the conversation disagree, the most recent of the two applies, and your prompt decides which should give way. To let the caller have the last word, add a line such as:

When a voice detection contradicts what the caller told you about themselves, set the gender again to what they said.

To make the voice final instead, tell the agent not to change the gender once a voice detection was reported to it.

Adjusting when the agent sets the gender

With configure_tts enabled, the agent uses a tool to set the caller's gender in the text-to-speech engine. The tool's default description suits almost every use case, and we recommend keeping it.

If your use case needs a different tool description, you may adjust it via the tool_description parameter.

{
    "user_gender_detection": {
        "enable": true,
        "configure_tts": true,
        "tool_description": "Set the gender of the person you are speaking to. Call this only when the caller explicitly asks to be addressed as a man or as a woman."
    }
}