Speech customization

Defines the language (e.g., "en-ZA" for South African English) of the bot conversation and is used for speech-to-text and text-to-speech functionality. The value is obtained from the service provider.

• Speech-to-text:

  • Azure: The parameter is configured with the value from the 'Locale' column in Azure's Speech-Text table (e.g., "en-GB").

  • Google: The parameter is configured with the value from the 'languageCode' (BCP-47) column in Google's Cloud Speech-to-Text table (e.g., "nl-NL").

• Text-to-speech:

  • Azure: The parameter is  configured with the value from the 'Locale' column in Azure's Text-to-Speech table (e.g., "it-IT").

  • Google: The parameter is configured with the value from the 'Language code' column in Google's Cloud Text-to-Speech table (e.g., "en-US").

  • AWS: The parameter is configured with the value from the 'Language' column in Amazon's Polly TTS table (e.g., "de-DE").

Note:

• This string is obtained from the speech-to-text and text-to-speech service provider and must be provided to AudioCodes, as discussed in Text-to-speech service information and Speech-to-text service information.

• If different languages are used for the text-to-speech and speech-to-text services, use the ttsLanguage and sttLanguage parameters instead.

Defines the language (e.g., "en-ZA" for South African English) of the bot conversation and is used for the speech-to-text service.

The parameter is required if different languages are used for the text-to-speech and speech-to-text services. If these services use the same language, you can use the language parameter instead.

The value is obtained from the service provider.

Defines the language (e.g., "en-ZA" for South African English) of the bot conversation and is used for the text-to-speech service.

The parameter is required if different languages are used for the text-to-speech and speech-to-text services. If these services use the same language, you can use the language parameter instead.

The value is obtained from the service provider.

Defines the voice name for the text-to-speech service.

• Azure: The parameter is configured with the value from the 'Short voice name' column in Azure's Text-to-Speech table (e.g., "it-IT-ElsaNeural").

• Google: The parameter is configured with the value from the 'Voice name' column in Google's Cloud Text-to-Speech table (e.g., "en-US-Wavenet-A").

• AWS: The parameter is configured with the value from the 'Name/ID' column in Amazon's Polly TTS table (e.g., "Hans").

• Almagu: The parameter is configured with the value from the 'Voice' column in Almagu's TTS table (e.g., "Osnat").

Note: This string is obtained from the text-to-speech service provider and must be provided to AudioCodes, as discussed in Text-to-speech service information.

Defines a case-sensitive regular expression (regex) that determines which voiceName values VoiceAI Connect blocks. If the bot sends an update of the voiceName as a session or activity parameter and the expression matches the voiceName, this voiceName is not accepted and the voiceName remains unchanged.

Defines the AWS text-to-speech voice as Neural Voice or Standard Voice.

• true: Neural Voice

• false: (Default) Standard Voice

Note:

• Refer to the Voices in Amazon Polly table to check if the specific language voice supports Neural Voice and/or Standard Voice.

• This parameter is applicable only for AWS.

Enables the language detection feature.

• true: Enables language detection.

• false: (Default) Disables language detection.

Sets the language detection mode (Azure only). They can be one of the following values:

• at-start: (Default) Activates language detection only for the first 5 seconds of recognition.

• continuous: Activates language detection until recognition is accomplished.

Defines a list of up to 10 alternative languages (in addition to the current language) that will be used to detect the language spoken.

Object example (sttContextId parameter is optional):

{
  "language": "de-DE",
  "voiceName": "de-DE-KatjaNeural",
  "sttContextId": "<context ID value>"
}

Note: If languageDetectionMode is at-start, only the first 4 languages will be used (Azure limitation).

Defines the confidence level that a language recognition must reach to enable a language switch. For example, a value of 0.35 indicates that a 35% or above confidence level of a language match must be reached to enable a language switch.

The valid value range is 0.1 to 1.

Enables the Continuous ASR feature. Continuous ASR enables VoiceAI Connect to concatenate multiple speech-to-text recognitions of the user and then send them as a single textual message to the bot.

• true: Enabled

• false: (Default) Disabled

This parameter is applicable when the Continuous ASR feature is enabled.

Defines a special DTMF key, which if pressed, causes the VoiceAI Connect to immediately send the accumulated recognitions of the user to the bot. For example, if configured to "#" and the user presses the pound key (#) on the phone's keypad, the device concatenates the accumulated recognitions and then sends them as one single textual message to the bot.

The default is "#".

Note: Using this feature incurs an additional delay from the user’s perspective because the speech is not sent immediately to the bot after it has been recognized. To overcome this delay, configure the parameter to a value that is appropriate to your environment.

This parameter is applicable when the Continuous ASR feature is enabled.

Defines the automatic speech recognition (ASR) timeout (in milliseconds). The timer is triggered when a recognition is received from the speech-to-text service. When VoiceAI Connect detects silence from the user for a duration configured by this parameter, it concatenates all the accumulated speech-to-text recognitions and sends them as one single textual message to the bot.

The valid value range is 500 (i.e., 0.5 second) to 60000 (i.e., 1 minute). The default is 3000.

Note: The parameter's value must be less than or equal to the value of the userNoInputTimeoutMS parameter.

• Azure speech-to-text service: This parameter controls Azure's Custom Speech model. The parameter can be set to the endpoint ID that is used for accessing the speech-to-text service. For more information on how to obtain the endpoint ID, see Microsoft's Speech Service. Once you have successfully created your custom endpoint in Azure, click the name of the new endpoint, and then copy the value in the 'Endpoint ID' field, as shown in the following example:

• AudioCodes DNN speech-to-text service: This parameter controls the context.

• AudioCodes generic speech-to-text API: If this parameter has a value, this JSON parameter is added to the “start” message (which is sent from VoiceAI Connect to the speech-to-text engine each time a speech recognition session begins) with the parameter value:

"sttContextId": "<context ID value>"

Note:

• The parameter can be used by all bot providers, as long as the speech-to-text service is any of the above.

• For Azure speech-to-text service, the Custom Speech model must be deployed on the same subscription that is used for the Azure speech-to-text service.

• When using other speech-to-text services, the parameter has no affect.

AudioCodes generic speech-to-text API:

This optional parameter is used to pass whatever data is needed from the bot to the speech-to-text. Third-party vendors can use this to pass any JSON object they want to implement.

Note: Only relevant for AC-STT-API.

When using Google's Cloud or Microsoft Azure speech-to-text services, this parameter controls Speech Context phrases.

The parameter can list phrases or words that is passed to the speech-to-text service as "hints" for improving the accuracy of speech recognitions. For example, whenever a speaker says "weather" frequently, you want the speech-to-text service to transcribe it as "weather" and not "whether". To do this, the parameter can be used to create a context for this word (and other similar phrases associated with weather).

For Google's Cloud speech-to-text service, you can also use the parameter to define the boost number (0 to 20, where 20 is the highest) for context recognition of the specified speech context phrase. Speech-adaptation boost allows you to increase the recognition model bias by assigning more weight to some phrases than others. For example, when users say "weather" or "whether", you may want the speech-to-text to recognize the word as "weather". For more information, see https://cloud.google.com/speech-to-text/docs/context-strength. 

You can also use Google's class tokens to represent common concepts that occur in natural language, such as monetary units and calendar dates. A class allows you to improve transcription accuracy for large groups of words that map to a common concept, but that don't always include identical words or phrases. For example, the audio data may include recordings of people saying their street address. One may say "my house is 123 Main Street, the fourth house on the left." In this case, you want Speech-to-Text to recognize the first sequence of numerals ("123") as an address rather than as an ordinal ("one-hundred twenty-third"). However, not all people live at "123 Main Street" and it's impractical to list every possible street address in a SpeechContext object. Instead, you can use a class token in the phrases field of the SpeechContext object to indicate that a street number should be recognized no matter what the number actually is.

For example:

{
  "sttSpeechContexts": [
    {
      "phrases": [
        "weather"
      ],
      "boost": 18
    },
    { 
      "phrases": [
        "whether",
      ],
      "boost": 2
    },
    { 
      "phrases": [
        "fair"
      ]
    },
    {
      "phrases": [
        "$ADDRESSNUM"
      ]
    }
  ]
}

Note:

• The parameter can be used by all bot providers when the speech-to-text service is Google or Azure. When using other speech-to-text services, the parameter has no affect.

• For more information on Google's speech context (speech adaptation) as well details regarding tokens (class tokens) that can be used in phrases, go to https://cloud.google.com/speech-to-text/docs/speech-adaptation.

• For more information on Microsoft's Azure Phrase Lists for improving recognition accuracy, click here.

• This parameter replaces the sttContextPhrases and sttContextBoost parameters (now deprecated).

• AudioCodes generic speech-to-text API: This optional parameter is passed as-is whatever data is needed from the bot to the speech-to-text.

Enables the use of Google speech-to-text custom model adaptation for speech-to-text calls. For more information on how to create model adaptation, go to Google's documentation.

Prevents the speech-to-text response from the bot to include punctuation marks (e.g., periods, commas and question marks).

• true: Punctuation is excluded.

• false: (Default) Punctuation is allowed and if present, included.

Note: This feature requires support from the speech-to-text service provider.

Defines the Azure speech-to-text recognition mode.

Can be one of the following values:

• conversation (default)

• dictation

• interactive

Note: The parameter is applicable only to the Microsoft Azure speech-to-text service.

Defines the Google speech-to-text interaction type.

Can be one of the following values:

• INTERACTION_TYPE_UNSPECIFIED

• DISCUSSION

• PRESENTATION

• PHONE_CALL

• VOICEMAIL

• PROFESSIONALLY_PRODUCED

• VOICE_SEARCH

• VOICE_COMMAND

• DICTATION

By default, the interaction type is not specified.

For more information, see Google speech-to-text documentation.

Note: The parameter is applicable only to the Google speech-to-text service.

Defines the audio transcription model for Google Speech-to-Text.

• latest_long

• latest_short

• telephony

• telephony_short

• command_and_search

• phone_call

• video

• default

By default, the parameter is undefined.

For more information on Google's transcription models, click here.

Note:

• For the enhanced speech recognition model, see the sttEnhancedModel parameter.

• The parameter is applicable only to Google.

• If the chosen model is not supported (e.g., no enterprise Google account), calls may fail.

Enables Google's Speech-to-Text enhanced speech recognition model. There are currently two enhanced models - phone call and video. These models have been optimized to more accurately transcribe audio data from these specific sources.

• true

• false (default)

If configured to true, select the model for transcription (only phone_call or video), using the sttModel parameter. If the model is not selected, VoiceAI Connect automatically sets the sttModel parameter to phone_call.

For more information on Google's enhanced speech recognition model, click here.

Note:

• If the chosen model is not supported (e.g., no enterprise Google account), calls may fail.

• The parameter is applicable only for Google (but not supported for One-Click integration with Dialogflow ES).

When using Google's Cloud speech-to-text service, this parameter controls Speech Context phrases.

The parameter can list phrases or words that is passed to the speech-to-text service as "hints" for improving the accuracy of speech recognitions.

For more information on speech context (speech adaptation) as well details regarding tokens (class tokens) that can be used in phrases, go to https://cloud.google.com/speech-to-text/docs/speech-adaptation.

For example, whenever a speaker says "weather" frequently, you want the speech-to-text service to transcribe it as "weather" and not "whether". To do this, the parameter can be used to create a context for this word (and other similar phrases associated with weather):

"sttContextPhrases": ["weather"]

Note:

• The parameter can be used by all bot providers when the speech-to-text service is Google.

• When using other speech-to-text services, the parameter has no affect.

Defines the boost number for context recognition of the speech context phrase configured by sttContextPhrases. Speech-adaptation boost allows you to increase the recognition model bias by assigning more weight to some phrases than others. For example, when users say "weather" or "whether", you may want the speech-to-text to recognize the word as weather.

For more information, see https://cloud.google.com/speech-to-text/docs/context-strength.

Note:

• The parameter can be used by all bot providers when the speech-to-text service is Google.

• When using other speech-to-text services, the parameter has no affect.

Defines the speech-to-text model.

Defines the time to wait (ms) between transcribed words (i.e., detects gaps of pauses between words) before Deepgram's UtteranceEnd feature sends the UtteranceEnd message (i.e., end of spoken utterances). For more information, see Deepgram's documentation on UtteranceEnd.

The default is 1000 ms.

Defines the value of the sesssionObjects field of the CreateSession message. The sesssionObjects field is an array of objects -- wordsets, domain LM, and/or speaker profiles -- to load for the duration of the session. In-call updates will send a Load message with this value in the objects field.

For example, for a Nuance domain-specific language model (LM):

nuanceSessionObjects = [
   {
      "id":"dlm_bank",
      "type":"application/x-nuance-domainlm",
      "url":"https://host/dropbox/krypton/banking-dlm.zip",
      "weight":100
   },
   {
      "id":"ws_places",
      "type":"application/x-nuance-wordset",
      "body":"{\"PLACES\":[{\"literal\":\"Abington Pigotts\"},{\"literal\":\"Primary Mortgage Insurance\",\"spoken\":[\"Primary Mortgage Insurance\"]}]}"
   }
]

Note: This parameter is applicable only to Nuance speech-to-text.

Fields in this object can set the fields builtinWeights, domainLmWeights, useWordSets and recognitionParameters (except audioFormat) of the Recognition message.

For example, to set the formatingType field of the recognitionParameters field of the Recognize message to "num_as_digits":

nuanceRecognizeFieldsObject = {
   "recognitionParameters":{
      "formattingType":"num_as_digits"
   }
}

Note: This parameter is applicable only to Nuance speech-to-text.

Defines the parameters field value in the RecognitionRequest sent to the Nuance server.

For example, for a Nuance word set:

nuanceGRPCRecognitionResources =

[
  {
    "inlineWordset": "{\"PLACES\":[{\"literal\":\"Abington Pigotts\"},{\"literal\":\"Primary Mortgage Insurance\",\"spoken\":[\"Primary Mortgage Insurance\"]}]}"
  }
]

Note: This parameter is applicable only to Nuance speech-to-text gRPC.

Enables VoiceAI Connect's speech detection feature.

• disabled: (Default) Speech Detection feature is disabled and VoiceAI Connect activates speech-to-text according to the default behavior (as mentioned above in Silence and speech detection).

• enabled: Speech Detection feature is enabled and VoiceAI Connect activates the speech-to-text service only when voice of the user is detected.

• on-bot-prompt: This option is applicable only when the Barge-In feature is enabled (bargeIn parameter configured to true). Enabling on-bot-prompt mode disables speech detection while the speech recognition action takes place.

Enables fast speech-to-text recognition.

If the parameter is enabled, use: {"singleUtterance": false }

• true

• false (default)

Note: This parameter is only applicable to the following:

• Google speech-to-text

• Azure speech-to-text

• Nuance speech-to-text

Enables the Barge-In feature.

• true: Enabled. When the bot is playing a response to the user (playback of bot message), the user can "barge-in" (interrupt) and start speaking. This terminates the bot response, allowing the bot to listen to the new speech input from the user (i.e., VoiceAI Connect sends detected utterance to the bot).

• false: (Default) Disabled. VoiceAI Connect doesn't expect speech input from the user until the bot has finished playing its response to the user. In other words, the user can't "barge-in" until the bot message response has finished playing.

Enables barge-in on DTMF. For more information on DTMF, see Receiving DTMF digits notification.

Defines the maximum time (in milliseconds) that VoiceAI Connect waits for input from the user.

If no input is received when this timeout expires, you can configure VoiceAI Connect to play a textual (see the userNoInputSpeech parameter) or an audio (see the userNoInputUrl parameter) prompt to ask the user to say something. If there is still no input from the user, you can configure VoiceAI Connect to prompt the user again. The number of times to prompt is configured by the userNoInputRetries parameter.

If the sendEventsToBot parameter is configured to noUserInput (or, the userNoInputSendEvent parameter is configured to true) and the timeout expires, VoiceAI Connect sends an event to the bot, indicating how many times the timer has expired.

The default is 0 (i.e., feature disabled).

Note:

• DTMF (any input) is considered as user input (in addition to user speech) if the sendDTMF parameter is configured to true.

• If you have configured a prompt to play when the timeout expires, the timer is triggered only after playing the prompt to the user.

Defines the maximum time that VoiceAI Connect waits for user input before disconnecting the call.

The value can be 0 (no timeout. i.e., call remains connected) or any number from 100. The default is 300000 ms (i.e, 5 minutes).

This parameter can be changed by the Administrator and bot.

Note: DTMF (any input) is considered as user input (in addition to user speech) if the sendDTMF parameter is configured to true.

Defines the maximum number of allowed timeouts (configured by the userNoInputTimeoutMS parameter) for no user input. If you have configured a prompt to play (see the userNoInputSpeech or userNoInputUrl parameter), the prompt is payed each time the timeout expires.

The default is 0 (i.e., only one timeout).

For more information on the no user input feature, see the userNoInputTimeoutMS parameter.

Note: If you have configured a prompt to play upon timeout expiry, the timer is triggered only after playing the prompt to the user.

Defines the textual prompt to play to the user when no input has been received from the user when the timeout expires (configured by userNoInputTimeoutMS).

The prompt can be configured in plain text or in Speech Synthesis Markup Language (SSML) format:

By default, the parameter is not configured.

Plain-text example:

{
  "name": "LondonTube",
  "provider": "my_azure",
  "displayName": "London Tube",
  "userNoInputTimeoutMS": 5000,
  "userNoInputSpeech": "Hi there. Please say something"
}

SSML example:

{
  "name": "LondonTube",
  "provider": "my_azure",
  "displayName": "London Tube",
  "userNoInputTimeoutMS": 5000,
  "userNoInputSpeech": "<speak>This is <say-as interpret-as=\"characters\">SSML</say-as></speak>"
}

For more information on the no user input feature, see the userNoInputTimeoutMS.

Note:

• If you have also configured to play an audio prompt (see the userNoInputUrl parameter), the userNoInputSpeech takes precedence.

• This feature requires a text-to-speech provider. It will not work when the speech is synthesized by the bot framework.

• The supported SSML elements depend on the text-to-speech provider:

  • Google

  • Azure

  • AWS

• For more information on using SSML for text in the message activity, see message Activity.

Defines the URL from where the audio prompt is played to the user when no input has been received from the user when the timeout expires (configured by userNoInputTimeoutMS).

By default, the parameter is not configured.

For more information on the no user input feature, see the userNoInputTimeoutMS.

Note: If you have also configured to play a textual prompt (see the userNoInputSpeech parameter), the userNoInputSpeech takes precedence.

Enables VoiceAI Connect to send an event message to the bot if there is no user input for the duration configured by the userNoInputTimeoutMS parameter, indicating how many times the timer has expired ('value' field). For more information on this event, see Speech customization.

• true: Enabled.

• false: (Default) Disabled.

Note:

• The feature is applicable only to Azure, Google, and AudioCodes API (ac-api).

• For more information, see Receiving no user input notification.