Configuring multiple speech providers

VoiceAI Connect supports connecting to various speech services and enables configuration of parameters for each service (e.g., your credentials to the service). A combination of a speech service and its parameters is termed a provider. Typically, a bot is associated with one speech-to-text (STT) provider and one text-to-speech (TTS) provider.

However, there are scenarios where greater control over provider usage is required, and VoiceAI Connect offers you this control through the following features:

• Load balancing between providers: To overcome limitations of a single speech service provider, you can define multiple parameter sets (e.g., each in a different region or with different credentials) and let VoiceAI Connect balance the load between them.

• Fallback to alternative providers: If there’s a failure to connect to the primary provider, additional providers can be configured as backups to ensure uninterrupted service.

• Dynamical provider change: The bot can switch speech providers during a call. For instance, different speech-to-text providers may excel in recognizing specific types of information (e.g., dates or street names), and VoiceAI Connect allows the bot to switch to the most suitable provider as needed during the conversation.

How do I use it?

Load balancing and fallback speech-to-text providers

For configuring load balancing and/or fallback speech-to-text providers, you need to configure a provider of type "list" and configure its providers list with the relevant providers. You then need to configure the sttProvider bot parameter with the name of the provider that you configured with the type "list".

The Administrator can use the following bots parameter for controlling this feature:

The Administrator can use the following providers parameter for controlling this feature:

{
  "providers": [
    {
      "name": "provider_1",
      "type": "azure"
    },
    {
      "name": "provider_2",
      "type": "azure"
    },
    {
      "name": "group_1",
      "type": "list",
      "providers": [
        {
          "name": "provider_1",
          "priority": 0
        },
        {
          "name": "provider_2",
          "priority": 1
        }
      ]
    }   
  ],
  "bots": [
    {
      "name": "bot_1",
      "sttProvider": "group_1",
      "maxSttRetriesPerPriority": 0
    }
  ]
}

The following table lists the fields inside the providers array elements:

By setting different values to the priority field, you can achieve various load balancing and/or fallback behaviors.

• Example 1: For load balancing between two providers without any fallback, specify the providers without any priority (i.e., using the default priority 0), and restrict fallback between them:

{
  "providers": [
    {
      "name": "provider_1",
      "type": "azure"
    },
    {
      "name": "provider_2",
      "type": "azure"
    },
    {
      "name": "group_1",
      "type": "list",
      "providers": [
        {
          "name": "provider_1",
          "priority": 0
        },
        {
          "name": "provider_2",
          "priority": 0
        }
      ]
    }   
  ],
  "bots": [
    {
      "name": "bot_1",
      "sttProvider": "group_1",
      "maxSttRetriesPerPriority": 0
    }
  ]
}

• Example 2: For configuring a simple fallback, without load balancing, specify a different priority for each provider:

{
  "providers": [
    {
      "name": "provider_1",
      "type": "azure"
    },
    {
      "name": "provider_2",
      "type": "azure"
    },
    {
      "name": "group_1",
      "type": "list",
      "providers": [
        {
          "name": "provider_1",
          "priority": 0
        },
        {
          "name": "provider_2",
          "priority": 1
        }
      ]
    }   
  ],
  "bots": [
    {
      "name": "bot_1",
      "sttProvider": "group_1"
    }
  ]
}

• Example 3: For configuring load balancing between two providers, with a third fallback if both fail, specify several providers with the same priority:

{
  "providers": [
    {
      "name": "provider_1",
      "type": "azure"
    },
    {
      "name": "provider_2",
      "type": "azure"
    },
    {
      "name": "alternative_provider_3",
      "type": "azure"
    },
    {
      "name": "group_1",
      "type": "list",
      "providers": [
        {
          "name": "provider_1",
          "priority": 0
        },
        {
          "name": "provider_2",
          "priority": 0
        },
        {
          "name": "alternative_provider_3",
          "priority": 1
        }
      ]
    }
  ],
  "bots": [
    {
      "name": "bot_1",
      "sttProvider": "group_1",
      "maxSttRetriesPerPriority": 0
    }
  ]
}

Dynamic speech-to-text provider change

On bot message, where the sttProvider parameter is different than the current speech-to-text provider, VoiceAI Connect stops the current speech-to-text session with the current provider, and the next speech-to-text request will be with the new provider (if the new speech-to-text provider is listed in the additionalSttProviders parameter list value).

To integrate this feature in your bot, see Changing the speech provider.

The Administrator can use the following bot parameter to define a list of speech-to-text providers allowed for dynamic change:

Example

The following shows an example of a bot configuration for two speech-to-text providers:

{
  "name": "MyBot",
  "provider": "my_azure_provider",
  "displayName": "My Bot",
  "credentials": {
    "botSecret": "..."
  },
  "additionalSttProviders": [
    "my_azure_provider",
    "my_google_provider"
  ]
}

Example:

The following shows an example of a bot configuration for one group of providers, and additional speech-to-text providers comprised of a group and two separate providers:

{
  "providers": [
    {
      "name": "provider_1",
      "type": "azure"
    },
    {
      "name": "provider_2",
      "type": "azure"
    },
    {
      "name": "provider_3",
      "type": "azure"
    },
    {
      "name": "provider_4",
      "type": "azure"
    },
    {
      "name": "provider_5",
      "type": "azure"
    },
    {
      "name": "group_1",
      "type": "list",
      "providers": [
        {
          "name": "provider_1",
          "priority": 0
        },
        {
          "name": "provider_2",
          "priority": 1
        }
      ]
    },
    {
      "name": "group_2",
      "type": "list",
      "providers": [
        {
          "name": "provider_3",
          "priority": 0
        },
        {
          "name": "provider_4",
          "priority": 1
        }
      ]
    }
  ],
  "bots": [
    {
      "name": "bot_1",
      "sttProvider": "group_1",
      "additionalSttProviders": [
        "group_2",
        "provider_5"
      ]
    }
  ]
}

Dynamic text-to-speech provider change

On bot message, where the ttsProvider parameter is different than the current text-to-speech provider, VoiceAI Connect stops the current text-to-speech session with the current provider, and the next text-to-speech request will be with the new provider (if the new text-to-speech provider is listed in the additionalTtsProviders parameter list value). To integrate this feature in your bot, see changing-the-speech-provider.htm.

The Administrator can use the following bot parameter is used to define a list of text-to-speech providers:

Example

The following shows an example of a bot configuration for two text-to-speech providers:

{
  "name": "MyBot",
  "provider": "my_azure_provider",
  "displayName": "My Bot",
  "credentials": {
    "botSecret": "..."
  },
  "additionalTtsProviders": [
    "my_azure_provider",
    "my_google_provider"
  ]
}