Customizing tool response
Maximum response length
By default, responses from customer-defined tools are limited to 100K characters. This safeguard prevents malfunctioning tools from overloading the conversation history and helps control unnecessary LLM processing costs.
This default can be changed by configuring the ‘Max response length’ parameter.
Error response
In case of an error customer-defined tools return generic error response to LLM.
This is done on purpose to prevent exposure of internal tool implementation / API details to the agent’s business logic.
Use the following Agent’s configuration parameter to make tool error response more explicit and include HTTP response and text fields:
{"explicit_tool_errors": true}
Response timeout
The ‘Timeout (sec)’ parameter defines how long the agent waits for the tool's response. Note that the conversation is paused while the tool is executed. Consider using progress messages, as described in Progress and error messages, if your tool execution takes too long.
No tool response
For agents that are expected to always call tools, you can set the no_tool_error advanced configuration parameter to the error that will be returned to LLM if it tries to provide textual response.
{
"no_tool_error": "Text responses are not supported. Use `play_url` tool."
}
Note that for most AI Agent deployments, setting this parameter is not appropriate and will completely break the agent’s behavior. Use this parameter with great caution and only when your prompt explicitly instructs LLM to never provide textual responses
Tool responses in message history
Tool responses are preserved in message history, by default. This provides the LLM with better context and enables it to respond to follow-up questions using tool results from previous utterances. However, this may be problematic if, for example, the tool response is very long, as it will increase the context size and correspondingly the conversation costs. In such cases, you can redact the specific tool’s responses from message history, by setting the ‘Response in message history’ parameter to redact.