Custom tool responses

Maximum response length

The custom tool response is limited to 100K characters, by default. This ensures that a misbehaving tool doesn’t “flood” the conversation history and therefore, doesn’t cause excessive LLM costs.

This default can be changed by configuring the ‘Max response length’ parameter.

Error response

In case of an error custom tools return generic error response to LLM.

This is done on purpose to prevent exposure of internal tool implementation / API details to the agent’s business logic.

If you configure the following Agent’s advanced configuration parameter:

{"explicit_tool_errors": true}

The tool error response will be more explicit and will include HTTP response code and text:

{
    "status": "error",
    "code": 404,
    "text": "Document not found"
}

Response timeout

The ‘Timeout (sec)’ parameter defines how long the agent waits for the tool's response. Note that the conversation is paused while the tool is executed. Consider using progress messages, as described in Progress and error messages, if your tool execution takes too long.

No tool response

For agents that are expected to always call tools, you can set the no_tool_error advanced configuration parameter to the error that will be returned to LLM if it tries to provide textual response.

{
  "no_tool_error": "Text responses are not supported. Use `play_url` tool."
}

Note that for most AI Agent deployments, setting this parameter is not appropriate and will completely break the agent’s behavior. Use this parameter with great caution and only when your prompt explicitly instructs LLM to never provide textual responses

Tool responses in message history

Tool responses are preserved in message history, by default. This provides the LLM with better context and enables it to respond to follow-up questions using tool results from previous utterances. However, this may be problematic if, for example, the tool response is very long, as it will increase the context size and correspondingly the conversation costs. In such cases, you can redact the specific tool’s responses from message history, by setting the ‘Response in message history’ parameter to redact.