Multi-agent orchestration modes

Multiple agents can interact one with another using one of the following orchestration modes:

The following sections describe each of these modes in details, their pros and cons and recommended use-cases for each of them.

Consult orchestration mode

In consult orchestration mode, all communication with user is handled by the “main” agent.

For every question asked by user, the “main” agent decides whether to answer it on its own or to pass_question / send_message to another agent. If it decides to “consult” with another agent, the “control” is passed to this agent. But, as soon as the other agent has the answer / response ready (or wants to send some message to user), the “control” is returned to the “main” agent. Which sends the response to the user and waits for the next question.

When the user asks the follow-up question, the “main” agent again decides whether to answer it on its own, or to pass_question / send_message to another agent. And so on and so forth.

The following diagram demonstrates communication flow in consult orchestration mode.

Consult orchestration mode is ideally suited for scenarios where the user's questions can be categorized into separate “domains” handled by individual agents, and where each sub-agent implements a relatively simple interaction sequence with user (for example question / answer sequence).

Since all communication with the user is handled by the “main” agent, it has good visibility over the conversation context, and therefore can decide where to route the follow-up questions.

Note, however, that “main” agent has no knowledge of the other agents’ prompts. Therefore, if the later implements multi-sequence information collection, the“main” subagent may have trouble understanding this sequence and routing the follow-up questions to the correct sub-agent.

Delegate orchestration mode

In delegate orchestration mode, once the question / message is passed to another agent “control” stays there – allowing other agent to reliably handle follow-up conversation with the user as it sees fit. The “control” may be passed to the “main” / other agent upon explicit call to pass_question / send_message tools by the “currently active” agent.

The following diagram demonstrates communication flow in delegate orchestration mode.

Delegate orchestration mode is ideally suited for scenarios where sub-agents perform complicated multi-step interactions with the user. Since the “control” remains in the active agent, there is no risk that “main” agent may misroute the follow-up question to the wrong agent, as may happen in consult orchestration mode. Note, however, that each agent has its own context (message history) – therefore as the user is passed from one agent to another, some parts of the conversation become invisible to the currently active agent. This may be partially mitigated by using send_message to pass context summary to another agent.

It is possible to use delegate mode for simple question and answer scenarios instead of with consult mode described above. However, if you do so, you may be tempted to define “direct transitions” between multiple agents – e.g. by telling each agent to “pass question” to another agent if the question falls into a different domain. Such an approach may require careful prompt engineering to prevent “loops” – where the first agent passes a question to the second agent, but the second agent passes the question back to first agent, and so on and so forth.

Delegate with history orchestration mode

Delegate with history orchestration mode improves on the delegate mode by implementing a shared context (message history) for all agents involved in the conversation. When the “control” is passed from one agent to another, the first message in the message history (System Prompt) is updated to match the currently active agent, but the rest of the message history remains unchanged. This ensures that current agent has full context of the current conversation, regardless of which agent handled the specific user interaction.

While delegate with history mode typically behaves reasonably well, it increases the context size and correspondingly implies slightly higher LLM costs. Also, depending on the prompt and LLM being used, presence of the interaction completed by other agents in message history may confuse LLM and produce hallucinations / cause unexpected behaviour. Finally, you need to be careful if you equip your agents with different LLMs with different max context sizes – as the smallest one may effectively determine the context size for the whole conversation.

Different orchestration modes for specific sub-agents

For complex multi-agent topologies you may need to use different orchestration modes for specific sub-agents (under the same agent). This can be achieved by using orchestration_mode advanced configuration parameter. Specify a valid JSON dictionary with sub-agent name as a key and one of the following values: 'delegate', 'delegate_with_history', 'consult'. There is no need to include all sub-agent names - any sub-agent that is not mentioned in the variable will be configured according to regular orchestration mode configuration parameter.

For example: 

{
    "orchestration_mode": {
        "test-agent-2": "delegate",
        "test-agent-3": "consult"
    }
}