The simplest way to build a multi-agent system is to give one agent a tool for calling the others and let it hand out the work. The design feels natural because that is how a team of people works: a manager assigns tasks and the team does them. It also works well in a demo.

In production we moved away from it. The model is worth trusting with decisions about what to do, and how the work gets done is better left to code. Below we explain where this split came from and what it costs us.

What the coordinator actually spends its time on

A coordinator agent is expected to think: take the question apart, decide who should do what and pull the conclusions together. But open the traces and most of its decisions turn out to be about mechanics: how long to wait for an answer, whether to retry a call that did not come back, whether to run steps one after another or together, how to pass a result to the next agent and when to stop.

The coordinator makes each of these decisions from scratch, in every conversation, depending on the situation. It usually gets them right, but when it gets one wrong, the mistake is expensive.

Here is an example from our own work. A full analytical report takes fifteen to twenty minutes to write. The coordinator waited fifteen minutes for it, got a timeout and started the report again. The first run kept going, because a timeout on the side that waits does not stop the side being waited for. Over one night the report was started ten times. There was no flaw in the model's reasoning: the tool returned a timeout, and starting again looked sensible. The mistake was that the model was making this decision at all.

What the model does "almost always"

Some things a model does almost always. For the text of an answer that is enough: a clumsy phrase is easy to see and easy to fix. For numbers, money and data passed between agents, "almost always" means that sooner or later an error lands in a report and nobody notices.

Numbers. A model can count the rows of a table, but it does so unreliably. One of our agents recounted a brand's stores and was off by more than a factor of two, and for five countries it wrote "not found" although the data was there.

Handing over data. When one agent retells a result to the next, the caveats get lost. "The source did not respond for this country" turns into "there is nothing in this country", and the next agent builds a conclusion on it.

Money. A budget for paid search written into the prompt is something the model takes into account but does not always respect. A limit that has to hold has to be checked by code before the call.

How the split works

The model is responsible for judgement: how big the question is, how to break it down, what each step should achieve, whether what has been collected is enough and what it means. Code is responsible for everything that has to happen every time.

The model decides what, code decides how

The planner writes the plan once, as a graph of steps. A step can build on the result of an earlier one: for example, first find candidate countries, then run a separate step for each of them. In simplified form such a plan looks like this:

{
  "steps": [
    {"id": "s1", "goal": "Countries where similar brands opened in the last three years"},
    {"id": "s2", "for_each": "s1.countries", "goal": "Entry requirements and rent in the main city"},
    {"id": "s3", "goal": "Local distributors in the category"}
  ]
}

Steps s1 and s3 are independent, so the code runs them together. Step s2 waits for s1 and is repeated for each country it finds. Each step is carried out by a separate collector agent and returns a fact sheet in which every number is tied to a row in a table. Code checks those numbers before they go any further. At the end the planner decides once whether what was collected is enough, and it can add at most two more steps.

A similar split appears in the LLMCompiler and ReWOO papers: a planner, an executor and a step that decides whether the result is sufficient.

One more detail took us a while to reach: the planner does not see the list of tools. When it did, it assigned sources to steps itself and chose them worse than the collectors, which know their own tools. It once sent a question about website traffic to third-party pages, although the answer was in our search data. Now the planner states the goals, and the executors decide where to look.

What it costs us

The split has a price. The plan is fixed at the start, and if something unexpected turns up along the way, the system has only one point at which it can change its mind. The code that executes the plan has to be written and maintained, and in our case it runs to several thousand lines.

The multi-agent setup itself is expensive too. According to Anthropic, it uses roughly 15 times more tokens than an ordinary chat. On the same questions, our cascade of several agents cost several times more than a single agent with all the tools. So simple questions never reach the planner: a question about a single fact goes to one collector with a small budget. The team of agents is used where the work really can be split into independent parts.

The opposite mistake

Once you decide that code should own the mechanics, it is easy to go further and start moving judgement into code as well. The agent did not think of what to search the web for? Let code write the task for it. We went through this too: within a week the code had accumulated rules that decided for the agents what was missing and where to look for it. The system became rigid, and every new case needed another code change.

The line we settled on comes down to one question: must this rule hold every time, regardless of the content? Counting, money, waiting, storing and passing data pass this test and live in code. What to look for, which source fits and what the numbers mean depend on the question. Agents decide those, following instructions that can be edited without a deploy.

What's next

This is how the research engine in our analytical agent works. We are now moving it into the Flutch platform so that other agents can use it. But the principle does not depend on the platform. If you are building a team of agents, look at the decisions your coordinator makes and count how many of them are about meaning and how many are about mechanics.