Five agents, three human approval gates

I built a system where five specialised agents take a marketing brief and turn it into a running ad campaign: one shapes strategy, one writes copy, one generates imagery, one talks to the ads platform, one reads back performance.

The demo is the easy part. What actually took the time was everything around the fact that this pipeline spends a client's money.

Every ad is created paused

The single most important line in the whole system sets an ad's status to paused on creation.

An LLM pipeline that can create a live ad is a pipeline that can spend a budget on a hallucinated audience at three in the morning. Created-paused means the worst bug in the creative path produces a bad draft that someone deletes, rather than a charge someone has to explain.

Three gates need a human before anything proceeds: the generated image, the budget, and a final preview. They are not configurable. A gate you can switch off is a gate that will be switched off during a busy week.

JSON mode silently disables tool calling

This one cost me a day and I have not seen it written down clearly anywhere.

Agents were given tools — fetch the brand profile, look up past campaigns — and told to return structured output via response_format: { type: 'json_object' }. They stopped calling the tools. Not an error: they'd answer confidently from nothing, or return an empty reply.

Forcing a JSON response format suppresses tool calling. The model is constrained to emit a JSON object, and a tool call is not that, so it does the thing it is allowed to do: makes something up that fits the schema.

The fix is two phases. First a free-form pass with tools available and no format constraint, looping until the model stops asking for tools — capped, because an agent that keeps calling tools forever is a real failure mode and not a rare one. Then a second call that takes that transcript and structures it:

// phase 1 — research, tools on, no response_format
let messages = [system, user];
for (let round = 0; round < MAX_TOOL_ROUNDS; round++) {
  const res = await llm({ messages, tools });
  if (!res.tool_calls?.length) break;
  messages = messages.concat(res.message, await runTools(res.tool_calls));
}

// phase 2 — structure it, no tools, format enforced
const shaped = await llm({
  messages: messages.concat(structuringPrompt),
  response_format: { type: 'json_object' },
});

Two calls instead of one. Worth it: phase one can look things up, phase two reliably returns something parseable. There's a fallback for the structuring call coming back empty, because occasionally it does.

Log every call or you cannot answer the cost question

Every model call is written to a table: which agent, which campaign, tokens in and out, latency, and whether it failed.

This felt like overhead until the first time someone asked what a campaign cost to generate. Without per-call logging the honest answer is a shrug and a provider invoice covering everything you ran that month. With it, you can attribute cost per brand, notice one agent quietly burning most of the budget, and catch a prompt change that doubled token usage.

It is also how you find out that your retry logic is retrying more than you thought.

Agents are a state machine wearing a hat

The framing that made this tractable: it isn't really "agents talking to each other". It's a twelve-stage state machine where some transitions happen to call a model and three of them wait for a human.

Once it's a state machine, the boring questions become answerable. What happens if the process dies at stage seven? Where does it resume? What if the human never approves? None of those have good answers in a free-form agent conversation, and all of them are routine in a state machine with persisted stages.

If you take one thing from this: decide early what your system is allowed to do without asking. Everything else — the gates, the paused status, the audit log — falls out of that answer.