Multi-Agent Architecture Playbook
A decision framework for deciding when — and how — to move from a single agent to a multi-agent system: maturity model, readiness checklists, build-vs-buy, governance, security, cost, and a reusable coordination-pattern scorecard.
- A multi-agent system that cannot coordinate effectively is worse than a single agent, not better. Coordination overhead is real and can dominate the cost and latency of simple tasks — multi-agent is a specialization decision, not a default upgrade.
- 79% of observed multi-agent failures trace back to specification and coordination issues — role unclarity, communication breakdowns, and missing validation — not base-model limitations. Fixing the model doesn't fix these failures; fixing the architecture does.
- Five coordination patterns cover almost all production multi-agent systems: Orchestrator-Subagent, Generator-Verifier, Agent Teams, Message Bus, and Shared State. Orchestrator-Subagent is the sane default because it keeps control flow and observability simple.
- The decision to go multi-agent should follow task decomposition, not precede it. If you can't cleanly describe which sub-task each agent owns and how failures propagate between them, you're not ready to split a single agent into several.
- Agent-to-agent communication needs the same security posture as any other internal API — authenticated, scoped, and logged — especially once agents cross team or vendor boundaries via a protocol like A2A.
- Most multi-agent pilots that fail do so quietly: task completion looks fine in the demo, but sub-task conflicts and supervisor drift never get surfaced until scale exposes them.
1. Scope of this playbook
This playbook is for architects deciding whether and how to decompose a task across multiple cooperating AI agents, and for those already running multi-agent systems who are seeing coordination failures in production. It assumes you've already read (or are separately deciding) the AI Agent Platform Playbook in this series — this playbook is about the architecture of agent cooperation, not the runtime that executes it. It does not cover cross-organizational agent communication in depth; that's the Agent-to-Agent (A2A) Playbook's territory, though the two overlap once your multi-agent system crosses a trust boundary.
2. Multi-agent maturity model
| Level | Characteristics | Typical failure mode |
|---|---|---|
| 0 — Single agent, growing prompt | One agent, one large prompt trying to handle every case | Prompt becomes unmaintainable; quality degrades as scope grows |
| 1 — Ad hoc splitting | Task manually split into two or three agents with informal handoffs | Handoffs lose context; no one owns the failure when a handoff breaks |
| 2 — Orchestrated sub-agents | A defined orchestrator delegates to specialized sub-agents with explicit contracts | Orchestrator becomes a bottleneck or single point of failure if not designed for partial failure |
| 3 — Governed multi-agent system | Explicit coordination pattern, shared observability across all agents, defined conflict-resolution and escalation paths | Coordination overhead becomes visible as a cost and latency line item that must be actively managed |
| 4 — Cross-boundary multi-agent | Agents from different teams or organizations coordinate via a protocol like A2A, with signed identity and audited handoffs | Rare outside vertical-specific production deployments (supply chain, financial services) as of 2026 |
Self-assessment checklist
- Can you name, for each agent in the system, exactly what it owns and what it explicitly does not own?
- When two agents' outputs conflict, is there a defined resolution mechanism, or does the system just pick one arbitrarily?
- Is there a single trace that shows a task's full path across every agent it touched, or do you have to reconstruct it from separate logs?
- Has anyone measured coordination overhead — the added latency and cost of splitting a task across agents versus one agent — for your actual workload?
- If one sub-agent fails or times out, does the system degrade gracefully, or does the whole task fail silently?
3. Is multi-agent even the right pattern?
Start from task decomposition, not from agent count. The honest test: can you describe the task as genuinely separable sub-tasks with different required skills, tools, or knowledge domains? If the answer is "not really, but it feels like it should be more modular," a single well-scoped agent with good tool access will usually outperform a multi-agent system on both cost and reliability.
Multi-agent earns its complexity when: sub-tasks require different tool access that shouldn't be combined in one agent's scope (security reason), sub-tasks benefit from different models or reasoning strategies (cost/quality reason), or the task genuinely requires parallel work that a single sequential agent can't do fast enough (latency reason). "It felt like a natural way to organize the code" is not one of these reasons, and it's the most common one given after the fact.
4. Readiness checklist
- Each agent's scope and tool access is defined explicitly, in writing, before implementation — not discovered through trial and error.
- A coordination pattern (Section 6) has been chosen deliberately, not defaulted to because it's what the framework's tutorial used.
- Distributed tracing spans all agents in the system, so a single task can be followed end-to-end.
- There's an explicit answer to "what happens when agent B disagrees with agent A" before it happens in production.
- Someone has calculated the token/cost overhead of the multi-agent design versus a single-agent baseline, even roughly.
5. Build vs. buy
| Layer | Build | Buy | Recommendation |
|---|---|---|---|
| Coordination pattern implementation | Custom orchestrator logic matched to your task decomposition | Framework-provided patterns (LangGraph graphs, CrewAI crews, Microsoft Agent Framework's orchestration primitives) | Buy the pattern primitives, build the specific decomposition — the pattern itself is well-trodden ground |
| Inter-agent messaging | Direct function calls within a single process for simple cases | A message bus (Shared State or Message Bus pattern) for loosely coupled or cross-team agents | Build simple (direct calls) until you have a concrete reason for a bus |
| Conflict resolution | Domain-specific logic — this is inherently custom to your task | N/A | Build; no vendor can encode your business's tie-breaking rules |
| Cross-organizational coordination | Custom point-to-point integration | A2A protocol, now at v1.2 under the Linux Foundation's Agentic AI Foundation, with 150+ organizations in production | Buy into the standard (A2A) once you're coordinating across trust boundaries |
6. Coordination pattern playbook
Rather than rollout phases, this playbook's core decision is which coordination pattern fits your task. Choose deliberately from the five that cover nearly all production systems:
Orchestrator-Subagent — a single orchestrator holds the plan, delegates to specialized sub-agents, and decides next steps. The default recommendation for most enterprise use cases: clear control flow, straightforward observability, one place to look when something goes wrong.
Generator-Verifier — one agent produces output, a second independently checks it against criteria before it's accepted. Strong fit when correctness matters more than speed.
Agent Teams — several peer agents with different specializations collaborate on a shared goal without a strict hierarchy. Higher coordination overhead; reserve for tasks that genuinely need multiple perspectives converging.
Message Bus — agents publish and subscribe to events rather than calling each other directly. Fits loosely coupled, asynchronous workflows across teams, at the cost of harder-to-trace causality.
Shared State — agents read and write to a common state store rather than passing messages directly. Useful when many agents need visibility into the same evolving context, but requires careful concurrency and conflict handling.
Pick one primary pattern per system. Mixing patterns without a clear reason is a common source of the coordination failures described in Section 11.
7. Governance & risk checklist
- Each agent has a named accountable owner, and the orchestrator (if present) has one too — "the system" is not an owner.
- There's a defined maximum delegation depth (an orchestrator delegating to a sub-agent that delegates further) to prevent runaway task trees.
- Conflicting outputs between agents are logged, not silently resolved, so patterns of disagreement can be reviewed.
- High-stakes decisions that emerge from multi-agent consensus still route through the same human-in-the-loop gates as a single-agent decision would.
- Version changes to any one agent in the system are treated as changes to the whole system for testing purposes.
8. Security checklist
- Inter-agent messages are authenticated — an agent should not blindly trust a message just because it arrived on an internal channel.
- Each agent's tool access remains scoped to its sub-task even within a trusted system; multi-agent is not a reason to relax least-privilege.
- Cross-organizational coordination (via A2A or similar) uses signed agent identity — A2A v1.2's cryptographic agent-card signing is the current baseline, not an optional extra.
- A compromised or misbehaving sub-agent cannot escalate its access by convincing another agent to act on its behalf.
- Rate limits and cost ceilings apply per-agent and in aggregate, since a coordination loop between two agents can runaway in ways a single agent's loop cannot.
9. Cost model & ROI
| Driver | Scales with | Notes |
|---|---|---|
| Coordination overhead | Number of agents × handoffs per task | The most commonly underestimated line item — a three-agent pipeline can cost several times a single well-scoped agent for the same task |
| Redundant context | Sub-agents re-establishing context the orchestrator already had | Shared-state or well-designed handoffs reduce this; ad hoc splitting maximizes it |
| Verification passes | Generator-Verifier and similar patterns | Trade direct cost for reduced downstream error cost |
| Escalation and human review | Coordination failure rate | Poor coordination design shows up here as higher escalation rates |
ROI for multi-agent should be compared explicitly against a well-optimized single-agent baseline, not against "doing nothing." Many multi-agent pilots would have looked like sufficient ROI against no automation at all, while a simpler single-agent design would have delivered the same outcome at a fraction of the coordination cost.
10. Organizational playbook
Assign one clear owner for the overall system architecture (the coordination pattern and orchestrator, if any), separate from the owners of individual sub-agents. This mirrors good microservices practice: someone owns the contracts between components, even if no one team owns every component. Without this role, coordination-layer decisions get made implicitly by whichever team happens to touch the orchestrator code last — a reliable source of the specification-ambiguity failures in Section 11.
11. Common failure patterns
- Supervisor drift: an orchestrator's plan diverges from what's actually happening as sub-agents report back, without a mechanism to detect the divergence.
- Silent conflict resolution: two agents produce contradictory outputs and the system picks one without logging that a conflict occurred, hiding a recurring design problem.
- Coordination-cost blindness: nobody measures the overhead of the multi-agent design against a single-agent baseline, so an expensive, fragile architecture persists because "it works."
- Trust-boundary creep: agents that were designed to trust each other implicitly within one team's system get extended to cross-team or cross-vendor use without adding the authentication and validation that boundary requires.
- The team-of-one anti-pattern: three agents each responsible for a trivial slice of a task that a single well-prompted agent would have handled in one pass.
12. Multi-agent architecture decision scorecard
Score each candidate coordination pattern 1–5 for your specific task:
| Dimension | Weight | Score (1–5) |
|---|---|---|
| Clarity of control flow / debuggability | High | — |
| Coordination overhead relative to task value | High | — |
| Fit to genuine task decomposition (not artificial) | High | — |
| Conflict-resolution mechanism defined | High | — |
| Cross-boundary readiness (if needed) | Medium | — |
| Observability across the full agent chain | High | — |
| Graceful degradation on partial failure | Medium | — |
If your highest-scoring pattern still scores low on coordination overhead relative to task value, revisit Section 3 — the right answer may be fewer agents, not a better-coordinated set of them.