Multi-Agent AI Systems: How to Build Teams of AI That Actually Work
A single LLM call can answer a question. Multi-agent AI systems can research a topic, analyze the data, write a report, and route it to your team — autonomously. Therefore, the shift from single-prompt AI to coordinated agent teams represents the biggest practical advancement in AI engineering since the first chat assistants shipped. This guide shows you how to build these systems with LangGraph, including the patterns that hold up in production and the pitfalls that quietly waste your token budget.
Why Single Agents Hit a Wall
A single AI agent trying to handle a complex task runs into three recurring problems: context-window limits (it forgets earlier steps), role confusion (it tries to be researcher, writer, and reviewer simultaneously), and reliability (one bad step derails the entire output). Moreover, debugging a single agent that made a mistake on step 7 of a 12-step process is nearly impossible, because every step shares one undifferentiated transcript.
Multi-agent systems solve this by decomposing the work into specialized roles. Each agent gets a focused system prompt, limited tool access, and a clear input/output contract. Consequently, a research agent can search the web and compile facts, an analysis agent can identify patterns and gaps, and a writer agent can produce the final output — each excelling at its narrow role rather than juggling all three.
This mirrors how human teams work. You do not ask your backend developer to also run UX research and write marketing copy; instead, you assemble specialists who collaborate through defined handoffs. The same logic, applied to LLMs, is the entire premise of agent orchestration.
LangGraph: Building Agent Graphs
LangGraph models multi-agent workflows as directed graphs in which nodes are agents or tools and edges define the flow of information. Unlike a simple sequential chain, LangGraph supports conditional routing, parallel execution, and cycles — an agent can loop back for self-correction. Additionally, its built-in state management tracks everything that flows between agents, which is what makes the system inspectable rather than a black box.
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from typing import TypedDict, Annotated, Literal
import operator
class ResearchState(TypedDict):
"""Shared state that flows through the entire agent graph"""
messages: Annotated[list, operator.add]
research_data: str
sources: list[str]
analysis: str
quality_score: float
final_report: str
iteration_count: int
def research_agent(state: ResearchState) -> dict:
"""
Gathers information from multiple sources.
Has access to: web_search, document_reader tools
"""
research_prompt = f"""You are a research specialist. Your job is to gather
comprehensive, factual information about the topic.
Previous research (if any): {state.get('research_data', 'None')}
Quality feedback (if any): {state.get('analysis', 'None')}
Requirements:
- Find at least 5 distinct sources
- Include specific data points, statistics, and quotes
- Note conflicting information between sources
- Flag anything that seems unreliable
"""
result = research_llm.invoke([
SystemMessage(content=research_prompt),
*state["messages"]
])
# Extract sources from the research
sources = extract_urls(result.content)
return {
"research_data": result.content,
"sources": sources,
"messages": [AIMessage(content=f"Research complete: {len(sources)} sources found")]
}
def analysis_agent(state: ResearchState) -> dict:
"""
Evaluates research quality and identifies gaps.
No tool access — pure reasoning.
"""
result = analysis_llm.invoke([
SystemMessage(content="""You are a critical analysis expert. Evaluate the
research data for completeness, accuracy, and gaps.
Score the research 1-10 on:
- Source diversity (multiple perspectives?)
- Data specificity (concrete numbers or vague claims?)
- Completeness (any obvious gaps?)
If score < 7, explain what's missing so the researcher can try again."""),
HumanMessage(content=f"Research to evaluate:
{state['research_data']}")
])
score = extract_score(result.content)
return {
"analysis": result.content,
"quality_score": score,
"messages": [AIMessage(content=f"Analysis complete. Quality score: {score}/10")]
}
def quality_router(state: ResearchState) -> Literal["writer", "researcher"]:
"""Route back to research if quality is low, otherwise proceed to writing"""
if state["quality_score"] < 7 and state.get("iteration_count", 0) < 3:
return "researcher" # Loop back for better research
return "writer" # Quality is good enough, proceed
def writer_agent(state: ResearchState) -> dict:
"""Produces the final report from validated research."""
result = writer_llm.invoke([
SystemMessage(content="""Write a comprehensive, well-structured report.
Use the research data and analysis to create something genuinely useful.
Include specific data points and cite sources."""),
HumanMessage(content=f"Research:
{state['research_data']}
"
f"Analysis:
{state['analysis']}")
])
return {"final_report": result.content}
# Build the graph
workflow = StateGraph(ResearchState)
workflow.add_node("researcher", research_agent)
workflow.add_node("analyst", analysis_agent)
workflow.add_node("writer", writer_agent)
workflow.set_entry_point("researcher")
workflow.add_edge("researcher", "analyst")
workflow.add_conditional_edges("analyst", quality_router)
workflow.add_edge("writer", END)
# Compile with checkpointing for fault tolerance
from langgraph.checkpoint.sqlite import SqliteSaver
memory = SqliteSaver.from_conn_string("checkpoints.db")
app = workflow.compile(checkpointer=memory)
Reading the Graph: How State Actually Flows
It helps to trace one full pass through the graph above, because the value of LangGraph only becomes obvious once you see the cycle in motion. Execution enters at researcher, which writes research_data and sources into shared state. Control then moves along a fixed edge to analyst, which reads that data, scores it, and writes both an analysis string and a numeric quality_score back into the same state object.
Next comes the interesting part. Instead of a fixed edge, add_conditional_edges hands control to quality_router, a plain Python function that inspects state and returns the name of the next node. If the score is below seven and the iteration budget is not exhausted, the router sends execution back to researcher with the analyst’s feedback already in state — so the second research pass is genuinely informed by the first critique. Otherwise it proceeds to writer and then to END. Because the router is ordinary code rather than another model call, it is deterministic, cheap, and trivial to unit-test. That separation of deterministic control flow from non-deterministic generation is the single most important idea in the entire framework.
The Patterns That Actually Work in Production
Pattern 1: Supervisor + Workers. One agent — the supervisor — decides which worker to delegate to based on the task. This works well for customer-support bots, where the supervisor routes billing questions to a billing agent and technical issues to a tech agent. Critically, the supervisor never does the actual work; it only routes, which keeps its prompt small and its behavior predictable.
Pattern 2: Pipeline with Quality Gates. Agents form a sequential pipeline (research → analysis → writing) with quality checks between stages. If a stage fails its check, it loops back, exactly as the code above demonstrates. This is the most reliable pattern for content generation and data-processing workflows, so it is usually the right place to start.
Pattern 3: Parallel Fan-Out / Fan-In. Multiple agents work simultaneously on different facets of one task, and a final agent merges their outputs. When analyzing a codebase, for instance, one agent reviews security, another reviews performance, and a third reviews style — then a final agent combines the three into a single report. Because the workers are independent, you can run them concurrently and cut wall-clock time substantially.
Pattern to avoid: Full mesh. Do not let every agent talk to every other agent. A fully connected mesh creates unpredictable, emergent behavior and makes debugging nearly impossible, since there is no canonical path through the system. Always keep a clear, directed flow of information.
Multi-Agent AI Systems: Tool Isolation and Safety
Each agent should have access only to the tools it genuinely needs. A research agent gets web search and document reading; a database agent gets read-only SQL; a deployment agent gets API access scoped to staging. This is not merely a security measure — it actively prevents agents from taking shortcuts that produce confidently wrong results, because an agent cannot misuse a tool it was never handed.
# Tool isolation per agent
research_tools = [web_search, document_reader, arxiv_search]
analysis_tools = [] # Pure reasoning — no tools needed
writer_tools = [grammar_checker, citation_formatter]
deploy_tools = [staging_api_client] # Never production access
# Each agent is created with its specific toolset
research_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514").bind_tools(research_tools)
analysis_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514") # No tools
writer_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514").bind_tools(writer_tools)Notice that the analysis agent is bound with no tools at all. That is deliberate: a critic with the ability to re-run searches will tend to “fix” gaps itself instead of reporting them, which collapses the separation of concerns the pipeline depends on. Withholding tools is sometimes the safest design decision you can make.
State Management — The Hard Part Nobody Talks About
The most common failure in multi-agent systems is not bad prompts — it is state management. When Agent A produces output that Agent B must interpret, the format has to be consistent and parseable. However, LLMs are inherently non-deterministic, so Agent A might format its output differently on every run, and a downstream regex that worked yesterday quietly breaks today.
The durable solution is typed state with validation at the boundaries. Use a Pydantic model for any structured handoff, validate at each transition, and include a fallback parser for the cases where the model drifts off-format. The example below shows how to coerce a free-text score into a guaranteed integer rather than trusting extract_score to always behave:
from pydantic import BaseModel, Field, field_validator
class AnalysisResult(BaseModel):
quality_score: int = Field(ge=1, le=10)
missing_topics: list[str] = Field(default_factory=list)
reasoning: str
@field_validator("quality_score", mode="before")
@classmethod
def clamp(cls, v):
# Models sometimes return "8/10" or "8.5" — coerce defensively
if isinstance(v, str):
v = float(v.split("/")[0].strip())
return max(1, min(10, round(float(v))))
# Ask the model for structured output, then validate before it enters state
structured = analysis_llm.with_structured_output(AnalysisResult)
result = structured.invoke(messages) # guaranteed shape, or it raises earlyEqually important, keep state objects as flat as possible — deeply nested state creates debugging nightmares, because a single malformed sub-field can be hard to locate. Checkpointing then closes the loop: a 20-minute workflow that fails at minute 18 should never restart from scratch. LangGraph’s SqliteSaver persists state after each node, so you can resume from the exact point of failure with the same thread ID.
Cost Management — Agents Can Get Expensive Fast
A quality-gate loop running five iterations, with three LLM calls per iteration, means fifteen API calls for a single task. At representative pricing of a few dollars per million input tokens, a complex research run can land somewhere in the range of a dollar or two — and that adds up quickly across thousands of users. The docs and most practitioners therefore recommend treating cost as a first-class design constraint, not an afterthought.
- Set hard maximum iteration limits (three loops is a sensible default for quality gates).
- Use cheaper models for routing and classification — a small, fast model for the supervisor, a stronger model only for the workers that need it.
- Cache tool results: if two agents need the same web search, run it once and share the result through state.
- Set per-request token budgets and abort gracefully when they are exceeded, returning partial work rather than silently looping.
For deeper context on giving agents tools safely and on persisting what they learn between runs, the companion guides on tool use and function calling and agent memory systems pair naturally with the patterns here.
When NOT to Use Multi-Agent Systems
For all their power, agent teams are frequently the wrong tool, and reaching for them reflexively is a common and costly mistake. If a task fits comfortably in a single context window and follows a predictable shape — summarizing a document, extracting fields, answering one bounded question — then a single well-prompted call is faster, cheaper, and far easier to debug. Adding a graph of agents only multiplies latency, cost, and failure modes without buying you anything.
Multi-agent designs earn their keep when the task genuinely decomposes into distinct skills, when intermediate quality control matters, or when independent subtasks can run in parallel. Conversely, avoid them when latency is critical and every loop hurts, when a deterministic pipeline of plain code would do, or when you cannot yet observe what each agent is doing. As a rule of thumb, start with one agent, prove you have hit a real wall, and only then split the work — because the orchestration you skip is the orchestration you never have to debug.
Related Reading:
Resources:
In conclusion, multi-agent AI systems are not just a research concept — they are the practical way to build AI applications that handle real complexity. Start with the pipeline pattern, add quality gates, isolate each agent’s tools, validate your state at the boundaries, and expand to more sophisticated topologies only as your use cases genuinely demand it.