The problem is not solved by making their replies more persuasive. Someone or something must establish the current request, assign the work, control the action, and record the outcome.
Orchestration is the coordination around those steps. It manages dependencies, state, handoffs, resource limits, review, and recovery. It can coordinate ordinary software, people, and model-directed stages in the same process.
Choose a coordination pattern for a reason
Traditional workflow systems can be stateful, concurrent, and capable of waiting for events. Adding an AI stage changes how some decisions are made; it does not invent those operating capabilities.
| Pattern | Useful situation | Main question |
|---|---|---|
| Fixed sequence | The steps and order are known. | Are all prerequisites checked? |
| Conditional routing | A request belongs to one of several defined paths. | How are ambiguous or multiple intents handled? |
| Parallel work with a join | Independent subtasks can run together. | What happens if one result is missing? |
| Model-directed planning | Discoveries determine the next permitted step. | What bounds the plan and its revisions? |
| Specialist handoff | A distinct capability or authority is needed. | What context and responsibility transfer? |
A single workflow often combines several patterns. Anthropic’s guide distinguishes predefined workflows from agents that dynamically direct their process. That distinction is useful when deciding where flexibility is needed. Building effective agents.
Write down the job before dividing it
Consider an invented quotation-review workflow. A coordinator supplies three quotations. Independent extraction steps gather specified fields. A comparison step checks completeness and conflicts. A person reviews the result. Purchasing is a separate process.
Every stage needs a clear input, output, owner, and completion condition. “Researcher finished” is too vague. “Required fields assessed, source locations recorded, and unresolved items listed” is useful.
Do not split work just to give several agents different job titles. Separate stages when they need different information, tools, evaluation, or authority. A deterministic calculation may serve better than another model-based reviewer.
Use shared state deliberately
Record a job identifier, input revisions, current status, assigned stage, completed outputs, unresolved issues, and outstanding approvals. Store references to large artifacts rather than copying them through every message.
Distinguish a task’s current state from its event history. State says what is true now. History helps explain how it became true. Both can matter when a customer changes a request or a run restarts.
LangGraph documents checkpoints for thread state and stores for cross-thread application data. A persistent backend is needed if recovery must survive a process restart; an in-memory example is not durable storage. Persistence documentation.
Define which component may change each field. If several workers can update the same order, use the destination system’s supported concurrency and version controls. A generated statement of ownership is not a lock.
Make handoffs explicit
A useful handoff includes the task, relevant source references, completed work, unresolved questions, allowed next actions, and a receiving owner. Avoid sending the whole conversation when a concise state record is enough.
The receiving stage should verify required inputs before proceeding. If it cannot act, it should return an explicit status or route to an identified queue. Quietly dropping a message leaves the workflow looking busy while work is stranded.
For a human handoff, include a practical next step. “Needs attention” gives a reviewer little help. “Quotation B omits delivery terms; confirm them before comparison approval” identifies the decision.
Control concurrent work and stale decisions
Parallel extraction can be useful because each quotation is independent. Changing the same customer record from two branches requires a different design.
Use durable task ownership and destination checks appropriate to the system. If an ownership lease can expire, account for a delayed worker that continues afterward. The destination must reject stale or conflicting writes using a supported version or fencing mechanism; a lock by itself does not settle every race.
Before applying an approved change, check that the record and payload still match the reviewed version. If a customer changes the request, invalidate assumptions affected by that change.
When branches disagree, preserve the disagreement and identify the authoritative evidence. A majority vote among agents that used the same wrong source is not independent verification.
Plan recovery around side effects
A workflow may restart after a remote service accepts a request. The local checkpoint may still show the earlier state. Blindly replaying the step can duplicate the external action.
Use stable operation identities and service-supported idempotency where available. Otherwise, reconcile using destination records or preserve an unknown outcome for investigation. The AWS Builders’ Library explains the relationship between retries, request identity, and intent. Idempotent API guidance.
Recovery is not always rollback. A sent message cannot be unsent merely by restoring a database snapshot. Some processes need a compensating action, such as a correction message or a separately authorized reversal. Name that action and its owner in advance.
Stop new work when cancellation arrives, and reconcile requests already in flight. Keep cancelled, failed, awaiting review, and outcome unknown distinct so staff can see what remains to be done.
Measure the whole coordinated process
Track accepted outcomes, waiting time, review effort, retries, duplicate attempts, unresolved states, and cost. Separate active processing time from time spent waiting for a person or another service.
Parallelism can reduce elapsed time for independent work. It can also increase resource use. Additional agents can help separate concerns while adding coordination and review overhead. There is no universal token multiplier or latency saving to apply to every architecture.
Evaluate the complete path on representative cases. Include a missing branch result, duplicate event, stale approval, conflicting update, cancelled task, and restart around a remote write. A successful individual agent does not establish that the handoffs work.
Choose an orchestrator after identifying the needs
List the capabilities the workflow requires: persistence, waiting, concurrency control, retries, human review, observability, access isolation, and supported integrations. Compare actual documentation and a small implementation against those needs.
A managed service can reduce some infrastructure work. A custom service can offer specific control. Both need a maintainer and a recovery procedure. Product marketing, simulated-agent research, and an unverified personal story are not substitutes for evidence about your workflow.
Start by mapping one job from intake to verified completion, including its failure branches. Then use agent development to build the controls and monitoring and recovery to plan daily operation.