Every agent I’ve worked on has gone through the same awkward adolescence, and these days it happens faster than anyone can watch. A tool starts timing out and a retry appears, the model returns something unparseable and a guard appears, somebody wants a human approval step and that appears too. None of it is unreasonable in isolation, every bit of it passes review, and by Thursday there are four levels of indentation in there that nobody has ever read top to bottom, because nobody sat down and wrote them in the first place.
That pile of retries and guards and early returns is a graph. It has nodes, it has edges, it has a state object dragged along between them. The difference from the old days is that there’s no author to go and ask, since the person who would have remembered why the retry is set to two attempts didn’t type it. So when someone wants to know what happens after the third tool failure, everyone opens the file and starts tracing with a finger, which is a strange way to run a system you’re charging money for.
Graph engineering is the fairly unglamorous act of writing that graph down on purpose, and I think it matters more now than it did when we typed everything, not less. The code is the cheap part now, since an agent will regenerate it on request and mostly do a decent job of it. What it won’t do is invent the structure you actually meant, because that lives in your head, or in a Linear ticket, or in a conversation from three weeks ago. Write the graph down and you’ve got the one artifact that survives the implementation being thrown away and rebuilt, which at current velocity is roughly weekly.
Three parts, and nothing else
The vocabulary here is refreshingly small, which I appreciate in a field that invents six new words a week.
A node is a unit of work, meaning it takes the current state, does one thing (calls the model, hits an API, runs a validation, waits for a person), and returns. An edge is a transition that node is allowed to make. State is the typed value riding along the edges, and it’s the only thing nodes are allowed to share.
That’s the entire vocabulary, and the power in it comes less from the three nouns than from the two restrictions hiding behind them, which are that a node can only go somewhere it declared an edge to, and it can only communicate through state. Those two rules are what turn a nice diagram into something the program will actually hold you to.
The same flow, twice: once where you have to read the nesting to see it, once where you don’t
If you’ve used LangGraph, this is more or less what it hands you on day one, with a StateGraph over a state schema and nodes joined up by edges and conditional edges. pydantic-graph arrives at the same place from the type system, reading a node’s outgoing edges off its return annotation, or letting you wire the steps up by hand if you’d rather be explicit about it. Either way you end up with the graph as an actual value sitting in your program rather than an idea spread thinly across it.
None of which is new, incidentally, since workflow engines have been describing work this way for decades and doing it rather well. The genuinely new part is that a model now picks some of the edges at runtime, and that turns out to be where all the interesting design pressure lives.
Determinism is a property of edges, not systems
The argument everyone seems to be having is “workflow or agent.” Either you hard-code the steps and get something reliable, or you let the model drive and get something flexible, and you should pick a lane. I’ve watched a lot of teams pick a lane, get burned, and then sprint to the other lane, where they get burned differently.
The nice thing about having an actual graph is that the question stops being a single choice about your system and becomes a series of small choices about individual edges. Every edge has someone who picks it, and there’s no rule saying it has to be the same someone twice.
| Picked by | Reach for it when | Cost |
|---|---|---|
| Code | the choice is a function of the state | nothing |
| The model | the next step depends on judgment about unstructured input | a model call, and a branch you can’t predict |
| A person | the call needs authority, not just judgment | wall-clock time, sometimes days |
My honest sense is that most edges people hand to the model are edges a conditional could have handled perfectly well, and every one of those is a model call you’re paying for in order to receive an answer you can’t predict. “If the record parsed go to write, otherwise go to repair” is an if statement. It has been an if statement since about 1957 and it does not need a language model to help it.
Where you genuinely do want the model, the thing that keeps it safe is a closed target set, so it’s choosing among transitions you declared rather than freestyling about what happens next. And on a person’s edge the graph simply stops, puts the state somewhere a human can look at it, and waits, which is a much saner thing than whatever you were doing before with a polling loop and a prayer.
Same graph, three different answers to “who picks this edge”
I want to be careful here, because there’s a version of this pitch that oversells badly. Graphs do not tame the chaos, because you get exactly as many failure modes as you had before, and a model that was capable of picking a daft edge is still perfectly capable of picking a daft edge. What changes is that every failure acquires a name and an address. “It hung” turns into “enrich retried three times against a rate-limited endpoint and then took the escalate edge,” which is a sentence you can actually do something about. The chaos was always there, it simply didn’t have a name attached to it.
And once the decision points have names, improving the thing becomes a series of small moves instead of a rewrite. An edge the model keeps getting wrong is an edge you can quietly hand to code. A branch nobody has taken in three months is a branch you can delete. Neither of those is available to you when the decision is buried in a prompt inside a loop.
State is what makes it enforceable
A graph that passes an untyped dictionary between its nodes is a picture, not a constraint. Every node can read anything, write anything, and develop a quiet dependency on a key that something else set three steps ago. Then someone renames the key, and you find out about it at runtime, in production, in a branch that runs one time in forty.
Typing the state is what turns the picture into something with teeth, and it’s the part of the stack where Pydantic really earns its keep, with two useful things falling out of it.
The first is that structured output becomes the seam between the model and the rest of your program. A node that calls a model hands back a validated object rather than a string you parse three functions later while hoping for the best. Pydantic AI is built around this, so you declare the output type and validation happens right at the node’s edge, which is a much better place for it than wherever it was happening before.
The second is that a validation failure becomes an edge instead of an exception. The model returns something that doesn’t fit the schema, and rather than blowing up, the graph takes a declared transition to a repair node, which gets one more attempt with the error included, and a declared path to give up if that also fails. The failure lives in the graph, so you can see it, count it, and put a ceiling on it.
There’s a third thing you get if you push a bit further and have each node declare which fields it reads and which it writes, which is that the dependencies become inspectable before anything runs at all. A node writing a field nobody reads is dead weight. A node reading a field that nothing upstream writes is a bug you just found without executing a line. It’s the cheapest static analysis available in this whole design and I’ve almost never seen anyone build it, though I should admit it only holds while your edges are static. The moment a model is picking one, “upstream” becomes a set of possibilities rather than a path, and the check softens into a warning.
Where to cut the nodes
Nobody really gets stuck on what a node is, they get stuck on where to put the boundaries, which is a much harder question wearing an easier question’s clothes. Cut too fine and you end up with forty nodes that each add one field to a dict. Cut too coarse and you have a single node called run_the_agent, which is just a while loop that has been through a rebrand.
The tempting heuristic is that a node should be the smallest thing you’d want to retry, resume from, swap out, or test on its own. It’s a perfectly good instinct and it falls apart almost immediately, because those four verbs want four completely different sizes. Retry wants small, per HTTP call, in process, with a backoff. Resume wants large, because every resume point is a serialization, a write, and a state schema you now own forever and get to migrate.
So you let the expensive one win: cut a node boundary where durability is worth paying for, and handle everything else inside a node. In practice that lands you in two fairly reliable places, plus anywhere you want a decision on the record.
- Contact with the outside world, when the call is expensive or non-idempotent. A cheap idempotent GET can happily retry inside a node, whereas a payment really cannot.
- A person, always and without exception, because approving a refund or reviewing a redline is a pause that might last four days, and a node boundary is the only place execution can legitimately stop and be picked up later.
The seams are already in the problem; the heuristic tells you which ones to cut on
Boundaries drawn that way tend to pay for themselves about three times over. A run’s position becomes a node name plus a state object rather than a stack frame, which means a LangGraph checkpointer or a durable execution engine can write it down and pick it back up. Retry and timeout policy attaches to the unit of work rather than the entire run. And the trace starts reading as a sequence of transitions rather than a wall of log lines, which is a genuinely nicer thing to hand someone who’s asking why the agent did what it did.
Two caveats worth knowing before you turn any of this on. The interrupted node re-runs in full when you resume, so switching on durability before your nodes are idempotent is a fun new way to charge somebody twice. And adding a boundary later is only cheap while nothing is persisted, because the moment you have live runs, splitting a node means those runs come back into a shape that no longer contains the node they stopped at. Get the coarse boundaries in early, even if you get all the fine ones wrong.
The agent becomes a node
Here’s the part I find most useful, and also the part most likely to be oversold at you by someone with a diagram.
The loop you already have doesn’t get thrown away when you adopt a graph. It becomes one node inside a bigger one, with typed state going in, typed state coming out, a timeout, and a set of declared edges. What that buys you is a contract at the boundary. The surrounding system stops caring that there’s a model in there at all and starts treating it as a step that either produces a valid output or takes a failure edge. Two agents that need to coordinate become two nodes with an edge and a shared schema between them, which is a considerably smaller problem than two event loops trying to have a conversation, mostly because the schema forces you to write down what one of them actually owes the other.
The contract holds at the boundary; the checkpoint stops there too
What the wrapper does not buy you is anything at all about the inside of the box, and this is where I’d push back on the enthusiasm.
- The sub-agent isn’t resumable just because the wrapper is. Your checkpoint knows you’re at node
researchwith a particular state going in. It has no idea the agent was eleven turns into a twenty-turn loop. Resume restarts the whole node, which means the bigger the agent inside, the more your best property degrades in exactly the place you were counting on it. - Nested state gets expensive faster than you’d expect. Either the parent carries the sub-agent’s message history, and you’re now serializing a growing transcript at every step along with whatever PII happens to be sitting in it, or it doesn’t, and you’ve thrown away the only context that explains the output.
- A pause inside a sub-agent has to bubble up. If the inner agent needs a human, that interrupt has to surface through the wrapper to whatever is driving the outer graph, and frameworks vary enormously in how gracefully they handle it.
- A typed output won’t catch plausible garbage. The failure you actually get is forty turns of work returning a beautifully well-formed object that happens to be wrong, and schema validation waves it straight through. A timeout and a token ceiling are a budget, not a correctness check. Checking the content is its own job and it belongs on an edge out of that node.
None of which is a reason not to do it. The agent gets considerably worse at surprising you, which was the entire point, and no better at being right, which was never what the wrapper was for.
Against my own argument
The easy version of this section is that small things don’t need graphs, which is true, costs me nothing to say, and helps nobody, so here’s the version that actually costs me something.
A closed edge set buys reliability by spending capability, and that’s a real trade rather than a clever framing. For an open-ended research or coding agent, the useful behavior is precisely the model stringing together steps that nobody enumerated in advance, and every edge you declare is a ceiling on what it’s allowed to discover. Those agents want a big tool catalogue and a loop, with the graph wrapped around them rather than threaded through them, giving you a budget, a termination condition, and somewhere sensible to put the human. Business process work runs the other way entirely, because refund approval, NDA redline, document extraction and reconciliation all have a shape that somebody could draw on a whiteboard, and in that world the closed set is the whole point.
The better objection is that this might all be scaffolding. Declared control flow exists because we can’t yet trust a model to sequence its own work, that gap is closing, and structure built to compensate for a temporary weakness has a habit of getting deleted. I think that’s directionally right and it doesn’t change my recommendation, because when the model does get good enough, handing an edge back to it is a small and pleasant afternoon. You delete a conditional, open up the target set, run the evals, and go home. In a nested loop you’d first have to find every place that decision is being made, which is a considerably worse afternoon. Being easy to dismantle is worth something, and the implicit version doesn’t offer it.
The honest cost in the meantime is indirection, since following declared edges across a set of nodes is genuinely worse than reading top to bottom, and for a flow that fits on one screen, top to bottom wins and you should let it.
Where to start
If you’ve got an agent in production and this all sounds like work you don’t have time for, the first pass is smaller than it looks.
Write down the edges you already have, using the control-flow keywords in your loop as the list, because they are the list. Count how many of the model-picked ones are choices a conditional could make and move those first, since they’re free and they’re where most of your non-determinism is quietly coming from. Give whatever edges the model keeps a closed target set, and put a real node wherever a person enters, because you’ll want that boundary the first time somebody asks why a refund took a week. Leave the coarse boundaries where durability is worth paying for, and don’t lose an afternoon making anything else a node.
You don’t need a framework for any of this, you just need to be able to point at the graph.