The Bottleneck Nobody Expects
Ask most people what limits how fast you can ship software with an AI agent, and they’ll say the obvious thing: how fast the AI writes the code. Buy more capability, generate more code, ship more often.
That’s not where the limit is. I know because I live on the other side of it.
The system I have in mind is the registration platform my collaborator Marek and I keep running for a global robotics championship — two volunteers, full-time jobs elsewhere, several pull requests a day. During a live registration window, there’s no such thing as “we’ll fix it tomorrow”; real operators are depending on it right then.
The code generation was never our bottleneck. The bottleneck — the thing that actually governs how fast you can safely ship — is how quickly you can trust what the agent wrote. Generation is cheap now. Trust is not. And trust is manufactured in exactly one place: review.
That’s the argument of this piece, and I’ll defend it plainly. In AI-assisted engineering, review is the product. Not a checkbox at the end. The main event. (If you want the bigger reframe behind this — why the engineer’s job moved to the two ends of the work — I made that case in From Vibe Coding to AI Engineering. This is the machine that makes it real.)
What “Review” Stops Meaning
Traditional code review asks one question: does the code on the screen look right? For an AI-assisted change, that question quietly breaks, for two reasons.
First, volume. When an agent can produce hundreds of lines in a minute, line-by-line reading stops scaling. You’d spend your whole day squinting at plausible code.
Second, and worse: the code can look completely right and still be wrong — wrong in ways that don’t live in any single line, but in the gap between what the code does and what you actually intended. Those are the bugs that ship. They pass the eyeball test precisely because the AI is good at producing code that passes the eyeball test.
So the question has to change. Not “does this look right?” but: “does this do what the plan said, no more and no less — and what happens to real people when it runs?” Answering that well, fast, repeatably, is the discipline. Everything below is how we make it tractable.
The Cheapest Review Happens Before a Line Exists
Everything I’ve said so far assumes there’s code to review. But the highest-leverage review happens earlier — before the agent has generated anything at all.
Look at the economics. If the approach is wrong, catching it in the plan costs you a sentence: “don’t touch that module — the constraint you’re missing is this.” Catching the same wrong approach after the agent has produced a full, polished, plausible solution costs you the entire diff — and worse, it costs you the pull to keep it, because it’s right there and it looks finished. Sunk cost is a real force even when the sunk thing was cheap to generate. A plausible-but-wrong solution is hardest to reject when it’s already sitting in front of you, done.
So the first review gate isn’t the code. It’s the plan. Before the agent writes a line, a human reads the intended approach and asks the direction questions: is this even the right change? Is the scope right? Did it catch the constraint that isn’t written down anywhere? A wrong plan caught here is a two-minute conversation. The same wrong plan caught after generation is a day you don’t get back.
This is why planning-first and reviewing-first are the same move seen from two sides. The plan I described in From Vibe Coding to AI Engineering isn’t only how you brief the agent — it’s the earliest, cheapest thing you can review, and the review you can least afford to skip. Review shifts left, all the way to before the code exists. Everything below is how you review what comes after that gate.
Scale the Scrutiny to the Risk
The first rule is the least glamorous and the most important: don’t review everything the same way.
A one-line copy fix and a change that touches how users are granted access are not the same animal, and treating them the same is how you either burn out or get sloppy — usually both. So the review effort is tiered to the change:
- A trivial change — a few lines, no data or permissions touched — gets tests and a sign-off. That’s it. Ceremony here is a waste.
- A medium change gets the full structured pass.
- A large or cross-cutting change — a new module, a schema change, anything touching access or money — gets the heaviest scrutiny we have, more than once.
This sounds like common sense. Almost nobody does it deliberately. The payoff is that you spend your reviewing energy where the risk actually is, which is the only way a two-person team survives a multiple-PR-a-day cadence without something eventually blowing up.
Many Reviewers, Not One Reviewer Wearing Hats
Here’s the part that took me longest to internalize, and it’s the part I’d most want another engineer to steal.
When you review a serious change, you want it looked at from genuinely different angles: Does this respect the architecture and stay in scope? What’s the blast radius if it fails, and is the rollback real? Does it actually deliver what the person who asked for it wanted? Will someone inheriting this code in six months be able to read it?
The tempting shortcut is to ask one reviewer — human or AI — to “cover all of those.” Don’t. One reviewer told to cover four perspectives produces a summary, not a review. It’s worth about one perspective of the four, because a single pass with a single context can only hold so much in focus at once, and it will quietly privilege whichever angle it noticed first.
The thing that works is independent passes, each with a fresh context and one job. Four reviewers each looking hard at one question will out-catch one reviewer glancing at four — every time. The whole value is the independence. The moment you collapse them to save effort, you’ve thrown away the thing that made the review worth running.
Bring In a Reviewer Who Doesn’t Share Your Blind Spots
There’s a second kind of independence, and it’s the one I’d call our secret weapon: on the riskiest changes, we bring in a second AI from a different vendor — a different model, trained differently, with different instincts about what’s suspicious.
The reason is simple and a little humbling. Every model has blind spots, and if your author and your reviewer are the same model, they share those blind spots exactly. A reviewer that thinks like the author isn’t a second opinion; it’s an echo. A model from a different lineage disagrees in useful places. It flags things ours waved through, precisely because it wasn’t trained to wave them through.
This is not “have AI review AI and call it safe.” It’s the opposite instinct: assume your primary tool has a bias, and deliberately recruit a reviewer that doesn’t share it.
Proof: Two Bugs That Passed the Eyeball Test
I mentioned two catches in From Vibe Coding to AI Engineering. Here’s what they actually teach about where in the machine each one got caught.
The first was a quiet privacy bug: one class of user could see a sliver of data that belonged to another. It was invisible in the diff — every line looked correct — because the bug wasn’t in the code’s logic; it was in what the running system would show to a particular kind of user. Our first pass, reading the change on its own terms, missed it. The catch came from a different-vendor pass that did the un-obvious thing: it traced, actor by actor, what each type of user would actually see on screen. It read the behavior, not the lines. That’s the cross-vendor layer doing exactly what it’s for — seeing what the author’s-eye-view didn’t.
The second was subtler. A small “New” badge — the kind that flags recently-changed items — would silently stop appearing under a change to how updates were grouped. No error. No failing test. Nothing wrong in the code as written. It was a regression that only existed in the rendered behavior a week later. A reviewer caught it by running the behavior forward in their head — imagining the state of the system next week, not just reading the state of the file today.
Neither of those was found by looking harder at the code. Both were found by looking at the behavior, from a vantage point the author didn’t have. That’s the whole game.
The Loop, and the One Step We Never Automate
Put together, a serious review is a loop: independent passes surface issues, the issues get fixed, and — on the heaviest changes — you go around again, because the fixes themselves deserve a look. AI does enormous work inside that loop. It drafts the passes, proposes the fixes, writes and even commits the code.
But the loop terminates in one place that stays stubbornly human: the merge. Nothing merges itself. No change reaches the system real people depend on without a person deciding it should — the person whose name is on that decision when someone later asks “who shipped this?” The agent writes and commits all day. A human merges. That single unautomated step is where accountability lives, and it’s the reason the speed doesn’t frighten me.
What I learned: when I stopped treating review as the tax you pay after the “real work” and started treating it as the work, the whole economics of AI-assisted engineering flipped. The bugs that would have shipped and cost a second fix — and a customer’s trust — get caught before they leave the building. The team’s scarce human attention goes to judgment, and the machine carries the typing.
Why it mattered: it’s the only reason two volunteers can responsibly run a system for thousands of people at a pace that used to require a team. Not because we generate code fast. Because we trust it fast — and we built the machine that earns the trust.
If From Vibe Coding to AI Engineering made the claim that the engineer’s job moved to review, this is the proof that the move is worth making. In a later piece, I get concrete about the unglamorous infrastructure — memory, guardrails, a review skill — that makes an agent trustworthy enough to run this way (The Prompt Is Overrated), and later still, what it felt like to point all of this at the system that runs a world-class AI competition (The AI Competition That Ran on Spreadsheets).
For now, one line to carry forward:
Generation is cheap. Trust is the product. Build the machine that manufactures it — and keep a human at the merge.