The Demo That Couldn't Ship
The first time I watched an AI agent build a working feature from a single sentence, I felt two things at once: awe, and a small, familiar unease.
The awe is easy to explain. You describe what you want, and the thing appears — complete, plausible, running. It feels like magic, and for a certain kind of work, it nearly is.
The unease took me longer to name. Then I placed it. I've spent over fifteen years around robots, and I've learned to distrust anything that looks finished before it has done any real work. I've built machines that were beautiful on the bench and useless on the field. The AI demo had that same quality — gorgeous, confident, and completely untested against the world it would actually have to survive in.
That gap — between looks right and is right — is the whole story of what's happening in software right now. And it's why I've stopped talking about "vibe coding" and started talking about AI engineering.
What "Vibe Coding" Is Good For (And What It Isn't)
The term comes from Andrej Karpathy, early 2025: describe what you want, let the AI write it, ship it. Don't read the code too closely. Go with the vibes.
For a weekend prototype, a landing page, a one-off script — this is wonderful. The surprise is acceptable, because nothing is riding on it. If it breaks, you shrug and try again.
Now move that same request into a system real people depend on. In my world, that's two of them: the registration platform that thousands of students and volunteers use to get to a global robotics championship, and — in my day job — engineering inside a regulated financial industry, where a plausible-looking mistake isn't a shrug. It's a customer-facing defect, a compliance problem, a number that's wrong on someone's account.
In those systems, "plausible" is not the bar. The bar is correct, defensible, and reviewed.
There's a piece of robotics education wisdom I keep coming back to here. Damien Kee, in a RoboCup workshop paper, argued that robots fail in classrooms when they're treated as toys — objects of entertainment — and succeed when they're treated as tools that extend what a student can do. As a RoboCup trustee, I've watched that play out for years, so the distinction isn't borrowed for me. It's lived.
AI in software has the exact same fork. Treat it as a toy — let's see what it generates — and you get demos and debt. Treat it as a tool — let me plan the work so the tool can do its part well, then check the result against the plan — and you get shipped software. Same model. Same tokens. Completely different outcome. The variable was never the tool. It's the practice around the tool.
The Context Gap
Here's what a vibe-coded request actually contains: a goal, an aesthetic, and a tolerance for surprise. Here's what it leaves out — everything that makes software survivable in production:
- System context — the architecture, the dependencies, the contracts with neighboring services.
- Risk context — what happens when this fails, who is harmed, what the rollback looks like.
- Quality context — the standards this codebase already holds itself to.
- Regulatory context — what an auditor will read, what an examiner will ask.
When you hand an AI a thin request, it fills the gap with guesses. The guesses are usually plausible. But plausible guesses are exactly the failure mode I recognized from the robot bench — the thing that looks finished and isn't.
This is what I've come to call the Context Gap, and here's the important part: it is not a flaw in the model. It's a flaw in the request. And a flaw in the request is something an engineer can fix.
The Job Didn't Shrink. It Moved.
The loudest fear about AI is that it replaces the engineer. I think that fear reads the situation exactly backwards.
What used to be the visible craft — typing the code — is now the cheapest part. But the rest of the job didn't disappear. It moved. As the developer Peter Steinberger put it, the hard thinking — planning, system design, dependency choice — is the engineer's remaining job. AI accelerates execution. It does not replace thinking.
So the engineer's value moved to the two ends of the work: upstream, into planning, and downstream, into review. In the middle — the typing — the agent takes over. The engineer becomes the architect of the request and the auditor of the response.
That's not a smaller job. On a hard system, it's the harder job. And it comes down to two habits.
One: write the plan before the agent touches code. Not a novel — a page. What is this change for, in business terms? What's in scope and what's explicitly out? What can go wrong and who's harmed if it does? What's the quality bar, named as standards and not vibes? What evidence will prove it works, written before the change? A plan like that closes most of the Context Gap in one move — and, not incidentally, gives the reviewer something concrete to check against.
The common objection is "that's just a design doc, we've always had those." Correct — and most of us quietly stopped writing them well, because doing it right felt slower than shipping. AI changed that math. A good plan is now the cheapest accelerant for everything downstream. The plan was always worth writing. AI made it pay back in days instead of quarters.
Two: review the result against the plan, not against the code. The wrong question for an AI-assisted change is "does this code look right?" It can look right and be wrong in ways you'll never spot line by line, and there's too much of it to read that way anyway. The right question is: does this deliver what the plan said it would — no more, no less? The bugs that matter usually aren't in the code. They're in the gap between what the code does and what the plan promised. (How that review actually works when a fast agent is producing the changes is the subject of a later piece, Review Is the Product.)
Proof, Not Theory: A System Real People Depend On
I want to be concrete, because this is easy to wave your hands about.
The RoboCup registration platform is run by a two-person volunteer team — my collaborator Marek and me — both with full-time careers elsewhere. It serves participants across 45+ countries, and it has to work during live registration windows when real operators are depending on it in real time. We ship changes to it at a pace of multiple pull requests a day.
That cadence is only sustainable because the discipline is codified, not improvised. Every non-trivial change gets a review pass scaled to its risk — a one-line fix and a cross-cutting refactor do not get the same scrutiny. Larger changes get multiple independent review passes, each with a fresh perspective, because a single reviewer told to "cover everything" produces a summary, not a review. On the riskiest changes we even bring in a second AI from a different vendor — a different model with different blind spots — precisely so it catches what ours didn't think to look for.
Two moments make the point better than any theory.
Once, a change shipped past our first review with a quiet privacy bug — one class of user could see a sliver of data belonging to another. It wasn't visible in the diff. It was caught by a review pass running a different model, because that pass read the behavior, not just the lines — it traced what each kind of user would actually see on the screen.
Another time, a reviewer flagged that a small "New" badge would silently stop appearing under a change to how updates were grouped — a regression nobody could see by reading the code, because it lived in the rendered behavior a week later. The reviewer caught it by running the behavior forward in their head.
Here's the part I care about most. In that whole system, nobody ships alone. The agent never picks what to build. Nothing auto-merges. And there is always a human whose name is on the decision. The AI is astonishingly useful inside those guardrails. The guardrails are what make the speed safe.
Keep It Human
So here is the line I'd defend anywhere: the AI writes the code — it even commits it — but a human merges it. The merge is where accountability lives, and it's the one step we never automate.
When an auditor — or a customer, or a teammate six months from now — asks "who decided this?", the answer has to be a person, not a prompt. That isn't nostalgia for a world where engineers typed everything by hand. It's accountability, and it's the one thing the tooling cannot take over, because accountability is the thing we least want to automate.
It's also, conveniently, where the rules are heading. The EU AI Act's transparency obligations become applicable on August 2, 2026, with more supervisory attention to generative AI building close behind. Planning, review, documentation, human accountability — the disciplines that make AI safe are turning out to be the disciplines regulators are about to require. The engineers who treated AI as a tool will already be standing there when that arrives. The ones who treated it as a toy will be explaining a year of accumulated debt.
What I learned: the arrival of AI didn't make engineering easier. It made the thinking matter more, by making the typing matter less. The craft moved to where it was always most valuable — deciding what to build, and being answerable for it.
Why it mattered: it changed what I look for in engineers, and in myself. Not "who can produce the most code" — the machine wins that now — but "who writes the clearest plan, asks the sharpest review question, and puts their name on the result with a straight face."
That's the whole shift, in one line:
Treat AI like a tool, not a toy. Plan the work. Review against the plan. Ship something real.
The demo will always look finished before it has done any work. Our job — the human job — is to know the difference.