Your AI Prototype Works. Now Comes the Expensive Part.

Every few weeks we hear some version of the same story: someone built a working app with an AI coding tool over a weekend, showed it to the team, and now the question on the table is how fast it can launch. The demo really did work — that's not in dispute. What's in dispute is what "worked" actually meant, because it meant something much narrower than most people building on top of it realize, right up until the invoice for finding out arrives.

None of what follows is an argument that AI-built software is unreliable, or that the tools are the problem. AI has made a genuinely hard part of building software — getting from a blank editor to something that runs — dramatically faster and cheaper than it's ever been. That's real, and it isn't going away. What it hasn't done is eliminate the engineering required to run that software safely once real users and real money are involved. If anything, that judgment is worth more now, not less, because there's more of it sitting unreviewed behind more working demos than at any point before.

What "it works" actually meant

A prototype has to be right once: on your laptop, with the three test accounts you created, clicking the same happy path you always click, while you're the only person using it. Production software has to be right constantly: for a user who pastes an emoji into a phone number field, for a database call that comes back slow at 2 p.m. on a Tuesday, for a queue that backs up because a downstream vendor is having a bad day, for a hundred people hitting the same endpoint at once instead of one. Those are different bars, not the same bar cleared at different levels of polish — and an AI coding tool asked to "build a signup flow" was solving for the first one, because that's the only one anybody showed it.

A prototype has to be right once. Production software has to be right every time.

AI can make architectural decisions — no one necessarily chose them

AI tools aren't incapable of reasoning about architecture. Modern models can discuss trade-offs, suggest patterns, and adapt when they're told more about a problem. The catch is the word "told." A model can only architect against the requirements and constraints it actually knows about, and a founder prompting "build me an appointment scheduling app" almost never states the ones that matter most: expected scale six months from now, compliance obligations, how fast the business needs to recover from an outage, whether one customer's data has to stay provably isolated from another's, what a vendor's SLA actually guarantees, how long records legally have to be retained. None of that is a secret the model is hiding. It's information nobody put in the prompt, because most people building a prototype don't yet know to ask the question.

That's a more useful way to state the problem than "AI writes bad code." We've written before about the distinction between code-monkey work and software engineering — turning an already-decided task into syntax versus deciding what the task should be in the first place — and this is that same gap showing up a level higher, at the scale of an entire system instead of one function. The code that comes out the other end of a prototyping session is often perfectly competent at implementing the assumptions baked into it: a single database table with no indexes because the demo only ever held forty rows, session state kept in memory because the demo only ever ran on one server, an admin route with no real authorization check because the only person who ever hit it was its builder. Every one of those is a coherent decision. It's just not one anyone made on purpose.

It's architecture nobody decided on.

Why "it works in the demo" doesn't mean it's safe to run

Every one of those undecided defaults has a dollar figure attached, and it usually shows up after launch, not before. A data model that's fine at a few dozen test rows can mean an unindexed table scan that gets slower every week as real usage grows, or a background job that fires once per row and quietly multiplies your API bill by however many users you signed up this month. An endpoint with no rate limiting doesn't matter until a bug in your own frontend, or someone else's scraper, hits it in a loop overnight and turns into a five-figure cloud bill by the time anyone notices. Two of the more expensive versions of this are common enough to walk through on their own.

The demo charged the card once. Production has to know whether it already did.

Here's the scenario in full, because it's worth walking through once: a customer clicks "pay," your server calls the payment provider, the provider processes the charge — and the response times out before it reaches your server. Your application never receives confirmation, so as far as it knows the request failed. The customer, seeing nothing happen, clicks "pay" again. Now there are two successful charges against one order, and nothing in the code ever decided that shouldn't be allowed to happen, because in the demo nobody ever double-clicked and nobody's network ever blinked.

That's what an idempotency check is for: a way of saying "if this exact request comes in twice, only the first one counts," usually by having the client attach a unique key to the attempt so the server can recognize a retry as the same request instead of a new one. It's a small piece of plumbing. It's also exactly the kind of decision a prototype has no reason to make, because a prototype is never retried by an anxious customer refreshing a page that didn't visibly respond — only production is.

Authentication says who you are. Authorization decides what you're allowed to see.

A prototype almost always gets the first one right, because the demo needs a login screen to feel real. It's the second one that quietly goes missing: a route like /api/customers/123/invoices that checks whether you're logged in, but never checks whether the invoices you're asking for are actually yours. Change the 123 to 124 in the browser's address bar, and you're looking at another customer's billing history — not because anyone hacked anything, but because nobody wrote the one line of logic that compares the ID in the URL to the ID on the session.

That gap is invisible in a demo, because the only customer who ever tests it is you, and you only ever ask for your own data. It's also exactly the kind of thing a reviewer who's actually read the code catches in an afternoon — and exactly the kind of thing that shows up as a headline if it ships first.

We've also written about why AI-generated code needs more testing, not less, once a model is doing the typing. The same logic applies here, one level up: the code that got you from idea to working demo by Friday is the exact code you're now proposing to trust with real user data and real money, and nothing about how quickly it got written checked whether it can survive that.

The human in the loop isn't a nice-to-have — it's the missing half of the job

Getting from prototype to production requires someone who can read the whole thing and explain what happens when each piece fails — not a fresh click-through to confirm the button still works, but an actual understanding of the data flow, the authorization model, and the failure modes. That person doesn't need to have written the code by hand. They do need to be able to answer "what happens if this call times out" without finding out live, in production, from a support ticket.

Whoever does this review is doing real engineering work, usually for the first time in that prototype's life — and it's worth being honest about what kind of person that takes. It's not someone who can produce syntax fastest under a clock; it's someone who can hold an ambiguous, already-built system in their head and reason about what's missing from it, which is a different skill than the one most technical interviews actually measure. You can see it in what they find: the auth check that only exists on the routes someone remembered to test, the retry logic with no limit, the field that's a string in one table and an integer in another because nothing ever forced the two to agree.

That reviewer doesn't have to already be on your team. Bringing someone in for a focused production-readiness pass is very often cheaper than the incident it prevents — the same way paying for an inspection before closing on a house is cheaper than discovering the foundation problem after you own it.

What this changes, in practice

Treat "the prototype works" as proof the idea is worth building, not proof the implementation is safe to build a business on — those are two different claims, and only one of them got tested by a demo. Before anything touches real user data or real money, put it in front of someone who didn't build it and have them answer the questions a demo never has to.

If you can't answer several of those with confidence, that's not necessarily a bad application. It's a prototype that's ready for production engineering, not yet a production system. Budget the hardening pass as its own line item — the way you'd budget an inspection — instead of squeezing it into launch week as a handful of tweaks.

Ask the same question about your CI pipeline that we've asked about testing AI-generated code more generally: is anything actually gating this before it reaches a real user, or is "it worked when I tried it" still the whole review process?

Where this leaves you

Cheaper implementation and cheaper judgment are not the same trend, and this is really an article about the second one. A working AI prototype is proof that the idea is good. It is not proof that the implementation is safe to build a business on, and the two are easy to confuse because the tool that got you from idea to demo is the same class of tool a lot of the industry is now telling you can also ship your production system. "Can generate a working demo" and "was designed to survive contact with real users" are different claims — only one of them actually got made. If you've got a prototype that's ready to become a real product, tell us what you're building and we'll help you find out what it actually takes to get there safely, before your users find out for you.