The ROI of Planning Before You Prompt: What the Evidence Actually Shows

If you have read anything about software planning in the last twenty years, you have met this number: a defect caught during requirements costs 10 to 100 times less than the same defect caught in production. It gets cited in conference talks, vendor whitepapers, and, yes, on planning-tool blogs like this one.

I want to start by admitting that I do not fully trust it. And then explain what I do think holds up, because the honest version of this argument is more useful than the marketing version.

The number everyone quotes, and why it is shaky

The 10-to-100x figure traces back mostly to Barry Boehm's work in the late 1970s and early 1980s, later reinforced by Capers Jones. The underlying idea is intuitive: a mistake in a requirements document is a sentence you rewrite, while the same mistake in production is a migration, a support queue, and a postmortem.

The problem is the provenance. Laurent Bossavit spent a book, The Leprechauns of Software Engineering, chasing citation chains like this one back to their sources. What he repeatedly found was that widely repeated software engineering "facts" rest on small, old, thinly documented studies, often reinterpreted by each person who cited them. The cost-of-change curve is one of his central examples.

Two things are worth noticing about the original data. It came from large waterfall projects in an era when a deploy was a physical event, and it measured defects, not design decisions. Neither maps cleanly onto a solo founder prompting Cursor on a Tuesday night.

So when you see that statistic in a blog post arguing that you should buy a planning tool, treat it the way you would treat any number that is suspiciously good for the person quoting it.

What I think actually holds up

Strip away the multiplier and a plainer claim survives: some mistakes get more expensive with time, and they are not the ones people expect.

Code is cheap to redo now. That is genuinely new. If a component is wrong, you regenerate it, and the cost is a few minutes and some tokens. The old economics of "rewriting is expensive" have weakened considerably for anything that is purely code.

Data is not cheap to redo. Once real users have real rows in your database, changing the shape of those rows means a migration, a backfill, and a window where both shapes have to work. That asymmetry has not changed at all, and AI has made it sharper by making everything on the other side of it faster.

Decisions that are implicit are the expensive ones. Not "we chose wrong" but "nobody chose, and the tool chose for us, differently each time." That is where the real cost sits in AI-assisted work, and it is a category the 1980s data was not measuring.

If I had to compress the argument for planning into one sentence with no vendor incentive attached: plan the things that are expensive to change, and stop planning the things that are not.

The AI-specific evidence, honestly characterised

There is newer research that speaks more directly to AI-assisted development. It is worth reading carefully, because it does not say what planning-tool marketing usually claims it says.

Google's DORA 2024 State of DevOps report found that increased AI adoption was associated with a decrease in delivery throughput and delivery stability. Their modelling put a 25% increase in AI adoption alongside roughly a 1.5% drop in throughput and a 7.2% drop in stability. That is the opposite of the productivity story most people were expecting in 2024.

GitClear's analysis of over 150 million changed lines found a related pattern: duplicated code blocks rising, and refactoring activity (code moved rather than added) falling. More code, arriving faster, with less of it being consolidated.

Now the caveats, because they matter. Both findings are correlational. Neither study tested "teams that planned first" against "teams that did not," so neither one demonstrates that planning is the fix. A perfectly reasonable alternative reading of DORA is that AI increases the volume of change, and volume itself strains review and deploy pipelines regardless of how well anyone planned.

What the two findings do establish, and this is the part I would defend, is that the bottleneck moved. It is no longer "how fast can we produce code." It is "how fast can we agree on what the code should be, and keep twenty generated files consistent with that agreement." Planning is one answer to that. It is not the only one, and it is not proven to be the best one.

What this looks like in practice

The practical version is narrower than "write a spec first." Most of the value concentrates in a few places.

Decide your data model before you generate against it. This is the one that is genuinely expensive to reverse, for the reasons above. An hour here is not process theatre.

Name your shared components before the tool invents them separately. An LLM generating page A has no memory of the button it wrote for page B. If you do not declare the shared set, you get five of everything.

Write down what "done" means for anything with branching behaviour. Auth, payments, permissions. Not because ceremony is good, but because an LLM fills unstated requirements with plausible guesses, and plausible-but-wrong is the failure mode that survives review.

Skip planning the rest. Layout, copy, styling, one-off scripts, anything you would happily regenerate. Planning those is where "planning-first" turns into procrastination with a nicer name.

The honest limits of this argument

I build a planning tool, so read the above with appropriate suspicion. Here is what I cannot claim.

I cannot tell you that planning produces a specific percentage improvement, because nobody has run that study on AI-assisted development, and the older numbers do not transfer. I cannot tell you that it helps on every project, because on a weekend prototype it almost certainly does not. And I cannot rule out that the observed AI quality dip is a tooling and review problem that better tooling and review will solve without any planning layer at all.

What I will say is that the cost asymmetry between code and data is real, it is observable in your own project, and it does not require anyone's research to verify. Change a column type on a table with live rows and time it. That is the experiment.

Related reading

• Why AI-generated code breaks in production, which catalogues the specific failure modes behind the aggregate numbers above.
• From user stories to database schema, the one planning step I would argue is close to non-negotiable.
• What is vibe coding, on when skipping all of this is the correct call.

If you want to try the data-model step on your own project, the Database Schema Generator is free and needs no signup. Paste your user stories, get an ERD and PostgreSQL DDL back, and audit it yourself.

---

Sources and further reading

• Barry Boehm, Software Engineering Economics (Prentice Hall, 1981): the original cost-of-change curve.
• Laurent Bossavit, The Leprechauns of Software Engineering: the citation-chain audit that should make you cautious about the above.
• Google, DORA 2024 State of DevOps Report: AI adoption measured against delivery throughput and stability.
• GitClear, Coding on Copilot: code duplication and refactoring trends across 150M+ changed lines.