Your Coding Agent Is Making Architectural Decisions Without Telling You

TL;DR — Give the same underspecified task to ten independent coding-agent sessions and you get ten reasonable, mutually incompatible implementations: three different exceptions for one missing file, with no shared narrow exception contract covering them all. The agent isn’t writing bad code — it’s silently making architectural decisions nobody decided. And even with a precedent it read, 9 of 10 forked an incompatible variant. The fix isn’t smarter prompts; it’s making decisions explicit and machine-enforced, and escalating genuinely new ones. Experiments and raw data are linked at the end — one model, so replicate before you trust it.

Somewhere in your codebase, a coding agent just made an architectural decision. It didn’t flag it. It didn’t ask. It won’t show up in any commit message as a decision — it looks exactly like code, because it is.

Here’s the one I keep coming back to, because I built it on purpose to watch it happen.

I gave ten different agent sessions the same task — not ten tasks, the same one, word for word: implement load_spec(path), a function that reads an OpenAPI file, parses it, and validates it. Ten cold sessions, none able to see the others’ work. One ordinary function.

Then I asked the only question a caller actually cares about: when the file isn’t there, what do I catch?

Three different answers came back. Seven sessions let Python’s built-in FileNotFoundError propagate. Two raised a custom SpecNotFoundError. One raised a bare SpecError. There’s no single narrow exception type a caller can reliably catch across all ten: you either know every variant, catch a tuple of unrelated types, or fall back to the dangerously broad except Exception. A developer who writes the obvious except FileNotFoundError silently misses three of their own project’s loaders.

And that’s just the failure a caller hits first. Six of the ten hard-imported a YAML library, so the package won’t even import without it; four made it optional. Four defined a whole SpecError / SpecParseError / SpecValidationError hierarchy — and two of those still reached past their own hierarchy to raise the built-in FileNotFoundError for a missing file anyway. Same function. Same three failure cases. Ten mutually incompatible contracts.

Not one of them is wrong. Each is a coherent, defensible implementation on its own terms — the kind of code that sails through review, because there was never a bad answer here to catch. There was only a question nobody had decided, answered ten different ways: how does this project report failure? Drop any two of these loaders into the same codebase and you have a convention no one chose and no one can point to, sitting in the repo with exactly as much authority as a decision you made on purpose.

(These were ten runs of one frontier model. The setup, every session’s output, and a second controlled experiment are in the repo linked at the end — along with an honest account of what that does and doesn’t prove.)

That last part is the whole problem, and I’ll come back to it. First I want to explain how this happens, because the obvious explanations are wrong and they send you after the wrong fix.

It didn’t forget, and it isn’t too dumb

Two answers people reach for.

“The model forgot.” Nothing was ever written down to forget. Each session started cold, read some code, made a fresh call.

“The model isn’t good enough yet.” Comforting, because the fix is to wait.

Waiting won’t help, and it’s worth being precise about why.

A model trained on enormous amounts of code has seen result types and exceptions. Repository pattern and active record. Hexagonal, layered, vertical slice. Every one of them was correct somewhere.

Training gives the model a wide range of plausible patterns. Repository context, your instructions, and your tooling narrow that range — genuinely, and modern agents are good at it. But narrowing isn’t the same as choosing. Unless your project has actually committed to one option, several perfectly reasonable implementations survive the narrowing, and the agent picks among them.

Look again at what those ten sessions did. They didn’t spray randomly across the space of error-handling styles — every one of them converged on Python’s dominant idiom and raised. The narrowing worked. And it still wasn’t enough, because “raise an exception” isn’t a decision; it’s a category. Inside that one category the ten sessions produced three different exception types for a missing file and two different rules about whether the package even imports without a YAML library. Narrowing gets you to a paradigm. It doesn’t get you to a contract — and the contract is the part a caller has to write code against.

The paradigm itself may not be your project’s choice either. I ran a parallel experiment in TypeScript — not a controlled language comparison, because the task and ecosystem differed, but close enough to reveal another form of convention selection: parse an untrusted config string, validate three fields, report failures however you see fit. The result tracked the surrounding language culture. All six Python loaders had raised; all six TypeScript sessions did the opposite — every one returned a discriminated-union Result ({ ok: true, value } | { ok: false, error }), not a single throw. This experiment can’t isolate language as the cause — the tasks weren’t identical — but it shows how strongly the implementation contract gets shaped by context the project itself never chose. And inside the TypeScript six the contract still diverged: four put the payload on .value, two on .config; five discriminated failures on error.kind, one on error.code; one tagged a bad parse "invalid-json", another "invalid_json". A caller that reads result.value silently gets undefined on two of the six.

Your codebase doesn’t need a good answer. It needs the same answer, two hundred times.

Now, will better models fix this on their own? Partly, and I don’t want to overclaim. A stronger model does read your repo better, retrieve context better, reason across files better.

But notice what capability is measured on: finishing the task. Consistency with an intention you never wrote down isn’t part of that score. A model can get steadily better at producing valid implementations while your project stays exactly as vague about which valid implementation it wants. That gap doesn’t close from the model’s side, because nothing on the model’s side is pointed at it.

Design principles won’t close it either. SOLID, cohesion, coupling — those help you judge a design. They don’t produce one. Hand the same problem to two good engineers, both applying single responsibility properly, and you’ll get two different sets of boundaries. Both defensible.

Your architecture is the thing that picks. It says: here, in this project, the boundary goes here — for reasons that have nothing to do with SOLID and everything to do with what you know is coming in six months.

The agent has the principles. It doesn’t have the six months.

A convention now exists that nobody decided

Everything above is annoying. This part is expensive, and it’s the actual subject of this article.

Agents work out conventions by reading the code around them. That’s reasonable. It’s usually right.

Watch it go wrong. I took one of those loaders — the version that raises — and paired it with a validator I’d built the same way, a validate_spec that returns a list of every problem it finds. Two functions, one package. Then I started a fresh cold session with one job: add a require_valid_spec() entry point, and follow the existing conventions so it feels native. I did not tell it the conventions disagreed.

It read the repo and did something I didn’t expect. It noticed:

This codebase has more than one error-handling convention. loader.py raises… validator.py does the opposite — it returns a value.

Then it had to pick one. It chose the loader’s raise model — and to make the validator fit, it wrapped validate_spec’s list of problems and re-raised them as a SpecValidationError.

Read that again. A function deliberately built to return a list of problems now gets its output thrown by the function one layer up. The accidental convention didn’t just spread. It mutated — and it quietly began overruling the other one. The next session that reads require_valid_spec will see raising treated as the house style, and may “fix” the validator to match. Nobody will have decided any of this.

That’s the real problem. Not “the agent made a bad call” — obviously bad calls are easier to catch in review. This one hides because nothing was ever wrong.

Here’s the part that should bother you most. That session only told me about the conflict because I explicitly asked it to state whether the conventions were consistent. A normal task doesn’t ask that. “Add a require_valid_spec function” produces the exact same file — the wrapper, the re-raise, the silent overruling — with no mention that a decision was ever made. The agent picks, and the pick enters the repo carrying precisely as much authority as the decisions you made on purpose.

The obvious objection: modern agents don’t work blind. They search the whole repository. Surely they can just look at what the conventions are?

They can look. That’s the trouble. Your repository contains, all mixed together:

• architecture you decided deliberately
• architecture you inherited
• a migration you’re halfway through
• an experiment you never cleaned up
• a shortcut you took under deadline
• something a previous agent invented last month

The agent sees all six as code. Nothing in a file marks it as intentional, provisional, or accidental. Reading the repo tells you what’s there. It doesn’t tell you what’s meant. That fresh session read two files that genuinely disagreed and had no way — none — to tell which one, if either, was the decision.

So the real problem isn’t that agents can’t see globally — they can. It’s that correctness is easy to verify locally (tests, types, compilers), but architectural intent usually isn’t written anywhere a machine can check.

I’m not the only one hitting this. Scott Spence, working on production client systems, describes the loop precisely: one plausible shortcut lands, gets copied, and later sessions start treating it as how the app works. His framing is the sharpest I’ve read — LLMs are pattern followers before they’re engineers. They find the nearest thing that looks like it works and carry on from there.

The broader cross-session problem shows up in the research too. A recent paper on building a 108,000-line C# system with an agent reports that assistants lose coherence between sessions, forget project conventions, and repeat known mistakes — and that a single manifest file stops scaling past a modest codebase. Holding architectural integrity at that scale took them roughly 25,000 lines of specifications, prompts, and rules — a 24% knowledge-to-code ratio, across 283 sessions. That’s the surrounding context, not proof of my specific loop; my own experiment and Spence’s account are the closer match for that.

So stop asking it to remember

The fix is smaller and duller than people want. Not better prompts. Not a longer architecture document.

Don’t try to make your coding agent remember your architecture. Make your repository enforce it.

Decision	Write down	Enforce with
Formatting	barely	formatter
Type safety	barely	compiler flags
Dependency direction	why it points that way	architecture test
Module boundaries	the reasoning	import rule
Error model	why this model	lint + tests
API compatibility	the contract	contract tests
Performance	the target	benchmark in CI
Eval thresholds	the definition	protected baselines
Product direction	yes	a human

The middle column carries the weight. A lint rule can block the forbidden import. It can’t tell the agent the parser must not know about HTTP because we need offline validation — and without that, the agent routes around the rule instead of rethinking the approach.

Written instructions can be silently misunderstood or ignored. Automated checks make violations visible — and can prevent them from shipping. That enforcement is more dependable than endlessly refining the prompt.

The idea of encoding architecture as executable checks is not new. Neal Ford and his colleagues called these checks fitness functions in Building Evolutionary Architectures. Architecture tests, ADRs, and CI gates also existed long before coding agents.

What is new is the failure mode. These mechanisms document or enforce decisions a team has already made. Coding agents create a problem one step earlier: an agent can make a new design choice before anyone realizes that a choice was made.

That choice enters the codebase. Future developers and agents copy it, and it quietly becomes the standard.

Existing checks can’t catch this because no rule exists yet. Before a decision can be enforced, the team must first notice it, discuss it, and make it explicit. Most tools don’t help with that step.

We fixed the error-handling inconsistency with two files.

The first was an architecture decision record that documented the choice and its reasoning:

Loaders raise exceptions because ignoring a failed load could corrupt everything that follows. The failure must be impossible to overlook.

Validators return a list because their purpose is to report every problem. Raising an exception would stop at the first problem and hide the rest.

Different jobs require different contracts. The difference is intentional.

The second file was a small script that reads the decision as data and enforces it in CI. The current package passes. If a future coding session changes a loader to return errors instead of raising an exception, the build fails immediately.

The ADR explains the decision. The CI check enforces it. The remaining challenge is detecting the next new decision before it quietly becomes the standard.

Anthropic’s own documentation makes the same case for Claude Code hooks: they exist to give deterministic control, so certain things always happen rather than depending on the model to choose to run them. A pre-tool hook can block a write before the file exists. The docs put the split plainly — CLAUDE.md persuades, hooks enforce.

Evaluation tooling needs one rule stated explicitly: don’t forbid agents from editing tests — they legitimately add tests and fixtures while building features. The real rule is narrower: an agent must never weaken or rewrite the criteria its own work is judged by, just to make its implementation pass.

Baseline datasets, regression fixtures, security policies, minimum thresholds — those are yours. Adding new tests, fine. Moving the bar it has to clear, never. Tell an agent to keep going until the evals go green and it will find the cheapest route to green. Sometimes the cheapest route is the threshold.

Now don’t oversell it

The tempting next step: specify everything, enforce everything, tell the agent to run until all gates pass, walk away.

That doesn’t work, and I’d rather say so.

Take a real requirement — parse a 100 MB OpenAPI spec in under two seconds. Gates: correct output, under two seconds, no forbidden dependencies, tests pass, architecture rules pass.

One agent writes implementation A. Another writes B. Both green.

A is simple and easy to extend. B is clever, tightly coupled, and will be miserable the day you add another retrieval strategy.

No test tells them apart. And if you try to specify maintainability precisely enough that a gate could tell them apart — you’ve started writing the implementation yourself.

That’s the trap, and it isn’t a hole in your gates. It’s what gates are. A spec detailed enough to remove judgment is the program. Every gap in a spec is a decision handed to whoever implements it. Close every gap and you’ve written source code in a worse language.

METR’s research on how long agents can run hits the same wall. They measure an agent’s reliable task length at a 50% success rate — and note that aiming for 80% success instead drops the length you can trust sharply. Work that’s reliability-critical and hard to verify needs even higher success rates before automating it pays off.

Hard to verify is exactly implementation A versus implementation B.

So, the honest version:

Executable architecture doesn’t remove judgment. It removes the need to keep re-making decisions you’ve already made.

Four levels, not two

Level	The decision is…	The agent should…
1	decided, machine-checkable	just enforce it
2	decided, not fully checkable	follow the stated intent, get reviewed
3	not decided yet	escalate, not invent
4	product or architectural direction	leave it to you

Level 4 isn’t coding at all. Should evaluation chase maximum accuracy if each run costs four times more? Should graph retrieval be a core capability or stay experimental? Should a policy violation fail closed? No improvement in code generation should answer those on a founder’s behalf.

Level 3 is the one we don’t have a satisfying mechanism for — and I think it’s where the interesting work is.

Here’s the distinction that took me a while to see. The ten-loader experiment made it concrete. There are two very different things you might want to catch:

Violation detection. You broke a decision we already made. Linters are excellent at this. So are type checkers, architecture tests, CI thresholds. Mature tooling, solved problem. My error-model checker does this the easy way: a load_* function that returns errors instead of raising gets flagged against a decision already on record.

Decision detection. You are making a decision nobody has made before. Our tooling is much weaker at this. When I added a brand-new diff_specs function whose error style matched no existing rule, the same checker could only say: nothing here covers this — it’s undecided, escalate it. That’s the most it can honestly do.

The second is much harder because the code alone can’t tell you whether a decision is new. You have to compare it with the decisions the team has already made — and most projects don’t keep a clear, usable record of those.

I don’t have a clean solution. What partly works is making sure a new decision can’t slip in unnoticed: require a short decision record before a new module lands, flag any pattern that has no precedent in the repo, and send a first-of-its-kind file to review instead of merging it like an ordinary change. You can’t automatically detect that a question was never decided. But you can make it costly to set a new precedent in silence. Remember the session that added require_valid_spec — the drift only showed up because something asked the question out loud. That question is the whole mechanism.

That’s the frontier. It’s where I’d spend the effort.

Monday

• Go find your drift. Grep for two implementations of the same thing — two error models, two ways to build the same object, two names for the same failure. It’ll be quicker than you think. Mine took ten sessions and about two minutes to appear.
• For every convention you’re about to write as a sentence, ask: can a tool check this? If yes, write the config instead.
• Keep the always-loaded instructions small. Push detailed rationale into focused documents the agent pulls in when it needs them — the C# project above used exactly this split, a small hot context plus a larger on-demand knowledge base.
• Put the check in CI so it fails on the agent’s turn, not yours.
• Make sure the agent can’t lower the bar it’s being measured against.

The goal was never zero human intervention.

It’s zero human intervention for decisions already made.

Good architecture makes important decisions.

Agent-ready architecture makes them stay decided.

---

Methods & limitations

The experiments here are real and reproducible — code, fixtures, and every session’s output are in the repo linked below. The honest caveats, because they’re the difference between evidence and a vibe:

• One model. Every session was Claude Opus 4.8. This is a within-model result; cross-vendor behavior (GPT, Gemini, and others) is untested and is the obvious next study. If you run it elsewhere, I want to know what you find.
• Small N, one domain. 6–10 sessions per condition, one task family (a file loader/validator) in Python and TypeScript. Directional, not precise — no confidence intervals claimed.
• Objective extraction. Contracts were read from the produced code, not from the agents’ self-descriptions (self-reports were spot-checked against source).
• What’s new vs. prior art. Encoding architecture as executable checks isn’t new — fitness functions and ADRs predate agents. The contribution is the agent-specific mechanism (decisions getting made silently across sessions) and the violation-vs-decision-detection split.

Reproduce or audit the experiments. The public repository includes:

• Model and harness: Claude Opus 4.8; each run used an independent agent with fresh context and no shared state.
• Run dates: August 2026.
• Method: exact prompts, experiment configurations, dependencies, and reproduction steps are documented in METHODS.md.
• Raw outputs: per-session results are preserved in variants/, validate_variants/, ts_demo/, rerun_variants/, and exp2/.
• Study records: pre-registrations and complete results are in EVIDENCE.md and EXPERIMENT-2-precedent-propagation.md.

Because model outputs are nondeterministic, reruns may not produce identical code; the repository provides enough to repeat the method and compare outcomes.

Sources

• Scott Spence, How I Stop LLMs Drifting In Production Codebases — https://scottspence.com/posts/how-i-stop-llms-drifting-in-production-codebases
• Aristidis Vasilopoulos, Codified Context: Infrastructure for AI Agents in a Complex Codebase — https://arxiv.org/pdf/2602.20478
• METR, Task-Completion Time Horizons of Frontier AI Models — https://metr.org/time-horizons/
• METR, Clarifying limitations of time horizon — https://metr.org/notes/2026-01-22-time-horizon-limitations/
• Anthropic, Automate actions with hooks (Claude Code docs) — https://code.claude.com/docs/en/hooks-guide
• Anthropic, Steering Claude Code: when to use CLAUDE.md, skills, hooks, and subagents — https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more
• Neal Ford, Rebecca Parsons & Patrick Kua, Building Evolutionary Architectures (O’Reilly) — on fitness functions