The most useful thing I can say about AI in a regulated codebase is that it is excellent at a specific set of jobs and quietly dangerous at a few others, and the two lists overlap more than people expect.
In the first post of this series, I argued that the workflow map comes before the acceleration. This is the second step: knowing which parts of that map AI actually improves, and which parts it makes look better while getting worse.
Where it genuinely helps
These are the places I reach for it first, and where I have seen teams get real value without much ceremony.
Drafting stories and plans. Turning a rough idea into a structured story with acceptance criteria, or a rough story into an implementation plan, is work that benefits from volume. The model produces a first draft in seconds. The draft is not the point. The point is that you now have something concrete to react to, which is easier than starting from an empty page.
Comparing options. For years the answer to “what else could we do?” was “we do not have time to explore that.” Now the exploration is cheap. Give the model the constraints and ask for two or three approaches, with the tradeoffs stated and a recommendation about which criterion should decide between them. You still choose. You just choose with more than one option on the table.
Explaining unfamiliar code. This one needs a habit attached. Before asking for an explanation, say what you are trying to find out and why. “I need to understand how personal data flows through this folder because it falls under the obligations that apply to this system. Explain the code from that perspective, not from a style perspective.” The explanation changes completely when you give it the lens. Without a lens, you get a description of what each line does. With one, you get a description of where the risk is.
Generating review questions. Ask what questions you should be asking about a change. You will not keep all of them, and a few will be wrong. But it surfaces questions you would have reached twenty minutes later, if at all.
Drafting tests. Test scaffolding is a good fit: the model can take the behavior you describe and produce the structure, and you spend your attention on whether the behavior is right instead of whether the setup is correct.
Reducing boilerplate. When the task and constraints are clear, this is the strongest case. It takes the repetitive work out of your head so your head is available for the parts that actually need a human.
None of that is the interesting part. The interesting part is what comes with it.
Where it costs you control
Every item below is a real failure mode I have seen or caused, and each one is more likely precisely because the output looks good.
Confident wrong code. The model will write an API call that does not exist, or use a library in a way that was never supported, and describe it as correct. Confidence and correctness are unrelated properties in generated text. The confidence is a style, not a signal.
Missing authorization. Ask for an endpoint that fetches a record and you get one that fetches a record. Not one that checks whether the caller is allowed to see it, and not one that limits the query to the caller’s tenant. Those requirements were not in the request, so they were not in the code. They are the first thing to disappear and the hardest thing to notice in review.
Business invariants nobody stated. The rule that a record of a certain type must never be deleted, that a sequence must stay in order, that a derived value is recomputed and never trusted from input. These live in people’s heads and in history, not in the code comments. A model working from the code will follow the code, including its mistakes.
Overexposure. Returning the whole entity is the default. It works today and becomes a data leak the day someone adds a sensitive field to the model, with no change to the endpoint and no reason for anyone to look at it again.
Agents editing what they should not. An agent asked to make a change will sometimes also update a config file, a migration, or a test fixture that happened to be in the way. It is trying to make the change succeed. Succeeding and being correct are different goals.
Context leaving the building. This is the one that has nothing to do with code quality. Whatever is in the editor, in the terminal, or pasted into the chat is now part of the request. Open tabs with credentials, a stack trace with a bearer token, a console dump with a customer’s data. The model does not have to be malicious to cause a problem. It only has to receive the data.
Authority you did not mean to delegate. If the process can deploy, and the agent runs the process, the agent can deploy. Nobody granted that permission on purpose. It came along with the access the agent inherited from you.
The pattern underneath
Look at that second list and notice what the items have in common. None of them is a model failure. Every one of them produces output that reads as correct.
That is why the risk profile of AI-assisted development is different from the risk profile of ordinary bad code. Bad code from a person usually looks bad. Bad code from a model looks like the rest of the codebase.
I have written about a workflow for building features with AI in the loop, and the part that matters here is that the loop needs defined checkpoints. Not because the model is untrustworthy in some abstract way, but because the specific failure mode is “looks fine”, and “looks fine” is the only thing a casual check can detect.
Where the brakes go
The rest of the series is organized around these gaps, in the order they occur.
The request comes first. Most of the risks above start as an incomplete or over-broad prompt, and the fix is to write the request the way you would write a small specification: one task, a stated role and reason, explicit constraints, a definition of done, and an instruction to surface assumptions. That is the next post, and it includes the one part people skip, which is what never goes into a prompt and what to do when it does anyway.
Then the output. Code that looks right is not code you can defend, and the difference is a small number of specific things: identity, authorization, tenant scope, and how much of the record you return. That is the post after that.
And then the sign-off. When there is more code than anyone can read, the question stops being “how do I review all of this” and becomes “how do I know what needs my attention, and how do I leave evidence that I looked.” That is the last post.
For now, the useful takeaway is narrower. Keep two lists. On one, the jobs where AI gives you speed you can absorb. On the other, the gaps that speed will widen. Most teams only have the first list.
Leave a Reply