There is a moment in every AI-assisted review where the change fills the screen and the summary above it says everything is handled. The summary is confident, well organized, and easy to read. That is the danger. There is simply more of it than there used to be, and the summary is what most people read first.
The prose is the least useful part of what came back. What matters is the behavior: what the system will now do, for whom, under what conditions, and what it will do when those conditions are not met. I own the diff. If it is wrong, the fact that I asked for it does not transfer the responsibility anywhere.
So the question I have had to answer for myself is how to review more code than I can read, without pretending I read it.
Do not ask the author to grade the paper
The first habit to break is the natural one: finish a change, then ask the same conversation to review it.
It cannot do that well. That conversation contains the reasoning that produced the change, so it evaluates the change against its own intent. It will confirm what it meant to do, explain why the approach was sound, and report that it is ready. Not because it is dishonest, but because that is the only perspective it has.
Open a new conversation. Give it the diff and the constraints, and nothing about how the change was produced. It should not know the intent, the steps, or the author’s justification. Then ask it to find what is wrong.
The framing matters more than the tooling. A fresh conversation with a mild request gets a mild review. Ask it to attack the change: assume there is a defect, find it, and explain what a reviewer should look at first. Give it the rules that apply, including the data handling obligations. Ask specifically about authorization gaps, missing negative cases, and assumptions that are not stated.
This is the same pattern I described in The Home Inspector as an Adversarial Agent. The reviewer produces a report of what looks wrong and why it matters. It does not fix anything. The report is the handoff, and the decision about what to do with it belongs to whoever is accountable.
One more thing: the report is not a pass. An adversarial review that finds nothing is not proof of safety. It is one more signal, and signals need to be weighed by someone who knows the system.
The person who already knows
Every team has someone who can look at a change and spot the problem in thirty seconds. They know the deployment quirk, the table that everyone treats as append-only, the integration that fails quietly. They are not doing anything magical. They have a set of checks they run without thinking about it.
That person is now a bottleneck in a way they were not before. When output triples, the reviewer’s capacity does not.
The answer is not to work them harder. It is to get the checks out of their head and into the system. Sit with them, ask what they look at and in what order, and write it down. Some of it becomes a prompt that goes to every review. Some becomes an analyzer rule. Some becomes a test. Some stays a conversation, because it needs judgment that is hard to encode.
The parts that get encoded do two things at once. They extend that person’s attention to every change instead of the ones that happen to cross their desk, and they make their judgment visible to the rest of the team. I have written about why code reviews were never meant to be about formatting, and this is the version of that argument for a world where the volume makes formatting review pointless: the value of a reviewer is the checks nobody else knows to run, and the highest-value move is to codify those checks.
Tests are the part you can trust
If instructions to a model are hopes, tests are evidence.
The distinction matters in regulated work because eventually someone will ask what you did to confirm that the change was safe. “I reviewed it” is weaker than it used to be, since there is more of it than anyone can review. “These checks ran and passed, and here is the list of what they cover” is stronger, because it is reproducible and it survives the reviewer leaving the team.
The tests that earn their keep are the ones aimed at the failure modes from the earlier posts in this series: the endpoint in the previous post that returns too much to the wrong caller, and the credentials that slipped into a request because a tab was open, which I got into in the post on writing the prompt.
- Negative cases: no identity, wrong tenant, expired session, insufficient role.
- Contract cases: the response shape is exactly what the caller needs and nothing more.
- Sequence cases: the downstream events still happen in the order the system depends on.
- Data handling cases: the fields that must not appear in a log or a response do not appear.
AI is good at generating the scaffolding for these. It is also good at generating tests that pass by testing the wrong thing, which is why a person has to confirm the coverage. The model writes the tests, the person confirms they test what matters, and the pipeline runs them every time. Nobody is reading the assertions on a 4000 line diff, but everybody can read a short list of what the suite is designed to catch.
Leave the receipts
Every request that produced a change should be traceable: what was asked, what came back, what was decided, and by whom.
I keep this for a practical reason as much as a compliance one. When something goes wrong, the first question is always why. Without a record, that question gets answered by memory and by whoever is most confident. With a record, the answer is on disk. You can see the request was incomplete, or the constraint was there and ignored, or the person reviewed it and missed something. All three are learnable. A vague memory of “we looked at it” is not.
In a regulated environment, this is also the evidence package. The ticket, the diff, the review reports, the approvals, the test results, all tied together. The automation can assemble it. The person still signs it.
That package only stays honest if someone owns keeping the standards current. A regulation changes, a new field gets classified, a retention rule is revised, and the copy in the repository drifts from the copy the auditor has. The automation will keep checking against whatever is on disk. So name the owner, put a review date on the document, and treat a change to it as a change to code.
Where the agent is allowed to go
One control that gets overlooked because it feels like infrastructure rather than process: what the agent can reach.
An assistant asked to make a change will sometimes decide it needs more context and go looking. It will read files you did not mention, and follow a reference into another part of the tree, and pull in a commit message from three weeks ago. Sometimes that is exactly what you want. Sometimes it brings back a specification that describes how the system used to work, or a story that has since changed, and now the code and the documentation disagree inside the same context. The result is worse than a narrow answer, because it is confidently built on a contradiction.
That gives me a simple rule for what belongs in the workspace and what does not.
The obligations and standards that apply to the system belong in the repository. They are current, they are the rules, and I want the agent to find them and use them.
Stale work items, old specifications, and notes that describe intent rather than current behavior do not belong in the repository. If the agent finds them, it will treat them as truth. If the code disagrees, the agent will try to reconcile them, and it will be wrong about one of them every time.
Beyond scope, there is permission. The practical boundary I have landed on is not the operating system. Managing filesystem permissions so that one process can read only certain directories is possible and it is also a great deal of work for a single developer’s laptop, and most operating systems were not designed for that kind of on-demand access control.
Containers were. Run the agent inside one, mount only what it should see, and let it have full access to everything in there. Then the boundary is the container, not a set of instructions the agent may or may not follow.
The longer-term direction, agents with their own identities and their own audit trail rather than borrowing yours, is appealing and also expensive. A separate identity costs money and administrative effort, and it has to be set up and maintained. Whether it is worth it depends on how much authority you are delegating. An agent that can only read a repository needs much less than an agent that can deploy.
Closing
The five posts in this series have been one argument. You can go faster than you ever could, and the faster you go the more the control has to be designed instead of assumed. Map the workflow first. Know which work AI improves and which it quietly breaks. Write the request like a small specification. Decide what defensible code means and encode it. Then review the behavior, with fresh eyes, with the checks the experienced person would have run, and with evidence that you did.
None of it is a restriction on speed. All of it is what lets me use the speed and still put my name on the result. That is what better brakes were always for: not to go slower, but to be able to turn when the road does. That part has not changed in three decades, and it is not going to change because the code arrives faster.
Leave a Reply