While preparing to teach an AI governance class at Improving, I compared adversarial agents to home inspectors.
In the classes I’ve been teaching, we describe an adversarial agent as a reviewer that tries to find what the previous agent’s evaluation missed. A home inspector does something similar. The inspector looks for problems, explains why they matter, and may recommend a specialist. The inspector does not make the repairs.
The report is the handoff. Someone else uses it to decide what to do.
One report, several decisions
One issue in my house was a bedroom window that sat too high to be an easy exit in an emergency. The house was built decades earlier, and the requirements had changed since then. The inspector explained the concern, but wasn’t going to alter the window.
Different people can read that same report and reach different decisions.
- A homebuyer may decide whether the house is still worth buying, or use the findings to negotiate.
- A lender may consider whether the property’s condition supports the mortgage.
- An insurer may look at the risks it would be taking on.
- The inspector’s employer may check whether the inspection followed its process.
The report is the same for everyone. What each person does with it depends on their responsibility.
That distinction helps me think about adversarial review. A reviewer needs enough information to judge the output against its purpose. It doesn’t need every detail about how the previous agent produced it. The author, tools, and steps shouldn’t matter. Start with the decision the review needs to support.
Review, report, hand off
An adversarial agent should find gaps and explain them. It can point to evidence and recommend what to check next. It should not take over the original agent’s job and apply a fix.
The person or process receiving the report can decide whether to accept the concern, ask for more information, send the work back, or bring in someone with deeper expertise.
The home inspector doesn’t need to be an electrician or plumber. If the report raises a specific electrical or plumbing concern, a specialist can take a closer look. The general inspection helps identify where that deeper review may be useful.
The same pattern applies to software work. A business analyst, user, stakeholder, platform engineer, or UX designer might each review a story. Each brings a different perspective. The review is useful when we know what decision that perspective should inform.
Start with the human workflow
When I look at a workflow, I start with three questions: who receives each output, what do they need to decide, and what happens when the output isn’t ready?
Those questions help me place AI in the workflow. Some steps may be suitable for AI to handle. Others may need a person, a tool integration, or a person working with AI. The choice depends on what the step needs to accomplish and who is accountable for the result.
When I map the human workflow first, I can name the input, the review criteria, the decisions available to the reviewer, and the point where a person or specialist needs to take over. Then I can decide where an AI agent helps instead of starting with an agent and searching for a job to give it.
I like this analogy because it makes the reviewer’s responsibility concrete: record what you found, explain why it matters, and hand the decision to someone who can act on it.
Leave a Reply