The first results from my workflow were rough. Out of 13 attempts, only four worked as expected. That is about 30 percent. I had connected steps that looked sensible on a diagram, but the early runs showed me where the handoffs were weak.
Some features had authorization problems. Dark mode failed in ways I should have caught earlier. The end-to-end coverage was not strong enough. A workflow can move work forward and still pass a bad assumption from one step to the next.
The first Talks Insight batch passed 4 of 13 attempts. The failures gave me specific handoff and verification gaps to investigate.
A sequence is not yet a trustworthy workflow
The harness moved a feature story through planning, implementation, tests, verification, and logging. I describe the current shape and some of its checks in Building a Development Harness I Can Trust. This post is about what the early failures taught me to pay attention to.
The plan step needed to produce something a person could inspect before the implementation began. The verification step needed to challenge the work, not just repeat the implementation agent’s confidence. And I needed to open the feature in a browser myself. Automated tests could pass while a screen still looked wrong or felt confusing.
That is why I treated the handoffs as part of the design. A strong output from one step does not guarantee a useful input for the next. Testing the Wrong Unit made a related point about evaluating a skill versus evaluating the job around it. For the harness, I needed to observe both the individual steps and the way their outputs traveled.
The checks have different jobs
I used deterministic checks where a rule could be stated clearly. If the output had to follow a particular shape, a hook could reject it before the next step. Those checks were useful because the same rule could run every time.
An adversarial review had a different job. It looked for problems that a simple structural check could not detect, including issues in the implementation and the interface. Neither check replaced my judgment. The plan and the finished feature still reached a person at decision points.
At one checkpoint, I reviewed the plan before implementation. Later, when the automated work had passed its checks, I opened the application and tried the feature. I could reject the result and send it back for changes, or approve it so the workflow could finish its logging and audit steps.
The video Stage 3 to 4: The Arrows on the Workflow Map captures the question I was working through: what governs the transition from one agent’s work to the next? The AI Maturity Model playlist is the larger journey. My practical answer was to put a check or a human decision at the handoff where the risk justified it.
Failure gave me a place to improve
I did not try to fix everything by adding more steps. I looked at what the workflow logged, what the review agents caught, and where I had to send work back. When an agent missed a dark-mode issue, I could strengthen the skill, add a check, or change what the reviewer was asked to inspect.
Sometimes the system caught an issue and sent the work back. Sometimes I had to notice the problem myself and explain what was wrong. Both outcomes were useful if they left a record. A silent failure would have given me no clue whether to change the prompt, the check, the test, or the workflow contract.
By the time I submitted the certification package, I had built 18 real features through the harness. That number came after many runs and corrections; it does not erase the four successful attempts out of the first 13. In the later harness write-up, I described how I continued using the workflow and tracking costs and rework.
A later Talks Trends batch gave me a different measure: 10 of 10 distinct slices resolved by per-slice final outcome. That was not 10 uninterrupted runs. In the broader 18-feature submission, 12 were full attempts and six stopped at human punch-out gates. I keep those results separate because the denominator and method change what the number means.
The Talks Trends result is reported separately from the broader 18-feature submission.
The Stage 3 series includes examples of improving individual skills through evaluation, including the Cypress-to-Playwright migration. At Stage 4, I needed the same willingness to learn, but I had another thing to evaluate: whether the connections between those skills were safe and useful.
The process did not become reliable because I added an agent with “review” in its name. It got better as I tied the review to specific risks, gave it evidence to inspect, and kept the human decision where the consequences called for one.
I trust the workflow more when I can see where it failed, what caught the failure, and what changed afterward. The handoffs are where that trust gets tested.
Leave a Reply