I had been focused on getting a working harness. Once it could take a feature through the workflow, I could see an outcome. But when it was time to prepare my Stage 4 submission, I realized I also needed to understand how the machinery produced that outcome.
I had to explain the workflow, the inputs and outputs, the guardrails, the human checkpoints, and the way failures were handled. Building something that worked had not automatically taught me every part of how it worked.
The records helped me retrace the work
While I was building the harness, I kept asking AI to log what we had done. Those activity logs captured decisions, failed attempts, and the next pieces of work. When I started preparing the talk and certification package, I had already forgotten some of the early steps. The records helped me reconstruct the sequence.
The harness also recorded what happened at each step, including failures, rework, and model-call costs. In my earlier write-up about the harness, I shared the cost breakdown and described how I used rework as a signal for where a skill might need improvement. Here, those same records became part of my learning: they showed me where the system struggled and gave me details to investigate.
The harness kept its run history, and I also kept activity logs in the project and in Obsidian. When a thought about another task came up, I captured it without stopping the work in front of me, then brought those notes together during a daily or weekly review. Those records helped me see what the workflow did, where it failed, and what I still needed to understand.
My digital brain filled in context
The certification workspace was also connected to my digital brain through an MCP server. It could draw on my blog posts, journals, talks, skills, and hooks. That mattered because the things I needed to understand were spread across different projects and written at different times.
Some of those notes reminded me of decisions I had made months earlier. Others helped connect a skill I already understood to a requirement I was still learning. The workspace could use that history to help me bridge the gap between the work I knew and the new parts of the harness I was trying to explain.
The connection was valuable because I had not organized every lesson into one certification folder. The context was distributed across the way I work: notes about past experiments, the skills themselves, and explanations I had written for other people. Retrieval brought some of those threads back together at the moment I needed them.
My July AI maturity self-assessment, part one and part two had already taught me to compare my impression with evidence. Certification preparation expanded that evidence from a score to the trail of work behind it. I wrote about journals and activity logs in Where the Critical Thinking Happens; this is the other side of that practice, using those records to recover the history behind a system.
Evidence gave me a reason to understand the details
In my Stage 4 certification account, I covered the package and its artifacts. As I prepared it, I asked the AI workspace to explain how the runner, approvals, audit logging, and guardrails made the workflow work.
I also supplied a browsable evidence viewer alongside the README because I did not know how much of the verification would be automated. That was my reason for including it, not a claim that the examiner used it.
The evidence viewer put the certification artifacts in one place.
The workflow view made the steps, checks, and human gates easier to inspect.
The AI Maturity Model playlist and Stage 3 to 4: The Arrows on the Workflow Map help place that work in context. For me, the value of preparing evidence was not only that somebody else could assess the submission. It gave me a map back into the system I had built.
Stage 5 is the next layer
My notes then turned from Stage 4 design to what came next: which workflows to build, and how a coordinator should choose among them.
In September, I listed discovery, front-end and back-end implementation with TDD, and specialized test workflows. Discovery might produce canvases, prototypes, and prioritized stories. They do not all apply every time: some problems can move from clear stories to implementation, while others need a prototype to resolve disagreement or uncertainty.
By October, I had defined the Stage 5 coordinator’s job: choose relevant Stage 4 workflows and pass useful output from one to the next. My October 24 journal entry about simplicity reinforced that this should not become one enormous pipeline. A dozen steps might be a few smaller workflows, with some running only when needed.
I also started thinking about conditional paths: when should discovery produce a prototype, when can a feature move straight to implementation, and how should QA or UAT feedback return to product decisions? I sketched routing feature work to the development harness and issues to troubleshooting. Then I started building a coordinator around real work items, keeping human approvals while I hardened it. By October 24, I was improving the Stage 5 coordinator and its operator UI. This is the next part of the story, not a Stage 4 feature.
Making the decisions visible
Stage 4 already has a human review gate: I open the feature in the browser and decide whether it feels right. The visual support I began thinking about afterward belongs to the Stage 5 work. I want the operator UI to make the workflow’s current step, status, logs, approvals, and next action easier to see. A click-through guide with screenshots and scenarios could also help me remember the original problem and what to verify when a feature reaches human review.
I was already finding usability gaps. Step codes did not tell me what each part did, and approving a run in one place before navigating elsewhere to resume it felt awkward. Making those steps understandable is part of making the coordination usable, not a cosmetic task.
The idea of mounting a new component with fake data came from seeing another harness’s visual setup. A view of the agents and their roles, or persona-based reviewers, might help as well. Those ideas are starting to connect with the operator UI work already underway. The operator UI is being improved; I am not claiming that component previews or persona agents are finished.
Stage 4 taught me to connect a few steps into a workflow I could use. Stage 5 is asking me to coordinate workflows, decide what evidence should pass between them, and make those choices visible to the people reviewing the work. The logs helped me understand how I got here. The next part is building that coordinator and its operator experience.
Leave a Reply