I finished the second half of my AI maturity assessment, covering governance, team practices, and stage mapping. The raw score came back higher than I would have given myself, so I paid closer attention to what the evidence actually showed versus what I feel I have internalized.
Main Findings
1. AI governance is documented but not yet systematic
I proactively feed the model current ground truth through changelogs when I know the way to accomplish something has changed. I document these situations as I review pull requests, work with AI to surface different patterns, and create skills to handle the right approach.
A more interesting question is what happens automatically when an AI agent produces output that does not meet standards. The design-endpoint workflow includes agent-on-agent review, where a reviewer agent checks the designer agent’s output before escalating to a human. I also use a second LLM to critique plans generated by a first LLM.
These patterns exist, but they are not yet everywhere in my workflows.
2. Team standardization is deliberate
The team standardized on Windsurf, now Devin Desktop, over Claude for our primary AI tool. We have a process for elevating individual learnings into team-wide standards. When one team member had findings using the Kimi model for cost and performance, we generalized that into team guidance during a daily scrum.
AI comes up in every daily scrum. We share what we are trying, what models we are using, and what results we are finding. We also have a two-hour lessons-learned meeting each sprint where we share findings and discuss what to tackle next.
3. Governance tooling integration is manual
This area needs work. I have a few “reviewer” agents and hooks, but not enough.
4. The stage mapping score is too generous
The scoring is too generous. Some of the evidence represents things I have done once or twice, not patterns that have been internalized and standardized.
The dimension analysis was more nuanced. It rated me at mastery for prompting engineering and task agents, which I mostly agree with. I must do better at using PromptFoo for evaluation. Workflow orchestration came back as strong, which is fair. I have some agents with full orchestration, like the graphic novel generation from user stories, but I need to document and improve those to make them easier to use.
I strongly disagree with the output evaluation and governance ratings. Those came back as mastery, but that does not represent my current practice. The assessment caught on to me trying PromptFoo, but I am not confident in my abilities using that tool or the approach to evaluation. I need to do more work in these areas.
5. Team activities are revealing
We did a blind spots exercise during the last sprint. I wrote my perspective on one side of a whiteboard, a team member wrote theirs on the other side without looking at each other’s sticky notes, and then we discussed what we found. I saw some blind spots the team member had, clarified what those were, and pointed out where we already have AI assistance and what needs work. One team member said they had the most productive sprint day yet.
I also created a workflow called analyze-feature that team members use every sprint. They give it a screenshot of an existing feature, and it documents what the feature does, where things live on the front end and back end, the routes that enable it, and where the data comes from. Team members use this before starting development to learn about the system without coming to me, then include that information in their planning with AI.
What I’m Taking Away
The score matters less than the patterns it surfaced. I know where the gaps are between capability and consistent application, and I have a clearer sense of what to focus on next.
What’s Next
Three clear next steps emerged:
-
Solidify workflow automation. I have examples of full orchestration, but my primary coding workflow still has manual execution. The recommendation is to create a dedicated workflow agent that orchestrates the story-to-tests-to-code sequence for a common feature type.
-
Improve evaluation practices. I need to move from trying PromptFoo to actually using it systematically as a unit test for individual prompts, not integration tests for entire workflows.
-
Validate readiness for the next stage. I need to work through the stage 3 certification, then strengthen the evidence requirements before claiming that level.
Leave a Reply