I gave a talk last week about AI maturity self-assessment at Improving. I showed up with my own assessment results, gaps exposed, and talked about what I’m actually working through.
One colleague mentioned he’s been using AI for a while, and that some of the “new” tricks people are discovering online are things he figured out a year and a half ago. He used a phrase we heard at an internal class: cognitive surrender. That’s what we do when we let AI handle details we used to carry ourselves. He likes it because he doesn’t have to think about everything anymore. But he also acknowledged something important.
You only get to that point after banging your head against a brick wall for thirty or forty years.
There’s built-in knowledge that comes from that kind of experience. Someone fresh out of school doesn’t have it. They need some head-banging of their own. Not because we want anyone to suffer, but because certain things you only learn by going through them.
The Assessment Process
I’ve been running a monthly AI maturity assessment on myself. I want to see where I actually am, what gaps I have, and what to work on next.
The assessment looks at seven dimensions. Workflow orchestration. Output evaluation. Team practices. It gives me a structured view of my progress. I thought I had gaps in certain areas. Now I know for sure, with reasons why. That’s the difference between guessing and knowing.
I built a multi-agent workflow in Devin to run this assessment. It gathers information from my journal entries, sprint docs, and AI infrastructure, then analyzes it against the maturity framework. The whole thing runs in a terminal window while I watch it work.
Why Share the Mess?
Someone shared feedback after the talk. What resonated with them: the methodical tracking, the emphasis on doing it regularly, the shared experience of being in this “crazy spot” together. They mentioned finding peace within chaos and that the practice of reiteration needs to be repeated over and over until it becomes natural.
Learning in public normalizes the feeling of being disorganized. It shows that we’re all figuring this out together.
I can present a polished presentation on AI maturity best practices. That would be the pristine showroom. Everything clean, everything working, no visible problems. But that’s not where the learning happens.
The learning happens in the messy car shop. Tools scattered everywhere. Half-finished projects on the bench. Grease on your hands. You’re figuring things out, trying approaches that don’t work, iterating until something clicks.
The Value of Work-in-Progress
When I share my actual assessment results, gaps and all, it does a few things. It shows that even people who’ve been working with AI for a while have areas to improve. It demonstrates a structured approach to tracking progress. It creates space for others to share their own struggles without feeling like they’re behind.
Another colleague asked a question at the end that we’re all wrestling with. How do we create ubiquitous rules and constraints that follow us across projects and tools, not just within a single workspace? He doesn’t want his AI tools telling him every idea is the greatest idea ever heard. Neither do I.
We don’t have a perfect answer yet. That’s okay. We’re working on it together.
What I’m Learning
The assessment isn’t a one-time fix. It’s an ongoing practice. Every month I run it, see where I am, and adjust my focus. Some months I make progress. Some months I backslide. The point is the regularity, not perfection.
Structure helps. Having a framework to assess against makes the vague feeling of “I should be better at this” into concrete actions. Sharing the messy parts creates connection. Cognitive surrender isn’t about giving up; it’s about choosing what to hold onto and what to let go of.
The car shop will never be as clean as the showroom. The real work happens in the mess.
Leave a Reply