Improving has been publishing a lot of content about the AI Maturity stages. Internally, we have a handy self-evaluation questionnaire that’s helping me understand where I am in my journey and figure out what I should be working on next.
One line in the questionnaire resonated immediately when I first read it: select the response that most accurately describes what someone would observe if they watched you work for the past 30 days.
Not “what do you think you do.” Not “what are your intentions.” What would an observer actually have seen?
I had been using AI tools actively for months. I had opinions about where I stood. But when I sat with that question honestly, I couldn’t answer it with confidence. The past 30 days were a blur. I had learned things, shipped things, changed how I worked in ways I couldn’t fully articulate anymore.
That’s the uncomfortable truth behind any AI maturity self-assessment: if you haven’t been tracking, you’re guessing.
The Speed Problem
A week after finishing a sprint, I struggle to name the features we built. Two weeks after learning a new prompting technique or setting up a new workflow, I’ve often forgotten I learned it. The names of things keep changing. Tools that didn’t exist six months ago are now central to how I work.
That has a real consequence: we cannot reliably self-assess from memory.
When I ask myself, “Do I systematically refine AI outputs based on feedback patterns?” I genuinely want to say yes. I think I do. But thinking I do and having evidence that I do are different things. One is a vibe. The other is a foundation.
The Documentation I Already Had
When I realized I couldn’t answer that questionnaire from memory, I asked myself what I actually had. The inventory was longer than I expected.
I’ve been journaling for years. I’ve been blogging (daily this year) for a long time. For the past eight to ten months, I’ve been recording every team meeting: daily scrums, sprint reviews, sprint retrospectives, pairing sessions. Not because I planned to use them for a self-assessment but because we have been finding great value in having those transcripts.
That documentation practice started well before AI became central to my work. It turned out to be exactly what I needed.
Giving the Evidence to AI
So I gathered everything: journals, blog posts, meeting transcripts, the skills and workflows I had built in Devin. I pointed AI at all of it, handed it the questionnaire, and asked it to assess me based on what it found.
The results were more specific than I expected. The AI cited blog posts I had written but no longer remembered writing. It referenced journal entries where I had described working through a particular problem. It noted patterns I hadn’t consciously recognized: the number of skills I had created, iterations I had gone through on agent prompts, moments in meetings where I had explained to teammates why I was approaching something a certain way.
For some answers, I read the AI’s reasoning and thought: I didn’t know I did that. I had done it. I just moved on without cataloguing it.
When technology moves this fast, that’s what happens. We work through things and keep going. Without a place to capture what happened, the experience evaporates. The questionnaire asks what an observer would have seen. If nothing was written down, even the observer is guessing.
Guessing Doesn’t Help You Grow
A few people I’ve spoken with have avoided filling out the questionnaire entirely. Not out of laziness. Out of self-protection. What if the score looks bad? What if it gets shared with a manager?
Avoiding it doesn’t make the gaps disappear. It just means flying blind together, each of us quietly uncertain while appearing confident.
These assessments aren’t designed to rank people. They’re tools to make gaps legible so we can do something about them. When my AI analysis flagged automated output evaluation and quality judgment as one of my weakest areas, that wasn’t discouraging. It was clarifying. I knew what to work on next. Before that, I had a vague sense something was missing. After that, I had a direction.
That’s only possible if there’s documentation to work from. Without it, the assessment produces a score that reflects how well you remember yourself, not how you actually work.
The Practice Is the Point
The answer isn’t to start journaling right before you want to run an assessment. That’s building a resume, not a practice. The documentation is useful precisely because it accumulates over time, capturing things you’ve already forgotten thinking about.
It doesn’t have to be elaborate. Write about what you’re learning. Record meetings you can revisit. Note when something worked or failed, and why. Not as extra homework. As a habit that makes your own growth visible to you.
When I ran through the questionnaire with actual evidence, I trusted the results more. I could argue with them, refine them, point to specifics. That’s a conversation you can’t have with a score you arrived at by guessing.
What I’m learning is that tracking isn’t just for accountability. It’s for honesty. And you can’t be honest about where you are if you have no record of how you got here.
Leave a Reply