A few weeks after I built a workflow for my AI maturity self-assessment, I started another one. Same sources, different questions.
Every run reprocessed everything. Every run cost tokens. I wanted a single place where my sources lived once, got indexed once, and could be queried again with new questions. I also wanted incremental updates: new blog posts only, new journal entries only, not the whole archive every time.
So I started building what I am currently calling digital brain (I’ll find another name for it…).
From a workflow to a small system
This is my first attempt at building a RAG system. I worked with AI to clarify what I wanted, then we built it together. I am using a vector database on PostgreSQL with the pgvector extension, running in a Docker container because I already use Postgres for other projects. The embedding model is Nomic Embed Text.
For sources that need to stay private, I run the embeddings through LM Studio. For public sources, I can use a different provider. The idea is simple: match the provider to the sensitivity of the data.
The backend indexes sources. The frontend runs analyses based on different assessments. I started with the AI maturity self-assessment, then added a second one, and I will likely add more. Each assessment has a system prompt, additional resources (e.g., questionnaire), and an output format, and it lets me pick which sources to include. I can also filter by recency: last 30 days, last 100 days, and so on.
Sources, not just files
A source is a folder on my machine, usually inside my Obsidian vault, but it can be anywhere. Each source has a label, a slug, and an LLM provider for indexing.
Right now I have a source for blog posts, one for journals, one for sprint docs, and one for my Devin skills and workflows. I have not added my talk transcripts and presentations yet, but I will. Each source can also point to multiple folders, so my blog posts source could include both my personal blog and the Improving Thoughts blog.
For the analysis itself, I can choose a different provider. I am using Gemini with an API key for the assessment step, while the embeddings stay local. This keeps the sensitive indexing on my machine and the better reasoning on Gemini.
What the assessment noticed
This morning I ran the AI maturity assessment.
It noticed I recently migrated end-to-end tests from Cypress to Playwright, and it connected that to my AI maturity reasoning: Playwright is more AI-friendly. It found that I have been working on skills for AI assistance and end-to-end tests. It also noticed I improved my writing style skill, the one I use to draft blog posts, to fix transcription errors like “Cloud Code” into Claude Code and “Devon” into Devin.
I asked the assessment to find at least two pieces of evidence for each answer. It did. That is the part I like most. It gives me breadcrumbs, not just a score or a label. It reminds me of the work behind the answer, and that matters because I move fast and I forget.
Still rough, still useful
The system is early. I want to add dark mode because I am a dark-mode person and the app is too bright. I want to add more source types and a better view of what is happening during indexing. But every time I open it, I notice one more friction point and I fix it. That is the loop.
I am moving from a single workflow that answers one question to a small system that can answer many questions from the same ground truth. The digital brain is a way to take my own scattered output, listen to it, and learn from it.
I will write another post about what the assessment actually found. This one is about the system and why it exists: keep the source material once, and reuse it many times.
What I am learning
What I am learning is that the value is in having the sources organized, accessible, and reusable. Once they are in one place, the questions become cheap. And the answers become more useful because I can see the evidence for them.
Leave a Reply