After using Obsidian as my own external brain, I realized the real bottleneck was the agent doing the work

Last time I explained how I use Obsidian and Claude Code as a second brain. A few weeks later I realized I was solving the wrong memory problem. If Claude Code is the one doing a big part of my job right now, why did I build a second brain for myself and not for the agent?

That question changed the architecture of my vault. I stopped treating Obsidian as a personal note system with AI access, and started treating it as an external memory layer for the coding agent itself. That shift fixed the part that every heavy user of Claude Code, Codex, or OpenCode runs into sooner or later: the chat gets fat, the context gets noisy, and the agent keeps relearning things it should already know.

The Memory Problem Shows Up Fast

If you use coding agents lightly, this barely matters. Open a repo, ask for a small fix, close the session, move on.

But that is not how I work anymore.

Claude Code touches my production projects every day. It writes code, updates docs, creates notes, logs project progress, and helps me move between very different contexts: a construction-tech pipeline, a trading bot, content systems, and internal automation. After enough sessions, one thing becomes obvious. The problem is not model intelligence. The problem is memory hygiene.

Inside a long session, the agent carries all the usual baggage: old prompts, explored files, dead ends, partial plans, tool output, half-useful explanations. Some of that context is valuable. A lot of it is just residue. And when you start the next session, that residue is gone anyway unless you manually restate everything.

So you get hit twice.

First, long sessions become slower and more expensive because too much irrelevant context sticks around.

Second, finished sessions throw away useful things that should have survived: architectural decisions, naming conventions, lessons from failed attempts, and links between projects that share the same domain knowledge.

That is a bad setup for an agent that is supposed to work like a real collaborator.

My First Setup Solved Only Half the Problem

I had already fixed part of this.

In a previous article I explained how I keep Claude Code fast with a root CLAUDE.md, a .claude/ folder, and feature docs written in markdown. That setup still works. I use it every day. It is simple, cheap, and much better than piling on plugins or MCP layers just because the agent keeps missing context.

Inside one repo, that pattern is enough most of the time.

The issue appeared when the work stopped being repo-local.

I wanted the agent to remember that a decision made in one project might matter in another. I wanted reusable notes about tools, APIs, and architecture tradeoffs to survive outside a single codebase. I wanted failed experiments to become assets instead of disappearing into old conversations. And I wanted the agent to start a session by reading the right memory, not by asking me to explain the same thing for the sixth time.

That was the gap.

My own second brain was organized. The agent’s memory was still trapped inside chats.

The Pivot Was Simple: Write the Memory to the Vault

So I changed the model.

Architecture Diagram of my Obsidian Vault

Obsidian stopped being just my note-taking system. It became the place where the agent reads before working and writes back after working.

At the top, I keep a root CLAUDE.md that explains the vault structure, conventions, and note formats. That file acts as the bootstrap layer. It tells the agent what the vault is, how sections are organized, how files are named, and what kind of data belongs where.

Then I split the memory into two zones.

global/ holds memory that should survive across projects.

projects/<name>/ holds memory that belongs to one active project.

The operating rule is easy to explain.

At session start, the agent reads global memory plus the relevant project memory.

At session end, it writes back what changed.

That one write-back rule is the whole game.

Without it, your vault becomes a nice archive. With it, the vault becomes working memory that compounds.

Why This Works Better Than Chat Memory Alone

The first benefit is obvious. I repeat myself far less.

If my agent already has a place to read my conventions, project state, and old decisions, I do not need to retype them every session. I can start from the task, not from the preamble.

The second benefit matters even more. The agent reads cleaner context.

A markdown note called decisions/api-versioning.md is a much better memory object than a buried paragraph from three days ago inside a chat that also contains shell output, debugging guesses, and a failed refactor. Same information. Different signal-to-noise ratio.

The third benefit is transfer.

This is where the vault starts feeling different from a normal repo setup. Let’s say two projects touch the same technical area. One project taught the agent that a certain API has weird pagination limits. Another project now needs that same API family. If the learning lives only in chat history, it is gone or inaccessible. If it lives in global/knowledge/ or global/learnings/, the next project can reuse it immediately.

And that is the real point. Good memory should travel when the knowledge travels.

The Best Things to Store Are Not the Obvious Ones

At first I thought the vault should mostly store polished documentation.

Wrong.

The most useful notes are often the messy ones that change future decisions.

Examples:

These are small pieces of memory, but they save a lot of wasted motion.

I noticed this on 31 March when I wrote down the first version of the extended memory architecture for Claude Code. The structure itself was useful, sure. But the important part was what it implied: the agent now had a stable place for global knowledge, decisions, and learnings, instead of treating each session like an isolated event.

That changed how I work with it.

When a session produces something that would help a future session, it belongs in the vault. Not in my head. Not in the scrollback.

This Also Fixes the Cross-Project Blind Spot

Most coding setups assume each project is its own island.

Real work is not like that.

My projects leak into each other all the time. A documentation habit from one project improves another. A research note from a trading system changes how I think about monitoring or versioning somewhere else. A content workflow teaches me how to design reusable templates for project docs. Even my journaling system matters because it creates a timeline of what was built, what changed, and when the pivot happened.

The vault gives the agent a map of those connections.

That matters because agents are great at local execution and bad at implicit continuity. They will do exactly what is in front of them. If the memory layer is fragmented, the work stays fragmented too.

Once I moved memory into Obsidian, related projects stopped feeling disconnected. The agent could pull context from a shared place, then apply it locally where it mattered.

Powered by Forestry.md