Tools and workflows for note-taking, bookmarking, and retrieval
There’s a moment every engineer knows. You’re staring at a Slack message from a teammate asking how to configure a specific service — and you know you’ve solved this before. Six months ago. Maybe a year. You remember the feeling of victory when you finally cracked it at 11pm after three hours of Googling. But the actual answer? Gone.
That moment used to happen to me constantly. And for a long time I just accepted it as part of the job. Engineers forget things. Google exists. Stack Overflow exists. We move on.
It wasn’t until I started noticing how much time I spent re-discovering the same things — the same flags, the same workarounds, the same architectural decisions and why we made them — that I realized this wasn’t a memory problem. It was a system problem. I didn’t have one.
The Early Days: My Brain Was the Database
At my first job, I was that guy with a .txt file on the desktop called notes.txt. Then notes2.txt. Then eventually notes_FINAL.txt. You know the type.
Knowledge management looked like this:
notes_FINAL.txt
- nginx restart command (which one again?)
- that thing about postgres max_connections
- TODO: read about redis eviction policies (never happened)
- fix deploy script (what script???)
Deployment was entirely manual. I’d SSH into the server, pull the latest code, run whatever migration script I’d written last week, restart the service, pray. The knowledge of how to do that lived in my head, in scattered files, and in the muscle memory of my fingers.
When I was the only engineer, that worked fine. The moment I had to onboard someone — or worse, when I came back from a two-week break — I realized I’d built a system that only ran on one machine: me.
EARLY KNOWLEDGE ARCHITECTURE
My Brain
┌──────────────────────────┐
│ - deployment steps │
│ - env variable names │
│ - why we use postgres │
│ - that weird cron fix │
│ - API quirks │
└──────────┬───────────────┘
│
▼
[ LOST when I leave, sleep, or forget ]
The bus factor was 1. The bus factor was me.
The Bash Script Era: Automation Without Documentation
The next phase felt like progress. I started writing bash scripts for deployments, which meant the steps were at least written down somewhere. But they were written in the language of instructions, not understanding. The scripts told you what to do, not why.
#!/bin/bash
set -e
echo "Pulling latest..."
git pull origin main
echo "Running migrations..."
./migrate.sh
echo "Restarting service..."
sudo systemctl restart myapp
echo "Done."
Sure, the deployment was automated. But when something broke — and things always break — you still needed a human who remembered why certain choices were made. Why do we restart instead of reload? Why is the migration script separate? Why does this need sudo? No one wrote that down.
The scripts were a step up from pure tribal knowledge. But they were still missing the connective tissue: context.
The Turning Point: When I Stopped Rediscovering the Same Things
The wake-up call came when I was setting up a new service and spent two hours debugging a Go MongoDB connection issue — only to find a comment I’d written in a completely unrelated repository six months earlier:
// NOTE: Always set ServerSelectionTimeout explicitly.
// Default is 30s and will silently hang during connection issues.
// Use 5s for fast-fail in production.
clientOptions := options.Client().
ApplyURI(mongoURI).
SetServerSelectionTimeout(5 * time.Second)
Two hours of debugging. One comment. Six months apart.
That was the moment I started building what I now call my PKB — Personal Knowledge Base. Not a note-taking app. A system. One with structure, retrieval, and intentionality behind it.
What a Personal Knowledge Base Actually Is
A PKB isn’t Notion with a bunch of half-finished pages. It isn’t a folder of bookmarks you never open. It’s a second brain that’s actually queryable — one that grows more useful the longer you maintain it.
For engineers specifically, it needs to solve three things:
PKB REQUIREMENTS FOR ENGINEERS
┌─────────────────────────────────────────────────┐
│ │
│ CAPTURE ORGANIZE RETRIEVE │
│ │
│ Fast enough Structured Find it in │
│ that you enough that under 30 seconds │
│ actually future-you or it doesn't │
│ do it understands exist │
│ │
└─────────────────────────────────────────────────┘
Most engineers nail one. Almost none nail all three. I failed at capture for years — the friction was too high, so I’d tell myself “I’ll write this up later” and never did. Then I failed at retrieval — I had notes but couldn’t find them. The organize problem I largely solved by stopping trying to be clever about folder structures.
The Stack I Actually Use
I want to be honest here: I’ve tried everything. Notion, Obsidian, Roam, plain markdown files in a git repo, Bear, Apple Notes, Evernote in 2015 like a fool. The stack doesn’t matter as much as the habits. But here’s what stuck for me:
Capture: Obsidian with a daily note as my inbox. Every fleeting thought, link, snippet goes here first. No organizing in the moment — that’s the enemy of capture.
Structure: Three top-level folders. That’s it.
/knowledge-base
├── /til ← "Today I Learned" — atomic, dated entries
├── /projects ← one folder per active context
└── /reference ← evergreen notes, patterns, decisions
Bookmarking: Raindrop.io with mandatory tags before saving. If I can’t tag it, I can’t save it. This forces a tiny bit of thinking upfront and makes retrieval dramatically better.
Retrieval: Obsidian’s full-text search + a weekly 15-minute review of my /til entries to promote useful ones into /reference.
TIL: The Most Underrated Engineering Practice
The single most impactful habit I built was writing TIL (Today I Learned) notes. Not tutorials. Not long-form docs. Just short, atomic, searchable entries:
# TIL: Go context cancellation doesn't stop goroutines
Date: 2024-11-03
Tags: golang, concurrency, context
Passing a cancelled context to a goroutine doesn't automatically
stop it. The goroutine has to *check* ctx.Done() itself.
## Bad assumption
func doWork(ctx context.Context) {
// I assumed this would stop when ctx is cancelled
go heavyComputation()
}
## Correct pattern
func doWork(ctx context.Context) {
go func() {
select {
case <-ctx.Done():
return
default:
heavyComputation()
}
}()
}
## Why it matters
Had a bug where cancelling a request context didn't stop
downstream work. Services were doing full computation even
after the client disconnected.
Source: https://pkg.go.dev/context
That’s it. Dates, tags, the wrong assumption, the right pattern, why it matters. Five minutes to write. Priceless six months later when you hit the same wall.
Building a Knowledge Retrieval Layer in Go
This is where things got interesting for me. I started treating my own notes like a queryable data source. I wrote a small CLI tool that indexes my Obsidian vault and lets me search by tag or keyword from the terminal, without opening a GUI.
Here’s a simplified version of the core logic:
package main
import (
"bufio"
"fmt"
"os"
"path/filepath"
"strings"
)
type Note struct {
Path string
Title string
Tags []string
Content string
}
// IndexVault walks a directory and builds an in-memory index of notes.
func IndexVault(vaultPath string) ([]Note, error) {
var notes []Note
err := filepath.WalkDir(vaultPath, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() || !strings.HasSuffix(path, ".md") {
return nil
}
note, err := parseNote(path)
if err != nil {
return err
}
notes = append(notes, note)
return nil
})
return notes, err
}
// parseNote reads a markdown file and extracts title, tags, and content.
func parseNote(path string) (Note, error) {
f, err := os.Open(path)
if err != nil {
return Note{}, err
}
defer f.Close()
note := Note{Path: path}
scanner := bufio.NewScanner(f)
var lines []string
for scanner.Scan() {
line := scanner.Text()
lines = append(lines, line)
if strings.HasPrefix(line, "# ") && note.Title == "" {
note.Title = strings.TrimPrefix(line, "# ")
}
if strings.HasPrefix(line, "Tags: ") {
raw := strings.TrimPrefix(line, "Tags: ")
for _, tag := range strings.Split(raw, ",") {
note.Tags = append(note.Tags, strings.TrimSpace(tag))
}
}
}
note.Content = strings.Join(lines, "\n")
return note, scanner.Err()
}
// Search filters notes by keyword across title, tags, and content.
func Search(notes []Note, query string) []Note {
query = strings.ToLower(query)
var results []Note
for _, n := range notes {
if strings.Contains(strings.ToLower(n.Title), query) ||
strings.Contains(strings.ToLower(n.Content), query) ||
hasTag(n.Tags, query) {
results = append(results, n)
}
}
return results
}
func hasTag(tags []string, query string) bool {
for _, t := range tags {
if strings.ToLower(t) == query {
return true
}
}
return false
}
func main() {
if len(os.Args) < 3 {
fmt.Println("Usage: kb <vault-path> <query>")
os.Exit(1)
}
vaultPath := os.Args[1]
query := os.Args[2]
notes, err := IndexVault(vaultPath)
if err != nil {
fmt.Fprintf(os.Stderr, "error indexing vault: %v\n", err)
os.Exit(1)
}
results := Search(notes, query)
if len(results) == 0 {
fmt.Println("No notes found.")
return
}
for _, n := range results {
fmt.Printf("[%s] %s\n Tags: %s\n\n", filepath.Base(n.Path), n.Title, strings.Join(n.Tags, ", "))
}
}
Running it looks like this:
$ go run main.go ~/obsidian-vault "context cancellation"
[2024-11-03-context-cancellation.md] TIL: Go context cancellation doesn't stop goroutines
Tags: golang, concurrency, context
[2024-09-14-http-timeout.md] TIL: HTTP client needs context timeout, not just deadline
Tags: golang, http, context
Nothing fancy. No embedding, no vector search, no LLM. Just fast, dumb, full-text search over files I own. It’s enough for 90% of retrieval needs.
The Evolution in Practice
Looking back, the arc is embarrassingly clear:
KNOWLEDGE MANAGEMENT MATURITY
Phase 1 Phase 2 Phase 3 Phase 4
(Early) (Scripts) (Ad-hoc) (System)
Brain Brain + Files + Structured
Only Scripts Bookmarks PKB + CLI
│ │ │ │
▼ ▼ ▼ ▼
Lost in Automation Hard to Queryable,
transit without find, maintained,
context no tags evergreen
The same evolution happened with deployment knowledge specifically. Early on: SSH + hope. Then bash scripts with no docs. Then CI/CD pipelines — but the why behind pipeline choices still lived in someone’s head. Now every architectural decision that’s non-obvious gets a /reference note with the alternatives we considered and why we didn't pick them.
DEPLOYMENT KNOWLEDGE EVOLUTION
SSH manually Bash script CI/CD Pipeline
┌──────────┐ ┌──────────┐ ┌──────────┐
│ ssh prod │ ──▶ │ deploy │ ──▶ │ .github/ │
│ git pull │ │ .sh │ │ workflows│
│ restart │ │ │ │ │
└──────────┘ └──────────┘ └──────────┘
Knowledge in Steps written, Steps + context
muscle memory context missing in PKB + ADRs
The containers didn’t just standardize our environments. They forced us to write down what the environment was — the packages, the versions, the configurations — and that artifact became documentation by default.
What I Wish I’d Known Earlier
Three things I’d tell my earlier self:
1. Write for 3am future-you. The person who’ll read your notes is exhausted and under pressure. Be specific. Include the error message. Include the command. Include the version number that matters.
2. Capture first, organize later. Organizing in the moment kills the habit. Dump everything into a daily inbox. Process it later. Imperfect notes that exist beat perfect notes that don’t.
3. Your retrieval system is only as good as your tagging. I have hundreds of notes I can’t find because I was inconsistent about tags. Spend 30 seconds on taxonomy when you write something. It pays compound interest.
The Honest Reality
I still lose things. I still rediscover things I’d solved before. The PKB doesn’t fix the problem completely — nothing does. But it’s changed the ratio dramatically. The things I’ve documented, I find. The things I haven’t documented, I’ve at least built the habit of asking: should I write this down?
That question alone — should I write this down? — is the real shift. Not the tool. Not the folder structure. Not the CLI I built on a Saturday afternoon. It’s the instinct that says: this is knowledge worth preserving, and present-me is the best person to capture it because present-me actually understands it.
Future-you is counting on present-you to do the work.