// Publication · Agent memory
We spent months building a four-tier memory system for our coding agents. Then we instrumented it to find out what was still failing. The answer wasn't the retriever — every fact we'd "forgotten" was sitting at rank 1. Nobody had searched.
Listen — the 100-second version
The findings and the two numbers that matter, without the full read. Narrated in the same voice that answers our line — 336 296 9877, if you'd rather hear it argue back.
There is a lot of writing about agent memory, and almost all of it is about the same two things: how you store facts, and how you retrieve them. Vector databases, chunking strategies, embedding models, hybrid search, knowledge graphs.
We built all of that. It works. And when we finally measured the whole loop end to end, we found the storage and retrieval layers were not where we were losing.
Retrieval works. Invocation doesn't. All five missed facts were rank 1 in search — the sessions simply never searched.
— internal audit, 10 sessions, 11 verified claims
That is the finding this piece is about, because it is the half of the problem almost nobody is working on. A memory system can be perfect and still be useless if the agent doesn't open the door.
Context first, because the finding only means something if the system underneath it is real. Four tiers, each with a different cost and a different job.
// four tiers — always-loaded at the top, cold and complete at the bottom
A single short file loaded into every session. It does not hold knowledge; it holds pointers. Hand-curated pins at the top, the rest generated and heat-windowed so recent, actually-used memories surface and stale ones fall below the fold.
Markdown files with frontmatter. Four types — feedback, project,
reference, user — read on demand when the index points at them.
Cross-linked, so following one lands you in the neighbourhood.
A retrieval service over years of notes and documents. Two-pass, live-first: current material ranks ahead of archived material rather than competing with it.
Full session transcripts, including sub-agent work. Semantic memory paraphrases; this tier is what you reach for when the exact wording matters — what was actually decided, and what was actually said.
// the doors — one verb each, because a door nobody can name is a door nobody opens
Ninety-seven lines of always-loaded index, routing to seven hundred and forty files. That ratio is the architecture. The common failure mode — one enormous instructions file that grows until it is truncated, ignored, or both — is a context-budget problem pretending to be a memory problem. You do not fix it by writing less down. You fix it by making the always-loaded layer a routing table instead of a library.
We did not design for this and only noticed when we counted. The largest single category is not
project state or reference material. It's feedback — 307 entries recording things we
got wrong, corrections we were given, and the reasoning behind a decision.
Project state goes stale on its own; the repository and the commit history already know what was built. What no artifact records is why we stopped doing it the other way. That is the memory that keeps paying, and it's the kind almost nobody writes down.
We built an evaluation harness — a fixed query set, precision scoring, and a weekly run — so changes to the system could be judged rather than argued about. The scores were good. Retrieval health sat at 99.9.
So we ran a different test. Instead of asking "can the system find this fact?", we took ten real working sessions and asked: "was there a fact in memory that would have helped, and did the agent go and get it?"
Eleven claims verified. Five confirmed misses. And in every single one of those five, the fact was already in memory, and it came back first when we searched for it afterwards. The system had the answer the whole time. It was never asked.
Whether the agent searched was predicted by how the task was framed, not by the model, the prompt, or the quality of the index. Sessions that began with "catch up on X" searched first, almost every time. Sessions that began with "fix X" or "look at X" went straight to exploring — reading files, running commands, re-deriving from scratch things that were already written down.
That reframes the fix completely. If invocation is the bottleneck, then a better embedding model buys you nothing. The intervention has to happen at the prompt boundary — the moment the task arrives and the agent decides what kind of problem it's looking at.
So that's where we put it: a hook that inspects the incoming task, notices the fix-and-examine shape, and nudges the agent to search memory before it starts exploring. Not a retrieval change. A habit change.
The honest version of a finding is the one where it happens to you after you've written it down.
While preparing this piece, an agent working on our own site was asked whether we had published anything newer on the subject. It searched filenames, found nothing, and reported back: no, there isn't. That was wrong. The document existed, it was substantial, and it had already been sent to a real person to build from — but it was named for the client it was written for, not the topic it covered.
The nudge had fired. It was skipped. When the search finally ran, the document came back at rank one — the exact failure mode described above, reproduced live, a week after we'd measured it.
Two rules came out of that, and both are now memories of the feedback type:
That is the flywheel working as intended: every miss becomes an evaluation row, and every correction becomes a memory. The system gets better by being caught.
You do not need our stack. The portable part is the shape, and you can start it this week. Point your own coding agent at this section if you like — it's written to be executed.
One always-loaded file. Cap it deliberately. It holds one line per fact, each pointing at a separate file. The moment you're tempted to paste knowledge into it, that's the signal to write a new file and link it instead.
Frontmatter with a name, a one-line description used to judge relevance, and a type. Start
with the four that earned their place for us: feedback, project,
reference, user. Write the description for the future reader who
is deciding whether to open it.
A single command to search everything, and a single command to save a fact. If saving takes more than one line, nothing gets saved. If searching has three possible entry points, the agent picks none of them.
Every time you correct the agent, that correction is the memory. Record the rule, the reason, and how to apply it. This will become your largest and most valuable category. It did for us.
Build a small fixed query set and score retrieval, so changes are measurable. Then run the harder audit: take real sessions, find facts that would have helped, and check whether the agent went and got them. The gap between those two numbers is the whole game.
If your audit shows the same pattern ours did, add a nudge where the task arrives, not in the retriever. Watch for fix-and-examine framing specifically — that's where the door goes unused.
Skills and tool definitions are an invisible tier. Two of our five misses were facts that lived inside a skill file, which our search couldn't see. Whatever your agent reads that your search doesn't index is a blind spot with no error message.
Superseding is not deleting. A retired fact needs to point at what replaced it, or you will re-derive the old answer and trust it.
Generated sections must be marked as generated. The moment part of a file is machine-written, a human will hand-edit it and lose the change.
Three things are queued, in this order.
A reranker in front of retrieval. Our own grading put us at parity with the commercial agent-memory products on architecture, ahead of most on the operations and evaluation layer, and behind on this one component. Retrieve broadly, rerank down to a handful. It's the cheapest remaining upgrade and we're publishing that we haven't done it yet.
Harder evaluation queries. A near-perfect score against your own question set means the question set is too easy. Ours needs cross-project questions, temporal ones — what changed about X since May — and negation traps. Right now the harness is a smoke detector, not a training signal.
Mining the verbatim tier for training data. Every session contains (context → memory action → outcome) triples. Those are labels. The long game is an agent that has learned when to reach for memory, instead of one that has to be nudged.
The invocation audit covered ten sessions, in one week, on one setup. It is directional, not conclusive, and we're saying so rather than rounding it into a law. The retrieval numbers are measured on our own hardware with our own corpus; treat them as evidence the approach works, not as a benchmark you can port. If you run this audit on your own agents, we'd genuinely like to hear what you find — particularly if invocation turns out not to be your bottleneck.
Prior art worth reading: the framing of memory management as a measurable, improvable skill comes from AutoMem, and the per-layer ablation and condition-tagged evaluation discipline from AgenticSTS. Both shaped what we shipped.
We build in the open because it's how we'd want to be sold to. If you're standing up something like this and want a second opinion — or you have a system that's broken, slow, expensive, or impossible — tell us. We'll say straight away whether it's something we can help with, and if it isn't, we'll say so.
Tell us what's broken →