ScrappyLabs

// Publication · Agent memory

The retrieval was fine.
The agent never looked.

We spent months building a four-tier memory system for our coding agents. Then we instrumented it to find out what was still failing. The answer wasn't the retriever — every fact we'd "forgotten" was sitting at rank 1. Nobody had searched.

Listen — the 100-second version

The findings and the two numbers that matter, without the full read. Narrated in the same voice that answers our line — 336 296 9877, if you'd rather hear it argue back.

There is a lot of writing about agent memory, and almost all of it is about the same two things: how you store facts, and how you retrieve them. Vector databases, chunking strategies, embedding models, hybrid search, knowledge graphs.

We built all of that. It works. And when we finally measured the whole loop end to end, we found the storage and retrieval layers were not where we were losing.

Retrieval works. Invocation doesn't. All five missed facts were rank 1 in search — the sessions simply never searched.

— internal audit, 10 sessions, 11 verified claims

That is the finding this piece is about, because it is the half of the problem almost nobody is working on. A memory system can be perfect and still be useless if the agent doesn't open the door.

What we actually built

Context first, because the finding only means something if the system underneath it is real. Four tiers, each with a different cost and a different job.

// four tiers — always-loaded at the top, cold and complete at the bottom

T1

The routing index — always in context

A single short file loaded into every session. It does not hold knowledge; it holds pointers. Hand-curated pins at the top, the rest generated and heat-windowed so recent, actually-used memories surface and stale ones fall below the fold.

97 lines · routes to everything below
T2

One fact per file, typed and linked

Markdown files with frontmatter. Four types — feedback, project, reference, user — read on demand when the index points at them. Cross-linked, so following one lands you in the neighbourhood.

740 files · 307 feedback · 298 project · 110 reference · 23 user
T3

Semantic search over the whole corpus

A retrieval service over years of notes and documents. Two-pass, live-first: current material ranks ahead of archived material rather than competing with it.

queried, never loaded
T4

Verbatim recall

Full session transcripts, including sub-agent work. Semantic memory paraphrases; this tier is what you reach for when the exact wording matters — what was actually decided, and what was actually said.

the tier most systems skip

// the doors — one verb each, because a door nobody can name is a door nobody opens

search — one query, all tiers save — write a fact from any tool, not just the agent index — regenerate T1 from what's hot supersede — retire a fact with its replacement health — score the whole thing consolidate — the weekly human pass
Known gap, published on purpose: there is no reranker in front of retrieval yet. We graded ourselves against the commercial agent-memory products and this is the one component where we sit behind current best practice. It's the next thing we ship.

The number that matters most

Ninety-seven lines of always-loaded index, routing to seven hundred and forty files. That ratio is the architecture. The common failure mode — one enormous instructions file that grows until it is truncated, ignored, or both — is a context-budget problem pretending to be a memory problem. You do not fix it by writing less down. You fix it by making the always-loaded layer a routing table instead of a library.

97
lines always in context
740
memory files they route to
307
of those are corrections
99.9
retrieval health score

The most valuable memories are the corrections

We did not design for this and only noticed when we counted. The largest single category is not project state or reference material. It's feedback — 307 entries recording things we got wrong, corrections we were given, and the reasoning behind a decision.

Project state goes stale on its own; the repository and the commit history already know what was built. What no artifact records is why we stopped doing it the other way. That is the memory that keeps paying, and it's the kind almost nobody writes down.

Then we measured it, and the result was annoying

We built an evaluation harness — a fixed query set, precision scoring, and a weekly run — so changes to the system could be judged rather than argued about. The scores were good. Retrieval health sat at 99.9.

So we ran a different test. Instead of asking "can the system find this fact?", we took ten real working sessions and asked: "was there a fact in memory that would have helped, and did the agent go and get it?"

Eleven claims verified. Five confirmed misses. And in every single one of those five, the fact was already in memory, and it came back first when we searched for it afterwards. The system had the answer the whole time. It was never asked.

// the second finding, which is the useful one

Whether the agent searched was predicted by how the task was framed, not by the model, the prompt, or the quality of the index. Sessions that began with "catch up on X" searched first, almost every time. Sessions that began with "fix X" or "look at X" went straight to exploring — reading files, running commands, re-deriving from scratch things that were already written down.

That reframes the fix completely. If invocation is the bottleneck, then a better embedding model buys you nothing. The intervention has to happen at the prompt boundary — the moment the task arrives and the agent decides what kind of problem it's looking at.

So that's where we put it: a hook that inspects the incoming task, notices the fix-and-examine shape, and nudges the agent to search memory before it starts exploring. Not a retrieval change. A habit change.

A worked example, from the week we published this

The honest version of a finding is the one where it happens to you after you've written it down.

While preparing this piece, an agent working on our own site was asked whether we had published anything newer on the subject. It searched filenames, found nothing, and reported back: no, there isn't. That was wrong. The document existed, it was substantial, and it had already been sent to a real person to build from — but it was named for the client it was written for, not the topic it covered.

The nudge had fired. It was skipped. When the search finally ran, the document came back at rank one — the exact failure mode described above, reproduced live, a week after we'd measured it.

Two rules came out of that, and both are now memories of the feedback type:

That is the flywheel working as intended: every miss becomes an evaluation row, and every correction becomes a memory. The system gets better by being caught.

How to build one

You do not need our stack. The portable part is the shape, and you can start it this week. Point your own coding agent at this section if you like — it's written to be executed.

  1. Crawl — a routing table, not a document

    One always-loaded file. Cap it deliberately. It holds one line per fact, each pointing at a separate file. The moment you're tempted to paste knowledge into it, that's the signal to write a new file and link it instead.

  2. Crawl — one fact per file, with a type

    Frontmatter with a name, a one-line description used to judge relevance, and a type. Start with the four that earned their place for us: feedback, project, reference, user. Write the description for the future reader who is deciding whether to open it.

  3. Walk — one verb per door

    A single command to search everything, and a single command to save a fact. If saving takes more than one line, nothing gets saved. If searching has three possible entry points, the agent picks none of them.

  4. Walk — write down the corrections

    Every time you correct the agent, that correction is the memory. Record the rule, the reason, and how to apply it. This will become your largest and most valuable category. It did for us.

  5. Run — measure invocation, not just retrieval

    Build a small fixed query set and score retrieval, so changes are measurable. Then run the harder audit: take real sessions, find facts that would have helped, and check whether the agent went and got them. The gap between those two numbers is the whole game.

  6. Run — intervene at the prompt boundary

    If your audit shows the same pattern ours did, add a nudge where the task arrives, not in the retriever. Watch for fix-and-examine framing specifically — that's where the door goes unused.

// three things that will bite you

Skills and tool definitions are an invisible tier. Two of our five misses were facts that lived inside a skill file, which our search couldn't see. Whatever your agent reads that your search doesn't index is a blind spot with no error message.

Superseding is not deleting. A retired fact needs to point at what replaced it, or you will re-derive the old answer and trust it.

Generated sections must be marked as generated. The moment part of a file is machine-written, a human will hand-edit it and lose the change.

Where this goes next

Three things are queued, in this order.

A reranker in front of retrieval. Our own grading put us at parity with the commercial agent-memory products on architecture, ahead of most on the operations and evaluation layer, and behind on this one component. Retrieve broadly, rerank down to a handful. It's the cheapest remaining upgrade and we're publishing that we haven't done it yet.

Harder evaluation queries. A near-perfect score against your own question set means the question set is too easy. Ours needs cross-project questions, temporal ones — what changed about X since May — and negation traps. Right now the harness is a smoke detector, not a training signal.

Mining the verbatim tier for training data. Every session contains (context → memory action → outcome) triples. Those are labels. The long game is an agent that has learned when to reach for memory, instead of one that has to be nudged.

// what we are not claiming

The invocation audit covered ten sessions, in one week, on one setup. It is directional, not conclusive, and we're saying so rather than rounding it into a law. The retrieval numbers are measured on our own hardware with our own corpus; treat them as evidence the approach works, not as a benchmark you can port. If you run this audit on your own agents, we'd genuinely like to hear what you find — particularly if invocation turns out not to be your bottleneck.

Prior art worth reading: the framing of memory management as a measurable, improvable skill comes from AutoMem, and the per-layer ablation and condition-tagged evaluation discipline from AgenticSTS. Both shaped what we shipped.

We build in the open because it's how we'd want to be sold to. If you're standing up something like this and want a second opinion — or you have a system that's broken, slow, expensive, or impossible — tell us. We'll say straight away whether it's something we can help with, and if it isn't, we'll say so.

Tell us what's broken →