Site icon NexGismo

Agent Memory Architecture: A Safe, Bounded File Store

Ashutosh Tiwari
Agent Memory Architecture: A Safe, Bounded File Store
TL;DR
  • Problem: An agent that writes its own notes can overwrite facts, lose half its index at load time, or write outside its folder.
  • Pattern: Keep a short index, one page per entity, validated paths, size caps and a scheduled lint pass.
  • When to use: Agents that carry knowledge between sessions and whose notes people must read, audit and correct.
  • Trade-off: Plain files are easy to audit but need search once memories run into the thousands.

Agent memory architecture is the set of layers that lets an agent keep knowledge between sessions: transcripts, notes, structured pages and a small index loaded at start. A sound design bounds the index, writes one page per entity, validates every path, and runs a maintenance pass for stale or broken notes.

— Ashutosh Tiwari, NexGismo

Agent memory is knowledge an AI agent gains from experience and retains between sessions, saved as context it can read later. Anthropic’s definition excludes weight changes, in-context learning within one conversation, and compaction. In engineering terms, it is a write path, a read path and a maintenance job.

Agent memory architecture became a front-page topic this week. Anthropic’s engineering team published The components of agent memory on November 6, 2026. The Hacker News thread had 159 points and 88 comments when we fetched it on November 8, 2026 (discussion).

The thread shows the real pain. One commenter reported deleting about 95% of the memories Claude Code had created. That is one anecdote, not a measurement. It still names the problem: a memory store needs limits and cleanup, or it decays.

This post turns those sources into an agent memory architecture you can build, with a Python 3.10 file store and tests we ran. For related reading, see semantic caching for LLMs and LLM structured outputs in Node.js.

What is agent memory architecture?

It is the layered design that decides what an agent saves, where it saves it, and what it reads back. Anthropic’s post describes layers from raw transcripts, through agent-written notes and structured pages such as a linked wiki, up to a small context injected into every new session. An offline “dreaming” pass moves information between layers and corrects mistakes.

Two facts matter for builders. First, the injected layer is small by design. Claude Code loads only the first 200 lines or 25KB of MEMORY.md at the start of each conversation. Second, memory is context, not enforcement. For behavior that must always happen, the same docs say to use a hook.

Why does a naive agent memory architecture fail?

It fails in four ways, each documented in a primary source.

  1. Over-logging. Anthropic reports that Sonnet 3.7 wrote 413 memory files across 225 folders in Pokémon Red and earned one badge. Fable 5 wrote nine files in two folders plus one top-level file and earned four badges.
  2. Over-fixed rules. When the team told models to “record only failures,” Opus 5 and Opus 4.7 lost much of the boost memory gave them.
  3. Shared pages. In a contract task, one shared page kept 41 of 57 facts (72%). One page per contract kept all 57 (100%).
  4. Unsafe paths. The memory tool is client-side, so your code runs every file operation the model asks for. A weak path check lets a prompt-injected model leave its folder.

The fourth failure is not hypothetical. GHSA-5474-4w2j-mq4c (CVE-2026-34451) covers @anthropic-ai/sdk versions from 0.79.0 up to, but not including, 0.81.0. The memory tool compared paths with a string prefix and no trailing separator. A sibling directory such as “memory-evil” next to “memory” passed the check. The advisory is rated moderate, with a CVSS v4 score of 6.3, and 0.81.0 fixes it.

The check below is the same bug class. It is our own illustration, not the SDK’s code. We ran it against a real sibling directory, and it answered True.

def naive_is_inside(root: Path, candidate: Path) -> bool:
    return str(candidate).startswith(str(root))        # the bug class behind CVE-2026-34451

How do you design agent memory architecture step by step?

Start from the read path, because the index is the part the model always sees. Our recommended order:

  1. Define what memory is for. Anthropic found that stating the goal beat fixed rules for stronger models.
  2. Keep a small index. One file lists pages, one line each. Details live in topic pages opened on demand.
  3. Use one page per entity. A customer, contract or service gets its own file, so writers stop overwriting each other.
  4. Validate every path. Check containment by path components after resolving symlinks.
  5. Cap sizes. The Claude docs advise capping file growth and the characters view returns.
  6. Run a maintenance pass. Lint for broken links and index overflow. Keep a human review step for model-proposed fixes.

The data flow in this agent memory architecture is short. The model calls a memory command. Your handler validates path and size, touches the filesystem, and returns text. A scheduled job later lints and consolidates the files.

Back-of-envelope: how much index fits?

The two load limits interact. Arithmetic: 25,000 bytes divided by 200 lines is 125 bytes per line. If the average index line is longer than 125 bytes (assumption: a title, link and summary can be), the byte cap cuts the index first.

How do you build a safe memory store in Python?

We wrote a standard-library module that implements the six memory-tool commands from the Claude memory tool docs: view, create, str_replace, insert, delete and rename. The docs say the tool is client-side, with Claude requesting file operations and your application running them. The excerpts below are verbatim from the file we ran on Python 3.10.12.

Path validation comes first. It rejects encoded characters, dot-dot segments, backslashes and null bytes. Then it resolves the path and checks containment with relative_to, which compares components, not characters.

    def _resolve(self, virtual: str) -> Path:
        if not isinstance(virtual, str) or "\x00" in virtual or "\\" in virtual:
            raise MemoryToolError(f"Error: invalid path {virtual!r}")
        if "%" in virtual:
            raise MemoryToolError("Error: encoded characters are not allowed in paths")
        if virtual != VIRTUAL_ROOT and not virtual.startswith(VIRTUAL_ROOT + "/"):
            raise MemoryToolError(f"Error: path must start with {VIRTUAL_ROOT}")
        if ".." in virtual.split("/"):
            raise MemoryToolError("Error: '..' segments are not allowed")
        rel = virtual[len(VIRTUAL_ROOT):].lstrip("/")
        real = (self.root / rel).resolve()          # follows symlinks
        try:
            real.relative_to(self.root)             # component-wise, not string prefix
        except ValueError:
            raise MemoryToolError("Error: path escapes the memory directory") from None
        return real

The read path comes next. This method returns exactly what a session start would load under the 200-line and 25KB rule, and says whether anything was cut.

    def injected_index(self) -> tuple[str, bool]:
        """Return (text that loads, truncated?) for MEMORY.md under the 200-line / 25 KB rule."""
        index = self.root / "MEMORY.md"
        if not index.exists():
            return "", False
        lines = index.read_text(encoding="utf-8").splitlines()
        kept, used = [], 0
        for line in lines[:INDEX_MAX_LINES]:
            cost = len(line.encode("utf-8")) + 1
            if used + cost > INDEX_MAX_BYTES:
                break
            kept.append(line)
            used += cost
        return "\n".join(kept), len(kept) < len(lines)

A separate lint() method flags dangling [[links]] and a truncated index. A model-driven “dreaming” job can then decide how to fix them.

What breaks at scale, and what are the trade-offs?

One HN commenter suggested keeping provenance for memory changes. An append-only change log beside the pages does that. Our module does not include one.

A hypothetical example

Consider a hypothetical support agent for an order service. With one shared customers.md, two parallel sessions can each read it, append a line and write it back. The second write erases the first. With customers/acme.md and customers/globex.md, both survive. This example is illustrative, not a real incident.

When should you not use file-based agent memory?

Skip it for strict isolation across thousands of tenants (use a database), millions of searchable items (use a search service), facts that must always apply (use a hook), or agents that run once. For repeat questions, a cache may serve you better; see semantic caching for LLMs and Bedrock prompt caching.

Agent memory architecture options compared
Option Strongest at Weakest at Pick it when
File store with index Human audit; errors can be found and deleted Search across thousands of pages One agent or project, notes people review
Wiki plus search Questions that span many documents Size growth; needs upkeep A large corpus read repeatedly
Database table per user Isolation, transactions, quotas Free-form notes the model writes Many tenants with strict boundaries
Static instruction file (CLAUDE.md style) Rules you write yourself Learning from experience Stable conventions under about 200 lines
Hook or configuration Behavior that must always run Flexible judgment Linting, formatting, policy checks

How do you test an agent memory architecture?

Test what fails silently in your agent memory architecture. We ran seven unit tests with python3 -m unittest. They cover the six commands, size caps, six escape attempts, a symlink escape, index truncation and a lost update. All seven passed. Raw output:

naive startswith says inside: True
rejected 6 escape attempts
symlink escape rejected
index lines loaded: 200 of 300; truncated: True
lint problems: 301 | first: dangling link [[page0]] in /memories/MEMORY.md | last: MEMORY.md truncated at load: 200 of 300 lines are visible
shared table: acme survived = False | globex survived = True
per-entity pages kept: 2 of 2
Ran 7 tests ... OK

Run the lint pass in CI, and send sibling-directory paths to every command.

Interview questions this topic prepares you for

These are questions to be ready for, not reports of what any company asked.

How would you design long-term memory for a coding agent?

Separate layers by cost. Keep raw transcripts searchable, let the agent write short notes, and promote stable facts into a small index loaded at session start. Cap the index, one page per entity, and open detail pages on demand. Add a scheduled pass that checks notes against transcripts and removes stale ones. Keep humans in the loop for promotions, because a wrong note repeats in every future session.

Follow-up they may ask: how would you measure whether memory helped?

How do you stop a model-controlled file tool from escaping its sandbox?

Treat model output as untrusted input. Require a fixed virtual prefix, reject dot-dot segments and encoded sequences, resolve symlinks, then check containment by path components, never by string prefix. Cap file sizes, and return errors that leak no real paths. Test a sibling directory such as memory-evil beside memory, because that exact case was behind CVE-2026-34451. Run the handler with the least filesystem permission it needs.

Follow-up they may ask: why is a trailing separator not a complete fix?

When would you pick a vector store over files for agent memory?

Pick a vector or search index when the memory count is too large to list in an index, or when queries are fuzzy and cross many entries. Keep files as the source of truth if people must audit them. In Anthropic’s Stainless test, wiki plus search beat either alone, so the answer is often both. Rebuild the index from the files, never the reverse.

Follow-up they may ask: how do you keep the index and the files in sync?

The NexGismo verdict: My recommendation: build your agent memory architecture from plain files, one page per entity, a short index and a lint job. Add search only when the index overflows, and add model-driven consolidation only when you can review its output. Most memory bugs come from missing limits and missing path checks, not from a missing database.

Frequently Asked Questions

What is agent memory in AI?

Agent memory is knowledge an AI agent gains from experience and keeps between sessions, saved as context it can read later. Anthropic’s definition leaves out weight changes, in-context learning inside one conversation, and compaction of older turns. In practice it is usually notes, pages and an index stored outside the model.

How is agent memory different from RAG?

Retrieval-augmented generation looks up facts in a corpus someone else wrote. Agent memory stores what the agent itself learned from earlier work, such as a fix that failed or a user preference. The two can share a search index, but memory needs write rules, pruning and correction, which a read-only document corpus does not.

What is the 200-line limit on MEMORY.md?

Claude Code’s auto memory loads only the first 200 lines or the first 25KB of MEMORY.md, whichever comes first, at the start of every conversation. Anything past that is not loaded at session start. Detailed notes belong in topic files, which Claude reads on demand with its normal file tools.

How do you secure an agent’s memory tool against path traversal?

Validate every path in every command. Require the /memories prefix, reject dot-dot segments and encoded sequences, resolve the path to its canonical form, and check containment component by component, not with a string prefix. Then test sibling directories and symlinks, because a prefix check without a trailing separator was the flaw in CVE-2026-34451.

Should an agent write every observation to memory?

No. Anthropic reports that a Sonnet 3.7 run wrote 413 memory files and earned one badge in Pokémon Red, while a Fable 5 run wrote about ten files and earned four. Save what screens, logs and transcripts cannot already tell the agent. Log less, keep a short index, and prune on a schedule.

Agent memory architecture is a small-systems problem with a security edge. Keep the index short and the pages separate. Validate every path the model sends. Lint the store on a schedule, and let people review what gets promoted. Build the plain version first and measure it. For the layers around the store, see Bedrock prompt caching and LLM structured outputs in Node.js.

Sources

Exit mobile version