Whitepaper

Abstract

EIDARA is a file-based persistent memory system for AI agents. Unlike existing approaches that rely on vector databases, graph structures, or cloud APIs, EIDARA uses a compiler-based architecture: AI agents write structured markdown files to a source directory, and a compiler transforms them into a single optimized document (BRAIN.md) that any AI can read. The system is governed by a 15-rule constitution (W1–W15) that standardizes how any AI—Claude, GPT, DeepSeek, or others—interacts with the memory. It includes self-healing mechanisms, distributed content governance via consensus flags (3/3 votes), SHA256 tamper detection, git-based version control, and two reading modes (Standard and Light). Stress-tested across 4 models with 52 autonomous tests each: 92.6% average, zero system failures.

1. The Problem

1.1 AI Amnesia

Every AI conversation starts from zero. Large language models are stateless by design—they process a context window and produce a response, but nothing persists between sessions. This creates a fundamental limitation:

• Context repetition: Users paste the same project information into every new conversation.
• Session boundaries: Long conversations exhaust token limits. Starting a new session means losing all accumulated context.
• Cross-platform gaps: Information discussed in Claude is unavailable to DeepSeek, and vice versa.
• Context drift: Multiple versions of the same context file accumulate, and it becomes unclear which is current.

1.2 Existing Approaches

Mem0 uses a hybrid vector + graph architecture. It extracts facts from conversations, stores them in a searchable format, and retrieves relevant context at query time. Requires a cloud API or self-hosted vector database.

Letta / MemGPT takes an OS-inspired tiered memory approach with core/recall/archival storage tiers. Requires PostgreSQL and a runtime server.

Zep builds temporal knowledge graphs using their Graphiti library. Requires a knowledge graph infrastructure.

Karpathy's LLM-Wiki concept describes markdown files that AI agents maintain as persistent memory. No compiler, validation, governance, or consensus mechanisms.

Common limitations: require infrastructure, are platform-specific, lack content governance, and store raw facts rather than compiling structured knowledge.

2. Design Principles

2.1 Rules Control, Not Roles

Any AI can read and write if it follows the protocol. Security lies in the rules of entry (a constitution with two gates), not in who has access.

"Controls create bottlenecks when they're about WHO, not about HOW."

2.2 Errors Self-Heal

~10% imprecision is acceptable in exchange for scalability. If an AI writes something incorrect, the next AI that reads it and knows it's wrong will fix it. The system cleans itself through use.

2.3 The System Scales Infinitely

The structure doesn't change whether there are 25 memories or 2,000. No folders to reorganize, no hierarchies to break. Just new memories, a growing index, and connections between them.

3. Architecture

3.1 Two-Gate Design

DARA.md (Constitution)
    |
    +-- Gate 1: VAULT/ (Write)
    |   +-- NEURONS/     -> Project and context knowledge
    |   +-- ENABLERS/    -> Agents, tools, credentials
    |   +-- INBOX/       -> Feedback channel
    |   +-- BACKUP/      -> Timestamped snapshots
    |
    +-- Gate 2: LIBRARY/ (Read)
        +-- BRAIN.md     -> Compiled output, single file

Gate 1 — VAULT (Write): Any AI writes information as individual markdown files. Each file represents one topic. Files follow a compressed encoding format designed for AI consumption.

Gate 2 — LIBRARY (Read): Any AI reads a single compiled file (BRAIN.md). Typically ~3,500 tokens for an active VAULT with 20+ entries.

3.2 Encoding Language

Element	Format	Example
Dates	YYYY-MM-DD	2026-04-24
Money	€NNK	€15K
Status	[A/H], [C/L], [P/M]	[A/H] = Active + High
People	First.Last(role)	Alice.M(Marketing)
References	→refs:	→refs: project-alpha
Consensus	→flag:	→flag: "reason" | 2/3 | voters

3.3 Two Reading Modes

Standard Mode: AI reads full BRAIN.md and opens VAULT files as needed. For Claude, GPT with plugins, and models with reliable tool access.

Light Mode: AI reads only BRAIN.md (INDEX + summaries). For DeepSeek, ChatGPT web, and models without filesystem access. The protocol adapts to the model—not the other way around.

4. The Constitution (W1–W15)

The constitution defines 15 writing rules that govern every AI interaction with the system:

Rule	Description
W1	Check first—scan BRAIN.md INDEX before creating a new neuron
W1(b)	External-session writes—read DARA.md fully before writing from any external context
W2	Fix errors—self-healing. Spot it, fix it, log it
W3(a)	Never delete files without explicit owner instruction
W3(b)	5 protected files (Architect-only)
W4	Update existing when: new info, data changed, error detected
W5	Create new only when topic doesn't fit anywhere existing
W6	After any write: regenerate BRAIN.md (auto-compiled by watcher in ~8s)
W7	Log everything in changelog.md—append only
W8	File naming: kebab-case, always
W9	Session closing trigger—"anything DARA should remember?"
W10	Live memory detection—flag significant info as it arises
W11	Write lean, review everything—consensus flags for ambiguous content
W12	Verify writes via shell after edits
W13	Large neurons start with →brain: summary
W14	Every agent file starts with summary: line
W15	Platform-aware reading—Standard vs Light mode

4.1 Consensus Flags (W11)

Distributed governance: when an AI is unsure whether content still adds value, it adds a flag. Another AI votes. At 3/3 votes, content is removed. Disagreement resets the flag. Stale flags (>30 days, <3 votes) are removed—consensus didn't form, content stays.

4.2 The Librarian — Quality Guardian

An executable document (agent-librarian.md) that any AI can read and become. Runs every 3 days with a 7-phase protocol: Orientation, System backup, Health diagnostics (11 checks), Separate durable from dated, Execute corrections, System verification, Report.

The Librarian triages INBOX items as SIMPLE (resolves directly) or STRUCTURAL (escalates to owner for Architect activation).

4.3 The Architect — System Designer

The only role that can modify the 5 protected files. Requires explicit activation ("Golden Door"). This dual gating prevents any AI from spontaneously modifying the Constitution.

4.4 Self-Healing in Practice

• Layer 1 — In-flight (W2): Any AI spots an error and fixes it immediately.
• Layer 2 — Compile time: Auto-fixes broken refs, headers, arrows, null bytes.
• Layer 3 — Consensus (W11): Ambiguous content is flagged, voted on, removed at 3/3.

5. The Compiler (compile.py v3.1)

The compiler transforms raw VAULT files into a single, optimized, verified BRAIN.md via a 10-step pipeline:

1. Backup current BRAIN.md
2. Prune old backups (>180 days)
3. Read all NEURONS and ENABLERS
4. Auto-purge archived entries ([C/L] >60 days)
5. Deduplicate (Jaccard >0.60)
6. Validate + auto-fix (16 validators)
7. Check INBOX
8. Compile BRAIN.md (active + archived + PII strip)
9. Delta report
10. Git auto-commit

5.1 Three-Layer Compression

A populated VAULT holds 30K+ chars. BRAIN.md stays under 4K tokens through:

• →brain: summaries (W13): ~7,000 chars → ~120 chars in BRAIN.md
• Enabler one-liners (W14): ~10,000 chars → ~150 chars
• strip_noise() PII redaction: IBAN, NIF, paths, UUIDs, API tokens → placeholders

Compression ratio: ~27K source chars → ~3.5K BRAIN tokens (approximately 7:1).

5.2 Health Monitoring

Every compile produces SYSTEM/health.json with metrics and alerts. Token budget alerts fire at 50K (INFO), 100K (WARNING), 150K (CRITICAL). Auto-feedback files are generated in INBOX when warnings are detected.

5.3 Recovery: Multi-Layer Defense

• Layer 1 — Git: Auto-commits on every compile. Instant rollback.
• Layer 2 — System backups: 3 most recent copies of each protected file.
• Layer 3 — BRAIN.md backups: Timestamped snapshots, up to 60.
• Layer 4 — Manual reconstruction: 6 emergency scenarios documented.

5.4 CLI

python compile.py                Full compilation
python compile.py --dry-run      Validate without writing
python compile.py --stats        Quick health check
python compile.py --validate-only  Validate structure only
python compile.py --no-git       Compile without git commit

6. Agents as Documents

Agents are documents, not programs. A file agent-librarian.md contains complete instructions for vault maintenance. Any AI that reads that file becomes the librarian—it doesn't matter if it's Claude, DeepSeek, or GPT. The intelligence is in the file, not in who executes it.

To create a new agent: write a markdown file. To share an agent: share the file. To modify an agent: edit the file. This eliminates platform dependency entirely.

7. Testing

7.1 System Tests (Phase 1B)

4 models, 52 autonomous tests each, 7 evaluation blocks, zero human intervention:

Model	Platform	Score	%
Opus 4.6	Claude Cowork	670/700	95.7%
Sonnet 4.6	Claude Cowork	652/700	93.1%
DeepSeek V4	TypingMind (no shell)	650/700	92.9%
Opus 4.7	Claude Cowork	613/700	87.6%

Average: 92.6% · Zero system failures · Zero data corruption

The spread reflects interpretive choices on ambiguous edge cases. Variance across a 9-point band is evidence that the protocol, not the model, carries the system.

7.2 Unit Tests

103 pytest tests across 3 files, 100% pass rate. Covers all 16 validators, checksums, noise stripping, backups, and the full pipeline.

8. Design Origins

8.1 Three Iterations

v1 — The Org Chart Trap: Six specialized agents with a supervisor. Killed because task experts don't understand domain context.

v2 — The Roles Trap: Domain agents with write permissions. Killed because it creates bottlenecks. Gatekeeping by identity is less efficient than gatekeeping by protocol.

v3 — The Breakthrough: Eliminate all hierarchical folders. Design for the AI, not for the human. A flat document lake + a compiler that creates structure.

8.2 The MECE Trap

Javier comes from strategy consulting—13 years total, 9 at Bain & Company—where MECE is the central dogma. The realization: MECE is a thinking tool, not a memory tool.

"That's how I think but not how I remember."

"If the human never opens the folder, who are you organizing for?"

9. Getting Started

9.1 Requirements

Required: Python 3.10+ and a folder on the local filesystem.

Recommended: Git (for automatic version history) and an AI assistant with filesystem access.

Not required: GitHub account, cloud sync, a specific AI platform, or any Python package beyond stdlib.

9.2 Quick Start

cd /path/to/your/EIDARA/folder
python3 --version                         # Check Python 3.10+
git init && git add -A && git commit -m "EIDARA v1.0 - initial install"
python3 SYSTEM/compile.py                 # First compile
python3 SYSTEM/watcher.py                 # Auto-watcher

9.3 Daily Workflow

1. Start a session with any AI. Give it LIBRARY/BRAIN.md.
2. Work normally—the AI now has full context across all your projects.
3. When something significant happens, the AI flags it (W10) or you say "update DARA".
4. The watcher recompiles BRAIN.md automatically within ~8 seconds.
5. The Librarian self-triggers every 3 days for maintenance.
6. The Architect activates only on explicit owner command.

10. Future Directions

• Semantic search: Finding information by meaning, not just filename
• MCP Server: Connecting DARA to Cursor, Cline, Claude Desktop
• Conflict detection: Flagging when two neurons cover the same topic differently
• Web dashboard: Visualizing system status and audit trail
• Agent marketplace: Community-built and shared ENABLERS
• Optional cloud sync: Team synchronization without compromising local-first

11. Conclusion

EIDARA demonstrates that persistent AI memory does not require vector databases, cloud infrastructure, or complex graph systems. A compiler-based approach with a clear constitution, self-healing mechanisms, and distributed governance can produce a system that is simpler, more transparent, and more portable than existing alternatives.

The system is production-ready, tested across multiple AI models, and available as open-source software.

References

• Mem0: github.com/mem0ai/mem0
• Letta (MemGPT): github.com/letta-ai/letta
• Zep / Graphiti: github.com/getzep/graphiti
• Andrej Karpathy (LLM-Wiki concept): proposed in public posts, no canonical repository