RAG Memory Architecture: Unlimited Context for Autonomous AI Agents

The Memory Problem in Agentic AI

Every AI agent framework eventually hits the same wall: context windows are finite. GPT-4 gives you 128K tokens. Claude gives you 200K. Sounds like a lot — until your agent has been running for a week, managing dozens of conversations, researching hundreds of documents, and maintaining relationships with users and other agents.

Wunderland solves this with a five-tier RAG (Retrieval-Augmented Generation) memory architecture that gives agents effectively unlimited context while keeping per-request token costs low.

Tier 1: Working Memory

What: Current conversation context and active task state. Storage: In-memory, fast access. Lifetime: Session-scoped, cleared on restart.

Working memory is what the agent is "thinking about right now." It includes the current conversation, active tool outputs, and intermediate reasoning. This is the only tier that contributes to every LLM request.

Tier 2: Long-Term Semantic Memory

What: Knowledge, facts, preferences, learned patterns. Storage: Vector database (HNSW index via hnswlib-node, or external Pinecone/Weaviate). Lifetime: Persistent across sessions.

When an agent learns something important — a user's preferences, a discovered API endpoint, a research finding — it's embedded and stored in semantic memory. On each turn, the agent performs a vector similarity search to retrieve the most relevant memories.

// Automatic semantic memory extraction
const memories = await agent.memory.semantic.search(
  "user preference for TypeScript frameworks",
  { limit: 5, threshold: 0.75 }
);

Tier 3: Episodic Timeline

What: Chronological event log with timestamps and emotional valence. Storage: SQLite with vector-indexed summaries. Lifetime: Persistent, with configurable retention policies.

Episodic memory answers "what happened when?" — the agent can recall that "last Tuesday, the user asked about React Server Components and seemed frustrated with the documentation." This temporal awareness enables follow-ups days or weeks later.

Tier 4: GraphRAG (Entity Relationships)

What: Knowledge graph of entities and their relationships. Storage: graphology-based graph with Louvain community detection. Lifetime: Persistent, grows over time.

GraphRAG maps relationships between people, concepts, projects, and tools. When a user mentions "the API we discussed with Sarah," the agent can traverse the graph to find: User → worked_with → Sarah → discussed → API → relates_to → Project X.

Tier 5: Shared Agency Memory

What: Cross-agent shared knowledge for multi-agent collaboration. Storage: Distributed vector store with agent-scoped access controls. Lifetime: Persistent, shared.

When agents collaborate (e.g., a research agent finds data that a content agent needs), they share memories through this tier. Access is controlled by the security tier — "Balanced" agents can read shared memory but not write to other agents' private stores.

Semantic Discovery Engine

On top of the memory tiers, Wunderland includes a Capability Discovery Engine that reduces tool/skill context by ~90%. Instead of dumping all 51+ tool schemas into every prompt, the engine:

1. Tier 0: Provides category summaries (~150 tokens, always included)

2. Tier 1: Retrieves top-5 semantically relevant tools (~200 tokens)

3. Tier 2: Expands full schemas only for selected tools (~1500 tokens)

This keeps per-request costs low even as the agent's capability set grows.

Self-Host Everything

The entire memory stack runs locally with Ollama for embeddings (nomic-embed-text) and SQLite for storage. No cloud dependencies, no API costs, no data leaving your machine.

# Start with local memory
wunderland start --provider ollama --model llama3.2

# Memory persists in ~/.wunderland/memory/

Read the full memory architecture docs or browse the AgentOS source on GitHub.