Why we replaced RAG with an LLM-native wiki

When we started building the multi-agent platform that runs cocharge’s commercial operation, the default answer to “how do agents know things” was retrieval. Embed the docs, chunk them, top-k at query time, stuff the context. It works in a demo. In production — across billing, host onboarding and support — it became the single largest source of subtle wrongness.

The failure mode nobody screenshots

RAG fails politely. It doesn’t crash; it returns a plausible chunk that’s almost the right policy, and the agent confidently acts on it. Debugging that means reconstructing which fragments were retrieved for a given turn — and the answer changes as the corpus drifts.

An agent that retrieves is only as good as its worst chunk boundary. An agent that reads a well-written page reasons like a colleague who actually read the docs.

What we did instead

Following Karpathy’s framing, we maintain a hand-curated, densely cross-linked wiki written for the model — short pages, explicit invariants, no redundancy — and load the relevant pages whole. Selection is cheap and legible; reasoning happens over coherent documents, not fragments.

async def resolve_context(task: Task) -> list[Page]:
    return wiki.select(
        task.domain,          # billing | onboarding | support
        max_pages=6,
        link_depth=1,         # follow one hop of [[links]]
    )

The win isn’t just accuracy. It’s that a wrong answer is now a fixable page, edited once and reviewed like code — not an opaque embedding-space mystery. Our operators improve the agents by writing.

When RAG still earns its place

This isn’t an argument against retrieval everywhere. For large, fast-moving, low-stakes corpora — search over thousands of support tickets, say — embeddings are still the right tool. The point is narrower: for the core operating knowledge an agent must get right, a curated, legible knowledge base beats a probabilistic one. Treat that knowledge as a first-class artifact, not a preprocessing step.