What is CAG vs RAG?

CAG vs RAG

CAG (Cache-Augmented Generation) pre-loads the entire knowledge base into the model's cached context. RAG dynamically retrieves relevant chunks at query time. They're complementary approaches with different tradeoffs.

Core Difference

RAG

pythonCopy
def rag_answer(question: str) -> str:
    # Dynamic retrieval at query time
    chunks = vector_db.search(question, top_k=5)
    context = "\n".join(chunks)
    return llm.invoke(f"Context: {context}\n\nQ: {question}")

CAG with Anthropic Prompt Caching

pythonCopy
import anthropic

client = anthropic.Anthropic()

# Load entire knowledge base once
knowledge_base = open("docs/all_company_docs.txt").read()

def cag_answer(question: str) -> str:
    return client.messages.create(
        model="claude-opus-4-6",
        max_tokens=1024,
        system=[
            {
                "type": "text",
                "text": knowledge_base,
                "cache_control": {"type": "ephemeral"}  # Cache for 5 min
            }
        ],
        messages=[{"role": "user", "content": question}]
    ).content[0].text

Prompt caching means the knowledge base is only processed once — subsequent queries use the cached KV state at ~10% the normal token cost.

When to Choose Each

Cost Comparison (Approximate)

Hybrid Approach

Many production systems combine both:

• CAG for core knowledge always needed (company policies, product specs)
• RAG for large supplemental document stores (support tickets, blogs)

Key Insight

CAG trades context window space for retrieval accuracy. It works best when your knowledge base is small enough to fit in context AND you have enough queries to amortize the initial cache cost.