Context Engineering vs RAG: When to Use Which

In 2024, RAG was the answer to any question about LLMs working with external data. By 2026, context windows had grown to 1–2 million tokens, prompt caching had cut the cost of repeat requests by 90%, and the term “context engineering” had crystallized into its own discipline. RAG hasn’t gone anywhere — but it’s stopped being the only solution.

This article covers: what context engineering is, how it differs from RAG, when long context is enough, when you need RAG, and how to combine both approaches. With cost calculations, code examples, and a decision framework. For the full picture of the discipline — six-layer context structure, persistence patterns, subagent isolation — see the context engineering guide.

What Is Context Engineering

Andrej Karpathy put it this way in June 2025:

Context engineering is the delicate art and science of filling the context window with just the right information for the next step.

The key phrase is “just the right.” Not maximum information. Not minimum. Exactly what’s needed for the next step.

Not Prompt Engineering

Prompt engineering is about how you phrase a question — the focus of a systematic approach to writing prompts. Context engineering is about what you feed the model before the question. The difference is fundamental:

Aspect	Prompt Engineering	Context Engineering
Focus	Wording the request	Assembling the input
Scope	A single prompt	The entire pipeline
Components	Role, format, chain-of-thought	System prompt + docs + tools + memory + examples
Metaphor	How to ask an expert a question	What dossier to hand the expert before asking

A perfect prompt with bad context gives a bad result. A mediocre prompt with precise context gives a good one.

Components of Context Engineering

The context of an LLM request is assembled from six sources:

1. System prompt — role, constraints, behavioral style
2. Project context — stack, conventions, architectural decisions (CLAUDE.md, .cursorrules)
3. Retrieved documents — RAG results, code search, grep
4. Tool results — responses from APIs, databases, MCP servers
5. Memory — persistent notes across sessions
6. Few-shot examples — samples of expected behavior

Context engineering orchestrates all six. RAG covers only item three.

Why It Became a Discipline

Three factors converged in 2025–2026.

Growing context windows. Claude Opus 4.8 and Sonnet 4.6 — 1M tokens (GA since March 2026). GPT-5.5 — 1M (922K input). Gemini 3.1 Pro — 1M standard, 2M in extended modes via API. When the window can hold an entire codebase, the question changes: not “how do I retrieve what’s relevant” but “what exactly should I put in.”

Prompt caching. Anthropic, Google, OpenAI — all shipped caching for static context. Cache reads on Claude cost $0.50/MTok instead of $5.00/MTok. Long context became 10x cheaper.

AI agents. Agents need memory across steps, tool results, a task plan, project context. A single prompt doesn’t capture that — you need a context-assembly pipeline.

Gartner declared in July 2025: “Context engineering is in, and prompt engineering is out.” Tobi Lütke (Shopify CEO) popularized the term. The discipline got a name.

What Is RAG

RAG (Retrieval-Augmented Generation) is a pattern in which an LLM receives relevant documents before generating a response. Three steps:

1. Retrieve — find relevant fragments in the corpus
2. Augment — add what was found to the prompt
3. Generate — produce a response informed by that context

How RAG Works Under the Hood

Documents → Chunking → Embedding → Vector Store
                                        ↓
Query → Embedding → Vector Search → Top-K chunks → LLM → Answer

Chunking — splitting documents into fragments. Chunk size determines search granularity: too small and you lose context, too large and you get noise.

Embedding — converting text into a numeric vector. Semantically similar texts get vectors that are close together.

Vector Store — a database for storing and searching vectors. pgvector, Pinecone, Qdrant.

Retrieval — finding the vectors closest to the user’s query. The top-K results go into the prompt.

Types of RAG

Naive RAG. The simplest implementation: fixed chunk size, a single embedding model, cosine similarity, top-5 into the prompt. Good for prototypes.

Advanced RAG. Hybrid search (vector + BM25), reranking, parent-child chunking, query expansion. Precision@5 climbs from 0.6 to 0.85+.

Agentic RAG. An agent decides what to search, in which index, and how many times. It can reformulate the query, ask for clarification, and combine results from multiple sources. Context engineering with RAG as one of its tools.

GraphRAG. Microsoft’s approach (2024), which builds a graph of entities and relationships on top of the corpus. Instead of searching flat chunks, the model traverses the graph and finds answers to “global” questions — about relationships between documents, trends, patterns. By 2026, GraphRAG and its lighter-weight cousins (LightRAG, HippoRAG 2) became a go-to solution for enterprise corpora with dense document interconnections — anywhere you need answers to questions like “how are A and B connected” or “what happened across the entire corpus.” The 2026 practice: vector + graph in a hybrid, routing by query type.

Adaptive RAG. A query classifier routes each question to the appropriate retrieval strategy based on complexity: a simple factual question gets direct search, a complex analytical one gets a multi-hop iterative agent. The 2025–2026 practice: RL-trained agents (inspired by approaches like OpenAI Deep Research and Search-R1) replace the fixed “retrieve → generate” pipeline with an autonomous loop of “plan → search → reason → critique → search again.”

When RAG Became the Standard

RAG appeared in a Facebook AI Research paper in 2020. It became the standard by 2023–2024, when context windows were 4K–32K tokens. Under those constraints, RAG was the only way to work with external data.

Context Engineering vs RAG: Key Differences

The Main Point: RAG Is Part of Context Engineering

RAG isn’t an alternative to context engineering. RAG is one tool in its arsenal. Comparing them is like comparing “cooking a meal” to “chopping vegetables.” Chopping is part of the process.

Analogy: RAG is the librarian who finds the right books. Context engineering is the editor who decides which books to request, which chapters to select, in what order to read them, and what to include in the final document.

Comparison Table

Criterion	Context Engineering	RAG
Scope	The entire context-preparation pipeline	Only document retrieval
Components	System prompt, memory, tools, RAG, examples, structured output	Chunking, embedding, vector store, retrieval, reranking
Data	Anything: code, API responses, files, memory	Documents in a vector store
Infrastructure	Prompt caching, MCP, memory files	Vector DB, embedding pipeline, indexing
Latency	Depends on context size	Depends on retrieval + reranking
Scaling cost	Linear with context size	Linear with corpus size (indexing)
When to update	When the task/project changes	When data in the corpus changes
Attribution	Not built in	Built in (source = chunk)
Abstraction level	Orchestration layer	Retrieval component

Four Strategies, One of Which Is RAG

Context engineering works with four context-management strategies:

Strategy	What it does	Examples
Write	Persists information externally	Memory files, scratchpads, CLAUDE.md
Select	Retrieves what’s relevant	RAG, grep, code search, MCP tools
Compress	Shrinks context	Summarization, compaction, tool-result cleanup
Isolate	Isolates tasks	Subagents with a clean context

RAG is an implementation of the Select strategy. One of four. Powerful, but not the only one.

When Long Context Is Enough Without RAG

Rule: If It Fits, Don’t Overcomplicate

The context window on Claude Opus 4.8 and Sonnet 4.6 is 1M tokens. Gemini 3.1 Pro is 1M. One million tokens is roughly 750,000 words, or 3,000 pages of text.

If the entire corpus fits inside the context window, a RAG pipeline adds complexity without benefit.

Scenario 1: A Codebase Under 100K Lines

The average startup has 50–80K lines of code. In tokens, that’s roughly 200–300K. Claude Code loads the files it needs into context via grep and glob, no vector store required. This works because:

• Code is structured (files, functions, modules)
• Search by name is more precise than semantic search for code
• Context refreshes with every request

Scenario 2: A Fixed Set of Documents

Internal company documentation — 50 documents, 200K tokens. Updated once a month. Two options:

Option A: A RAG pipeline

• Chunking + embedding + vector store + retrieval
• Infrastructure: Qdrant/Pinecone + embedding API
• Maintenance: re-indexing on updates
• Cost: $50–200/month (vector DB + embedding API)

Option B: Long context + prompt caching

• All 200K tokens in the system prompt
• Prompt caching: first request $1.25/MTok (5-min cache write), subsequent ones $0.50/MTok
• Infrastructure: zero
• 200K tokens × $0.50/MTok = $0.10 per request (cache read)

At 100 requests a day: option B costs $10/day, $300/month; option A costs $50–200/month. At that load, RAG is cheaper in raw dollars, but it requires infrastructure and upkeep. Long context makes sense at low load — up to ~50 requests/day ($150/month) — or when you factor the RAG infrastructure into total cost of ownership.

Scenario 3: Analyzing a Single Long Document

A 100-page legal contract (~80K tokens). RAG will chop it into chunks and lose the cross-references between sections. Long context preserves the whole structure — the model sees the entire contract and can spot contradictions between sections, something chunk-based search can’t do.

Long Context Cost Calculation

Claude Opus 4.8, 500K tokens in context, 100 requests a day:

Mode	Cost per request	Per day	Per month
No cache	$2.50	$250	$7,500
Prompt caching (5-min, cache read)	$0.25	$25	$750
Prompt caching (1-hour, cache read)	$0.25	$25	$750

Cache write (first request): $3.13 (5-min) or $5.00 (1-hour). But every subsequent read is $0.25. It pays off starting with the second request.

Claude Sonnet 4.6, same conditions:

Mode	Cost per request	Per day	Per month
No cache	$1.50	$150	$4,500
Prompt caching (cache read)	$0.15	$15	$450

Gemini 3.1 Pro, 500K tokens (>200K triggers long-context pricing):

Mode	Cost per request	Per day	Per month
Standard	$1.25	$125	$3,750
With context caching	~$0.16	$16	$480

When Long Context Doesn’t Work

• Document exceeds 1M tokens — doesn’t fit in a standard window. Gemini 3.1 Pro supports 2M in extended mode, but that’s the exception
• Latency-critical workloads — processing 500K tokens takes 5–15 seconds. If you need a response within 500ms, this isn’t an option
• Attention degradation — a Chroma study (2025) found that all 18 frontier models tested (Claude 4, GPT-4.1, Gemini 2.5) show a monotonic drop in F1 as context grows. The sharpest degradation happens in the 100K–500K token range
• Frequently updated data — the prompt cache is invalidated by any content change

When You Need RAG

Five Signals That RAG Is Necessary

1. The corpus exceeds the context window. An enterprise knowledge base, a legal precedent database, scientific literature — when the total data volume in tokens doesn’t fit within 1–2M tokens (even with Gemini 3.1 Pro in extended mode). Document count is secondary: 10,000 short articles might total under 50M tokens, while 500 long regulatory documents can easily run to 200M. RAG extracts the top-10 relevant fragments from a corpus of any size.

2. Data updates frequently. Jira tickets, Slack messages, wiki articles — data changes daily. A RAG index updates incrementally. A prompt cache is fully invalidated by any change.

3. Attribution is required. “Where did this answer come from?” is a critical question for compliance, legal, and medical use cases. RAG returns specific chunks with metadata (document, page, date). Long context is a black box: you get an answer, but the source is unclear.

4. Latency under 1 second. A chatbot on a website. The user expects a response within 500ms. RAG: vector search 20ms + reranking 50ms + LLM with 5K tokens of context 300ms = 370ms. Long context: an LLM call with 500K tokens takes 5–15 seconds.

5. Cost optimization at high load. 10,000 requests a day. RAG feeds in an average of 5K tokens → $0.025 per request (Sonnet). Long context feeds in 500K → $0.15 per request (cache read). Difference: $250/day vs $1,500/day.

Calculation: RAG vs Long Context by Cost

10,000 requests/day, Claude Sonnet 4.6 ($3/$15 per MTok):

Approach	Tokens in context	Cost/request	Per day	Per month
RAG (5K retrieved)	8K (system + retrieved)	$0.024	$240	$7,200
Long context (no cache)	500K	$1.50	$15,000	$450,000
Long context (cache read)	500K	$0.15	$1,500	$45,000
RAG + embedding cost	8K + embedding	$0.03	$300	$9,000

At 10K requests/day, RAG is 5x cheaper than even cached long context. Plus RAG cost scales linearly with request volume, not with corpus size.

Context Engineering Without RAG: Practical Techniques

Prompt Caching

The most underrated technique of 2025–2026. The static part of context (system prompt, documentation, examples) gets cached on the provider’s side. Repeat requests read from cache.

Cost on Claude:

Operation	Opus 4.8	Sonnet 4.6	Haiku 4.5
Base input	$5.00/MTok	$3.00/MTok	$1.00/MTok
Cache write (5-min)	$6.25/MTok	$3.75/MTok	$1.25/MTok
Cache write (1-hour)	$10.00/MTok	$6.00/MTok	$2.00/MTok
Cache read	$0.50/MTok	$0.30/MTok	$0.10/MTok

Cache read is 10% of the base cost. 90% savings on repeat requests.

Example: Anthropic API with prompt caching (Python)

import anthropic

client = anthropic.Anthropic()

# Static context — gets cached
system_content = [
    {
        "type": "text",
        "text": "You are an AI assistant for a law firm.",
    },
    {
        "type": "text",
        "text": full_legal_corpus,  # 200K tokens of documentation
        "cache_control": {"type": "ephemeral"}  # Cache this block
    }
]

# First request: cache write ($6.25/MTok for Opus)
# All subsequent requests within 5 minutes: cache read ($0.50/MTok)
response = client.messages.create(
    model="claude-opus-4-8-20260528",
    max_tokens=4096,
    system=system_content,
    messages=[{"role": "user", "content": "What are the risks in section 4.2?"}]
)

CLAUDE.md and System Instructions

Project context that an agent reads automatically. Claude Code loads CLAUDE.md on every run — stack, conventions, architectural decisions. Cursor uses .cursorrules. Windsurf uses .windsurfrules.

Context engineering without RAG: static project context fed directly into the prompt.

# CLAUDE.md
## Stack
- Python 3.12, FastAPI, SQLAlchemy 2.0, PostgreSQL
- Alembic for migrations, pytest for tests
## Conventions
- Type hints required
- Pydantic v2 for validation
- Async handlers by default
## Architecture
- Clean Architecture: domain → application → infrastructure
- Repository pattern for data access

Tool Use (MCP)

Instead of pre-loading all the data into context, pull it on demand. Model Context Protocol (MCP) standardizes access to external tools — see the guide to building production MCP servers for the implementation details.

// The MCP server hands over data on the agent's request
// The agent decides for itself when and what to request

// Instead of: load all 500 tickets into context
// The agent calls: search_tickets("auth bug priority:high")
// And gets: 3 relevant tickets, 2K tokens

The agent loads only what’s needed for the current step. Context stays clean.

Memory: Persistence Across Sessions

An agent stores knowledge for future sessions. Claude Code writes patterns to MEMORY.md. Cursor keeps context in .cursor/rules/.

# MEMORY.md (updated automatically by the agent)
## Learned Rules
- This project uses camelCase for API endpoints
- Migrations are named: YYYY-MM-DD_description.sql
- Tests are grouped by module, not by type

This is the Write strategy of the four context-engineering strategies. The agent writes notes for its future self.

Structured Output

A JSON Schema in the response guarantees a predictable format. Context engineering at the output stage: instead of free-form text, structured data.

from pydantic import BaseModel

class CodeReview(BaseModel):
    file: str
    severity: Literal["critical", "warning", "info"]
    issue: str
    suggestion: str
    line_range: tuple[int, int]

# The LLM returns exactly this structure
# No parsing, no regexes

Few-Shot Examples

Instead of fine-tuning, use 3–5 examples in context. Cheaper, faster, easier to update.

## Code review examples (follow this format):

File: auth.py:45-52
Severity: critical
Issue: SQL injection via f-string
Suggestion: Use a parameterized query
---
File: utils.py:12
Severity: info
Issue: Unused import
Suggestion: Remove import os

Few-shot examples take up 500–2000 tokens. With prompt caching, they cost $0.00025–$0.001 per request. Fine-tuning costs $25+ per training run and limits flexibility.

RAG: The Modern 2026 Stack

Chunking Strategies

Fixed-size chunking. 512–1024 tokens with 10–20% overlap. Simple, predictable, but it cuts through semantically connected blocks.

Semantic chunking. Splits along meaning boundaries — headings, paragraphs, topic shifts. Embed each sentence → if similarity between neighboring sentences drops below a threshold → chunk boundary. Higher quality, but more expensive to process.

Parent-child chunking. Small chunks (128–256 tokens) for search, but retrieval returns the parent (1024+ tokens) for context. Precise search plus full context.

Late chunking. Embed the entire document in one pass first, then split into chunks. Each chunk retains the context of the whole document in its embedding. Jina AI proposed the approach in 2024; by 2026 it had become the standard for long documents.

# Parent-child chunking with LangChain
from langchain.text_splitter import RecursiveCharacterTextSplitter

# Parent: large chunks for context
parent_splitter = RecursiveCharacterTextSplitter(
    chunk_size=2000,
    chunk_overlap=200,
)

# Child: small chunks for precise search
child_splitter = RecursiveCharacterTextSplitter(
    chunk_size=400,
    chunk_overlap=50,
)

# On search: find the child → return the parent

Embedding Models (2026)

Model	Dimensions	Price/MTok	MTEB Score	When to use
text-embedding-3-large (OpenAI)	3072	$0.13	~64.6	High accuracy, English-first
text-embedding-3-small (OpenAI)	1536	$0.02	~62.3	Budget option
Cohere Embed v4	1024	$0.12	~66.3	Multilingual, the only multimodal model (text + images)
Voyage 3.5	2048	$0.06	~65+	Best retrieval for code, legal, and medical text; 32K context
BGE-M3 (open-source)	1024	$0 (self-hosted)	~64+	Self-hosted, no vendor lock-in

Recommendation: for most use cases, text-embedding-3-small ($0.02/MTok). For production with high quality requirements, Voyage 3.5 (especially for domain-specific content) or Cohere Embed v4 (multimodality and multilingual support).

Vector Stores

Solution	Type	Latency (1M vectors)	Cost	When to use
pgvector	PostgreSQL extension	5–8ms (HNSW)	$0 (part of PostgreSQL)	Already running PostgreSQL, <5M vectors
Qdrant	Standalone, Rust	3–5ms	Self-hosted / $25+/mo cloud	Performance-critical, >5M vectors
Pinecone	Managed SaaS	10–20ms	$70+/mo (Serverless)	Zero-ops, scale to billions
Weaviate	Standalone, Go	5–10ms	Self-hosted / $25+/mo	Multimodal, GraphQL API
ChromaDB	Embedded	1–3ms (small datasets)	$0	Prototypes, <100K vectors

pgvector is the right starting point for 90% of projects. If PostgreSQL is already in the stack (and by 2026 it almost always is), pgvector installs with a single command:

-- Install pgvector (Supabase — enabled by default)
CREATE EXTENSION IF NOT EXISTS vector;

-- Table for documents
CREATE TABLE documents (
    id BIGSERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    metadata JSONB DEFAULT '{}',
    embedding VECTOR(1536)  -- text-embedding-3-small
);

-- HNSW index — fast approximate search
CREATE INDEX ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

RAG pipeline on Supabase + pgvector (Python):

import openai
from supabase import create_client

supabase = create_client(SUPABASE_URL, SUPABASE_KEY)
embeddings = openai.OpenAI()

async def index_document(content: str, metadata: dict):
    """Index a document: chunking → embedding → store"""
    chunks = semantic_chunk(content, max_tokens=512)

    for chunk in chunks:
        response = embeddings.embeddings.create(
            model="text-embedding-3-small",
            input=chunk
        )
        vector = response.data[0].embedding

        supabase.table("documents").insert({
            "content": chunk,
            "metadata": metadata,
            "embedding": vector
        }).execute()


async def search(query: str, top_k: int = 5) -> list[dict]:
    """Semantic search: query → embedding → vector search"""
    response = embeddings.embeddings.create(
        model="text-embedding-3-small",
        input=query
    )
    query_vector = response.data[0].embedding

    # pgvector cosine similarity search
    results = supabase.rpc("match_documents", {
        "query_embedding": query_vector,
        "match_threshold": 0.7,
        "match_count": top_k
    }).execute()

    return results.data

-- Supabase Edge Function for semantic search
CREATE OR REPLACE FUNCTION match_documents(
    query_embedding VECTOR(1536),
    match_threshold FLOAT,
    match_count INT
)
RETURNS TABLE (
    id BIGINT,
    content TEXT,
    metadata JSONB,
    similarity FLOAT
)
LANGUAGE plpgsql
AS $$
BEGIN
    RETURN QUERY
    SELECT
        d.id,
        d.content,
        d.metadata,
        1 - (d.embedding <=> query_embedding) AS similarity
    FROM documents d
    WHERE 1 - (d.embedding <=> query_embedding) > match_threshold
    ORDER BY d.embedding <=> query_embedding
    LIMIT match_count;
END;
$$;

Reranking

Vector search returns the top-20 candidates. A reranker re-scores them more precisely. It’s a two-stage architecture: fast recall (vector search) followed by precise scoring (reranker).

Model	Price	Quality	When
Cohere Rerank v3.5	$2/1K searches	High	Production, multilingual (100+ languages)
BGE Reranker v2.5	$0 (self-hosted)	Medium-high	Self-hosted, budget
Jina Reranker v2	$0.02/1K queries	High	Price/quality balance

import cohere

co = cohere.Client(api_key="...")

# Vector search returned 20 candidates
candidates = await search(query, top_k=20)

# Reranker re-scores relevance
reranked = co.rerank(
    model="rerank-v3.5",  # Cohere Rerank v3.5, April 2026
    query=query,
    documents=[c["content"] for c in candidates],
    top_n=5  # Keep top-5
)

# reranked.results — sorted by precise relevance

Hybrid Search: Vector + BM25

Vector search finds semantically similar content. BM25 finds exact matches (keyword search). Hybrid search covers both scenarios.

# Reciprocal Rank Fusion (RRF) — a simple way to combine results
def reciprocal_rank_fusion(
    vector_results: list,
    bm25_results: list,
    k: int = 60
) -> list:
    """Combines vector and keyword search via RRF."""
    scores = {}

    for rank, doc in enumerate(vector_results):
        scores[doc["id"]] = scores.get(doc["id"], 0) + 1 / (k + rank + 1)

    for rank, doc in enumerate(bm25_results):
        scores[doc["id"]] = scores.get(doc["id"], 0) + 1 / (k + rank + 1)

    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

pgvector + PostgreSQL Full-Text Search — hybrid search without extra infrastructure:

-- Hybrid search: vector similarity + text relevance
SELECT
    d.id,
    d.content,
    -- Normalized scores
    (1 - (d.embedding <=> query_embedding)) * 0.7 AS vector_score,
    ts_rank(d.fts, plainto_tsquery('english', query_text)) * 0.3 AS text_score,
    -- Combined score
    (1 - (d.embedding <=> query_embedding)) * 0.7 +
    ts_rank(d.fts, plainto_tsquery('english', query_text)) * 0.3 AS combined_score
FROM documents d
WHERE
    d.fts @@ plainto_tsquery('english', query_text)
    OR (1 - (d.embedding <=> query_embedding)) > 0.5
ORDER BY combined_score DESC
LIMIT 10;

GraphRAG and LightRAG

By 2026, Microsoft’s GraphRAG went from an academic paper to a production tool. The idea: instead of indexing flat chunks, build a graph of entities and relationships on top of the corpus, and traverse the graph when searching. This produces a fundamentally different class of answers.

When you need GraphRAG:

• Questions about relationships between documents (“which companies worked with X,” “how did Y’s position change over time”)
• Global summarization analysis of the entire corpus
• Thematic and pattern-search queries

LightRAG — a lightweight alternative from HKU (2024). Same entity graph, without Microsoft’s heavy infrastructure. Quality is comparable to GraphRAG at 10–30x lower cost and 6–13x lower latency.

The 2026 practice: production stacks for enterprise corpora with rich relational semantics increasingly use a vector + graph hybrid, routing the query to the right index based on question type.

# Simplified routing: vector vs graph
def route_query(query: str) -> str:
    """Determine the retrieval strategy based on query type."""
    relational_signals = [
        "how are", "related to", "relationship between",
        "what do", "have in common", "how has", "changed over time",
        "who worked with", "pattern"
    ]
    if any(signal in query.lower() for signal in relational_signals):
        return "graph"  # GraphRAG / LightRAG
    return "vector"  # Standard vector search + reranking

Agentic RAG

Classic RAG: query → search → result. Agentic RAG: the agent decides the search strategy.

# Agentic RAG: the agent chooses the strategy
tools = [
    {
        "name": "search_docs",
        "description": "Semantic search over documentation",
        "parameters": {"query": "str", "top_k": "int", "filters": "dict"}
    },
    {
        "name": "search_code",
        "description": "Search the codebase (grep, AST)",
        "parameters": {"pattern": "str", "file_types": "list[str]"}
    },
    {
        "name": "search_tickets",
        "description": "Search Jira/Linear tickets",
        "parameters": {"query": "str", "status": "str"}
    },
    {
        "name": "graph_search",
        "description": "GraphRAG — search the entity graph for relationship questions",
        "parameters": {"query": "str", "depth": "int"}
    }
]

# The agent receives the question: "Why does auth break after deploy?"
# The agent decides:
# 1. search_code("auth", file_types=["py"]) — find the auth module
# 2. search_tickets("auth deploy bug", status="recent") — find related tickets
# 3. search_docs("deployment auth configuration") — check the documentation
# 4. If needed, reformulate the query and search again
# Combines the results and answers

Agentic RAG is context engineering in action. The agent doesn’t just retrieve documents — it orchestrates search across multiple sources.

Decision Framework: What to Choose

Flowchart

Size of the data corpus?
├── < 200K tokens
│   └── → Long context (everything in the prompt + prompt caching)
├── 200K – 1M tokens
│   ├── Is the data static?
│   │   ├── Yes → Long context + prompt caching
│   │   └── No → RAG (incremental index)
│   ├── Latency < 1s?
│   │   ├── Yes → RAG
│   │   └── No → Long context is acceptable
│   └── Budget constrained + high load?
│       ├── Yes → RAG (fewer tokens per request)
│       └── No → Long context is simpler
└── > 1M tokens
    └── → RAG (required)
        ├── Need attribution? → RAG with metadata
        ├── Questions about relationships between documents? → GraphRAG / LightRAG
        ├── Complex multi-hop queries? → Agentic RAG
        └── Multilingual? → Cohere Embed v4 / BGE-M3

Decision Matrix

Scenario	Data size	Updates	Latency	Recommendation
Support chatbot	500+ documents	Weekly	<1s	RAG + reranking
Code assistant	50–100K lines	Continuous	2–5s acceptable	Long context (grep/glob)
Legal research	100K+ documents	Rare	Not critical	RAG + hybrid search
Product documentation	20–50 documents	Monthly	<2s	Long context + cache
Customer analytics	1M+ records	Daily	<500ms	RAG + structured retrieval
Onboarding assistant	30 internal docs	Quarterly	<3s	Long context + cache
Research assistant	10K+ papers	Weekly	Not critical	Agentic RAG + GraphRAG + reranking

Real-World Scenarios

Customer support bot. 2,000 FAQ articles, updated weekly, needs a response within 1 second, needs a link to the source. → RAG. Hybrid search (vector + BM25) + Cohere Rerank + attribution via metadata.

Code assistant (Claude Code). An 80K-line codebase, updated continuously, 2–5s latency acceptable, needs full file context. → Long context. Grep/glob for retrieval, the whole file goes into context, CLAUDE.md for project conventions.

Legal research. 50,000 court rulings, updated monthly, needs precise citation, complex queries (cross-references, relationships between precedents). → Agentic RAG + GraphRAG. Parent-child chunking, hybrid search, reranking, an entity graph for connections between cases, the agent decides the search strategy.

Internal knowledge base. 40 documents, 150K tokens, updated once a month, 50 requests/day. → Long context + prompt caching. All documents in the system prompt, cache read at $0.10/request. Cost: $5/day = $150/month. A RAG pipeline would cost more.

The Combined Approach

RAG + Context Engineering: Not Either/Or

Production systems in 2026 combine both approaches. RAG handles retrieval. Context engineering handles assembly.

┌─────────────────────────────────────────────────┐
│                Context Assembly                  │
│  (Context Engineering — orchestration layer)     │
├─────────────────────────────────────────────────┤
│                                                  │
│  1. System Prompt          ← Static, cached      │
│     ├── Role definition                          │
│     ├── Project context (CLAUDE.md)              │
│     └── Output format                            │
│                                                  │
│  2. Retrieved Context      ← RAG pipeline        │
│     ├── Vector search results                    │
│     ├── BM25 keyword matches                     │
│     └── Reranked top-K                           │
│                                                  │
│  3. Tool Results           ← On-demand via MCP   │
│     ├── Database queries                         │
│     ├── API responses                            │
│     └── Code search results                      │
│                                                  │
│  4. Memory                 ← Persistent storage  │
│     ├── Previous conversation summary            │
│     ├── User preferences                         │
│     └── Learned patterns                         │
│                                                  │
│  5. Few-shot Examples      ← Cached at the end   │
│                                                  │
│  → Assembled Context → LLM → Response            │
│                                                  │
└─────────────────────────────────────────────────┘

Example Architecture: A Support Bot with a Combined Approach

async def handle_query(user_query: str, user_id: str) -> str:
    """Combined approach: RAG + context engineering."""

    # 1. System prompt (cached, Write strategy)
    system = build_system_prompt(
        role="support_agent",
        company_policies=CACHED_POLICIES,  # prompt caching
        output_format=TICKET_SCHEMA
    )

    # 2. RAG retrieval (Select strategy)
    relevant_docs = await rag_search(
        query=user_query,
        top_k=5,
        rerank=True
    )

    # 3. User context (Select strategy — tools)
    user_history = await get_recent_tickets(user_id, limit=3)
    user_plan = await get_subscription_plan(user_id)

    # 4. Memory (Write/Select strategy)
    conversation_summary = await get_conversation_memory(user_id)

    # 5. Assembly (Context Engineering — orchestration)
    context = assemble_context(
        system=system,
        retrieved_docs=relevant_docs,      # RAG results
        user_context={
            "history": user_history,
            "plan": user_plan
        },
        memory=conversation_summary,
        examples=FEW_SHOT_SUPPORT_EXAMPLES  # Cached
    )

    # 6. Compress if needed (Compress strategy)
    if count_tokens(context) > MAX_CONTEXT:
        context = compress_context(context, target_tokens=MAX_CONTEXT)

    # 7. LLM call
    response = await llm.generate(context=context, query=user_query)

    # 8. Save to memory (Write strategy)
    await update_conversation_memory(user_id, user_query, response)

    return response

When Combining Is Necessary

• Personalization. RAG finds documents, but the response needs to be tailored to the user (plan, history, preferences). Without context engineering, you get a generic answer.

• Multi-source retrieval. Data lives in several systems: docs in a vector store, tickets in Jira, code in Git. RAG doesn’t know about Jira. Context engineering orchestrates all the sources.

• Stateful conversations. RAG is stateless. Every request is independent. A conversation needs memory — the Write strategy. RAG + memory = a combined approach.

• Quality optimization. RAG returned 5 chunks. But one is a duplicate, another is outdated. Context engineering filters, deduplicates, and prioritizes.

Metrics and Evaluation

Context Quality Metrics

Metric	What it measures	How to calculate
Relevance	How relevant the context is to the query	LLM-as-judge: “rate relevance 1–5”
Coverage	Whether the context covers all aspects of the question	% of question aspects backed by data in the context
Noise ratio	The share of irrelevant content	Tokens of irrelevant content / total tokens
Token efficiency	Useful tokens per unit of cost	Relevance score / cost per query

RAG Metrics

Precision@K — of the K retrieved documents, how many are relevant.

Precision@5 = 4 relevant out of 5 retrieved = 0.80

Recall@K — of all the relevant documents in the corpus, how many made it into the top-K.

10 relevant documents in the corpus, 4 made it into the top-5 → Recall@5 = 0.40

MRR (Mean Reciprocal Rank) — at what position the first relevant document appears.

First relevant document at position 2 → RR = 1/2 = 0.50
Average across all queries = MRR

NDCG (Normalized Discounted Cumulative Gain) — accounts for both the presence and the position of each relevant document.

End-to-End Metrics

Metric	What it measures	Tools
Answer accuracy	Whether the answer is correct	Eval set + LLM-as-judge
Faithfulness	Whether the answer is grounded in context, not a hallucination	RAGAS, DeepEval
Hallucination rate	% of claims without grounding in context	RAGAS faithfulness metric
Latency (p50, p95)	Time from request to response	Langfuse, LangSmith
Cost per query	Total cost: embedding + retrieval + LLM	Billing API + custom tracking

Eval Pipeline

from ragas import evaluate
from ragas.metrics import (
    faithfulness,
    answer_relevancy,
    context_precision,
    context_recall
)

# Eval dataset: question + reference answer + context
eval_results = evaluate(
    dataset=eval_dataset,
    metrics=[
        faithfulness,          # Is the answer grounded in context?
        answer_relevancy,      # Is the answer relevant to the question?
        context_precision,     # Is the retrieved context precise?
        context_recall,        # Does the context cover all aspects?
    ]
)

# Result:
# faithfulness: 0.92
# answer_relevancy: 0.88
# context_precision: 0.85
# context_recall: 0.78 → retrieval needs improvement

Bottom Line

Context engineering and RAG aren’t competitors. Context engineering is the discipline. RAG is one of the tools inside it.

Four rules:

1. Data fits in the context window and doesn’t change often — long context + prompt caching. Simpler, cheaper, more reliable.

2. The corpus is large, data updates, you need attribution or tight latency — RAG. With hybrid search and reranking.

3. You need answers about relationships between documents, patterns, or global corpus analysis — GraphRAG or LightRAG. An entity graph where flat vector search falls short.

4. Production — a combination. RAG for retrieval, context engineering for assembly, prompt caching for cost savings. A graph where relationships matter. Neither works as well alone as the two together.

Context windows will keep growing. Token costs will keep falling. But the principle stays the same: answer quality is determined not by how many tokens are in the context, but by how relevant they are. That’s what context engineering is about.