Context Engineering for LLM Agents: A Practical Guide

Context engineering answers a simple production question: what should the model see before it takes the next step?

That includes far more than the user prompt. An agent may receive project rules, conversation history, retrieved documents, tool schemas, tool results, memory, permissions, and an output contract. Any one of them can be missing, stale, contradictory, or malicious.

Prompt engineering still matters. It tells the model how to approach a task. Context engineering controls the evidence and state from which that task is attempted.

A Large Window Is Capacity, Not Recall

An accepted token count tells you what fits through an API. It does not promise that every fact will influence the answer equally or correctly.

The Lost in the Middle paper demonstrated that model performance on multi-document question answering and key-value retrieval could change with the position of relevant information. Later models and tasks behave differently, so “put everything at the edges” is not a universal fix. The engineering lesson is narrower: evaluate retrieval and reasoning on the actual length, ordering, and distractors used in production.

More context also creates straightforward costs:

• input tokens are processed on each model call unless a provider cache applies;
• repeated tool output can dominate an agent loop;
• obsolete instructions remain available to compete with current ones;
• sensitive data reaches every provider that receives the assembled request;
• a larger evidence set gives validation more claims to check.

The target is not the smallest possible prompt. It is the smallest context that still contains the evidence and constraints required by the task.

Model Context as Typed Sections

Do not assemble one undifferentiated text blob. Give each section a role and a trust level.

1. PLATFORM POLICY       non-negotiable safety and permission rules
2. PROJECT GUIDANCE      architecture, conventions, allowed dependencies
3. TASK                  goal, scope, deadline, output contract
4. CURRENT STATE         files, workflow state, recent tool results
5. RETRIEVED EVIDENCE    quoted or structured source material
6. EXAMPLES              demonstrations of the expected result

This is not a magical six-part prompt formula. Some tasks need three sections; others need ten. The value comes from visible boundaries:

• instructions are not mistaken for source data;
• retrieved pages cannot silently override system policy;
• stale state has an owner and timestamp;
• citations can point back to a source;
• the application can budget or omit each section independently.

Wrap untrusted material explicitly. A web page, support ticket, code comment, or uploaded document may contain text such as “ignore previous instructions.” That text is data to analyze, not policy to execute.

Define a Context Contract Per Task

Different tasks need different evidence. A code review needs the diff, nearby interfaces, test output, and repository conventions. It rarely needs every file in the repository. A support reply needs the customer’s message, relevant account state, and approved policy—not another customer’s entire history.

A task contract can make that boundary executable:

interface ContextContract {
  task: 'review_pull_request' | 'answer_support_ticket';
  requiredSources: string[];
  optionalSources: string[];
  forbiddenDataClasses: string[];
  maxAgeMinutes: Record<string, number>;
  tokenBudget: number;
  outputSchema: string;
}

The contract answers five questions before the model runs:

1. Which facts must be present?
2. Which sources are authoritative when they disagree?
3. How fresh must each source be?
4. Which data is forbidden for this task or provider?
5. What output can the application validate?

Without this contract, retrieval quality is judged by whether the response “looks good.” That standard fails quietly.

Select Before You Compress

Anthropic’s context-engineering guidance groups common operations into writing, selecting, compressing, and isolating context. The order matters.

Selection removes information that does not belong in this request. Compression rewrites information that still belongs but is too large. Summarizing a pile of irrelevant logs produces a shorter pile of irrelevant logs.

For code tasks, start with deterministic discovery:

• the changed files and nearby interfaces;
• callers and tests found by code search;
• the project instructions that apply to those paths;
• the exact failing command and its concise error;
• dependency or API documentation for the versions in use.

For knowledge tasks, combine retrieval signals rather than trusting one vector score. Metadata filters, keyword search, embeddings, reranking, recency, and access control solve different parts of the problem.

The separate RAG versus context guide provides a decision framework for retrieval-heavy workloads.

Preserve Provenance Through Retrieval

A retrieved paragraph without its source, date, and surrounding topic is easy to misuse. Keep provenance attached to the fragment:

{
  "text": "Refunds are available within 30 days...",
  "sourceId": "policy/refunds/v4",
  "sourceUrl": "https://docs.example.com/refunds",
  "updatedAt": "2026-07-11T09:30:00Z",
  "accessScope": "support-emea",
  "section": "Eligibility"
}

Anthropic’s Contextual Retrieval experiments add short document-level context to chunks before indexing. The general idea is useful even if you use a different stack: a fragment should carry enough context to be retrieved and cited correctly.

Never let ranking bypass authorization. Apply tenant and document permissions before content enters the model request, not after generation.

Treat Tool Definitions as Context

Tool names, descriptions, schemas, and returned data all consume context and shape behavior. More tools do not automatically create a more capable agent. Ambiguous tools create ambiguous choices.

A useful tool contract has:

• a specific action in its name;
• a description that says when to use it and when not to;
• typed input with domain names such as customerId, not data;
• structured output where the protocol supports it;
• bounded result size or pagination;
• an explicit error shape;
• server-side authorization independent of the model.

Load a small stable set eagerly. Discover specialized tools when the task needs them. Measure the extra discovery step against the token and selection cost of including every schema in every call.

For MCP systems, the production MCP server guide covers tool boundaries and operational controls.

Separate History, Working State, and Memory

These are different data products:

• History is the interaction transcript.
• Working state is what the current workflow has established.
• Memory is durable information expected to help future tasks.

Copying all history into every request mixes the three. A temporary debugging hypothesis can survive as if it were a decision. A user correction can be buried under earlier turns. Deleting one personal fact becomes difficult.

For a long-running coding task, keep an explicit checkpoint:

## Goal
Add idempotent invoice import without changing the public API.

## Verified state
- Parser tests pass.
- Database uniqueness constraint exists on provider + external_id.

## Decisions
- Retry import jobs; do not retry payment capture.

## Failed approach
- In-memory deduplication fails across workers.

## Next step
Add the concurrency test before changing the worker.

The checkpoint is inspectable, editable, and cheap to reload. The agent memory guide goes deeper into provenance, conflicts, retention, and deletion.

Compact at Checkpoints, Not by Panic

Compaction is lossy. A fluent summary can still omit the one constraint needed for the next step.

Compact when the workflow reaches a stable boundary: a test passes, a decision is accepted, or an incident phase closes. Preserve:

• the current goal and non-goals;
• verified facts with source references;
• decisions and their rationale;
• files or records changed;
• commands run and their result;
• failed approaches worth avoiding;
• unresolved questions and the next action.

Do not summarize secrets into a durable file. Do not preserve raw personal data when a stable identifier is enough. Keep the original source available when policy permits, and validate the compacted state before discarding history.

Isolate Work With a Narrow Handoff

A separate worker or subagent is useful when a subtask needs a clean context or a different permission boundary. Isolation is not the same as dumping the parent’s full transcript into a new window.

A good handoff contains:

• one bounded objective;
• the minimum relevant sources;
• constraints and forbidden changes;
• the required return format;
• how the parent will verify the result.

Parallel workers also create merge and consistency risk. Two agents changing the same plan or file are not “more context engineering”; they are a coordination bug.

Test Context Assembly Like Application Code

Model evals should diagnose the context pipeline, not only the final answer. Build cases for:

Record the context manifest used for each evaluation: source IDs, versions, ordering, token count by section, retrieval scores, and policy filters. Avoid raw PII in traces. If an answer regresses, the manifest helps distinguish a model change from a retrieval or assembly change.

The agent testing guide covers evaluation sets and release gates.

A Practical Review Checklist

Before the call:

• The task has a clear output contract and stopping condition.
• Required evidence is present and carries provenance.
• Authority and freshness rules resolve conflicting sources.
• Untrusted content is separated from instructions.
• Tenant, region, and data-class filters run before retrieval results are sent.
• Tool definitions are relevant, typed, and authorized server-side.
• The request fits a budget derived from the task, not the model maximum.

After the call:

• Structured output and domain rules pass validation.
• Claims can be traced to the supplied evidence.
• Tool calls respect permissions and idempotency.
• New durable facts have source, timestamp, and deletion policy.
• The context manifest is available for debugging without exposing raw PII.

Primary References

• Anthropic: Effective context engineering for AI agents
• Lost in the Middle: How Language Models Use Long Contexts
• Anthropic: Contextual Retrieval
• Model Context Protocol: Tools specification

Good context is not “everything the model might need.” It is a reviewed, traceable input state with enough evidence to complete one task and enough boundaries to prevent unrelated data from becoming instructions.