Context Engineering for LLM Agents: A Practical Guide

Context engineering answers a simple production question: what should the model see before it takes the next step?

That includes far more than the user prompt. An agent may receive project rules, conversation history, retrieved documents, tool schemas, tool results, memory, permissions, and an output contract. Any one of them can be missing, stale, contradictory, or malicious.

Prompt engineering still matters. It tells the model how to approach a task. Context engineering controls the evidence and state from which that task is attempted.

A Large Window Is Capacity, Not Recall

An accepted token count tells you what fits through an API. It does not promise that every fact will influence the answer equally or correctly.

The Lost in the Middle paper demonstrated that model performance on multi-document question answering and key-value retrieval could change with the position of relevant information. Later models and tasks behave differently, so “put everything at the edges” is not a universal fix. The engineering lesson is narrower: evaluate retrieval and reasoning on the actual length, ordering, and distractors used in production.

More context also creates straightforward costs:

• input tokens are processed on each model call unless a provider cache applies;
• repeated tool output can dominate an agent loop;
• obsolete instructions remain available to compete with current ones;
• sensitive data reaches every provider that receives the assembled request;
• a larger evidence set gives validation more claims to check.

The target is not the smallest possible prompt. It is the smallest context that still contains the evidence and constraints required by the task.

Model Context as Typed Sections

Do not assemble one undifferentiated text blob. Give each section a role and a trust level.

1. PLATFORM POLICY       non-negotiable safety and permission rules
2. PROJECT GUIDANCE      architecture, conventions, allowed dependencies
3. TASK                  goal, scope, deadline, output contract
4. CURRENT STATE         files, workflow state, recent tool results
5. RETRIEVED EVIDENCE    quoted or structured source material
6. EXAMPLES              demonstrations of the expected result

This is not a magical six-part prompt formula. Some tasks need three sections; others need ten. The value comes from visible boundaries:

• instructions are not mistaken for source data;
• retrieved pages cannot silently override system policy;
• stale state has an owner and timestamp;
• citations can point back to a source;
• the application can budget or omit each section independently.

Wrap untrusted material explicitly. A web page, support ticket, code comment, or uploaded document may contain text such as “ignore previous instructions.” That text is data to analyze, not policy to execute.

Define a Context Contract Per Task

Different tasks need different evidence. A code review needs the diff, nearby interfaces, test output, and repository conventions. It rarely needs every file in the repository. A support reply needs the customer’s message, relevant account state, and approved policy—not another customer’s entire history.

A task contract can make that boundary executable:

interface ContextContract {
  task: 'review_pull_request' | 'answer_support_ticket';
  requiredSources: string[];
  optionalSources: string[];
  forbiddenDataClasses: string[];
  maxAgeMinutes: Record<string, number>;
  tokenBudget: number;
  outputSchema: string;
}

The contract answers five questions before the model runs:

1. Which facts must be present?
2. Which sources are authoritative when they disagree?
3. How fresh must each source be?
4. Which data is forbidden for this task or provider?
5. What output can the application validate?

Without this contract, retrieval quality is judged by whether the response “looks good.” That standard fails quietly.

A Tested Context Manifest

The interface above is useful only if the application enforces it. We built a runnable context assembly fixture in strict TypeScript to make the decision visible before a model call.

The fixed task answers one synthetic refund-policy question. Its contract names one required policy, three optional source types, caller scopes, forbidden data classes, freshness limits, and a 430-token source budget. The assembler applies tenant, scope, data-class, and freshness gates before it ranks eligible candidates. Missing or oversized required evidence stops the run; optional evidence cannot crowd out policy.

The reviewed run started with nine candidates:

The manifest selected 424 of 430 supplied source tokens, leaving six unused. Those are fixture values, not counts from an API tokenizer. Sixteen tests passed: required-source failures, stale evidence, tenant and scope isolation, forbidden data, distractor overload, budget overflow, duplicate provenance, prompt injection boundaries, deterministic JSON, and the generated SVG.

The result is intentionally narrow. It proves the fixture’s filtering and serialization behavior. It does not prove that the ranking order fits another product or that a model will answer correctly. Download the code, tests, manifest, and evidence report and replace the synthetic contract with an eval from your own task.

Select Before You Compress

Anthropic’s context-engineering guidance groups common operations into writing, selecting, compressing, and isolating context. The order matters.

Selection removes information that does not belong in this request. Compression rewrites information that still belongs but is too large. Summarizing a pile of irrelevant logs produces a shorter pile of irrelevant logs.

For code tasks, start with deterministic discovery:

• the changed files and nearby interfaces;
• callers and tests found by code search;
• the project instructions that apply to those paths;
• the exact failing command and its concise error;
• dependency or API documentation for the versions in use.

For knowledge tasks, combine retrieval signals rather than trusting one vector score. Metadata filters, keyword search, embeddings, reranking, recency, and access control solve different parts of the problem.

The separate RAG versus context guide provides a decision framework for retrieval-heavy workloads.

Preserve Provenance Through Retrieval

A retrieved paragraph without its source, date, and surrounding topic is easy to misuse. Keep provenance attached to the fragment:

{
  "text": "Refunds are available within 30 days...",
  "sourceId": "policy/refunds/v4",
  "sourceUrl": "https://docs.example.com/refunds",
  "updatedAt": "2026-07-11T09:30:00Z",
  "accessScope": "support-emea",
  "section": "Eligibility"
}

Anthropic’s Contextual Retrieval experiments add short document-level context to chunks before indexing. The general idea is useful even if you use a different stack: a fragment should carry enough context to be retrieved and cited correctly.

Never let ranking bypass authorization. Apply tenant and document permissions before content enters the model request, not after generation.

Treat Tool Definitions as Context

Tool names, descriptions, schemas, and returned data all consume context and shape behavior. More tools do not automatically create a more capable agent. Ambiguous tools create ambiguous choices.

A useful tool contract has:

• a specific action in its name;
• a description that says when to use it and when not to;
• typed input with domain names such as customerId, not data;
• structured output where the protocol supports it;
• bounded result size or pagination;
• an explicit error shape;
• server-side authorization independent of the model.

Load a small stable set eagerly. Discover specialized tools when the task needs them. Measure the extra discovery step against the token and selection cost of including every schema in every call.

For MCP systems, the production MCP server guide covers tool boundaries and operational controls.

Separate History, Working State, and Memory

These are different data products:

• History is the interaction transcript.
• Working state is what the current workflow has established.
• Memory is durable information expected to help future tasks.

Copying all history into every request mixes the three. A temporary debugging hypothesis can survive as if it were a decision. A user correction can be buried under earlier turns. Deleting one personal fact becomes difficult.

For a long-running coding task, keep an explicit checkpoint:

## Goal
Add idempotent invoice import without changing the public API.

## Verified state
- Parser tests pass.
- Database uniqueness constraint exists on provider + external_id.

## Decisions
- Retry import jobs; do not retry payment capture.

## Failed approach
- In-memory deduplication fails across workers.

## Next step
Add the concurrency test before changing the worker.

The checkpoint is inspectable, editable, and cheap to reload. The agent memory guide goes deeper into provenance, conflicts, retention, and deletion.

Compact at Checkpoints, Not by Panic

Compaction is lossy. A fluent summary can still omit the one constraint needed for the next step.

Compact when the workflow reaches a stable boundary: a test passes, a decision is accepted, or an incident phase closes. Preserve:

• the current goal and non-goals;
• verified facts with source references;
• decisions and their rationale;
• files or records changed;
• commands run and their result;
• failed approaches worth avoiding;
• unresolved questions and the next action.

Do not summarize secrets into a durable file. Do not preserve raw personal data when a stable identifier is enough. Keep the original source available when policy permits, and validate the compacted state before discarding history.

Isolate Work With a Narrow Handoff

A separate worker or subagent is useful when a subtask needs a clean context or a different permission boundary. Isolation is not the same as dumping the parent’s full transcript into a new window.

A good handoff contains:

• one bounded objective;
• the minimum relevant sources;
• constraints and forbidden changes;
• the required return format;
• how the parent will verify the result.

Parallel workers also create merge and consistency risk. Two agents changing the same plan or file are not “more context engineering”; they are a coordination bug.

Test Context Assembly Like Application Code

Model evals should diagnose the context pipeline, not only the final answer. Build cases for:

Record the context manifest used for each evaluation: source IDs, versions, ordering, token count by section, retrieval scores, and policy filters. Avoid raw PII in traces. If an answer regresses, the manifest helps distinguish a model change from a retrieval or assembly change.

The downloadable fixture above is a starting contract test, not an end-to-end eval. Add your retriever, provider tokenizer, model output checks, and real failure distribution before using its thresholds as a release gate.

The agent testing guide covers evaluation sets and release gates.

A Practical Review Checklist

Before the call:

• The task has a clear output contract and stopping condition.
• Required evidence is present and carries provenance.
• Authority and freshness rules resolve conflicting sources.
• Untrusted content is separated from instructions.
• Tenant, region, and data-class filters run before retrieval results are sent.
• Tool definitions are relevant, typed, and authorized server-side.
• The request fits a budget derived from the task, not the model maximum.

After the call:

• Structured output and domain rules pass validation.
• Claims can be traced to the supplied evidence.
• Tool calls respect permissions and idempotency.
• New durable facts have source, timestamp, and deletion policy.
• The context manifest is available for debugging without exposing raw PII.

Primary References

• Anthropic: Effective context engineering for AI agents
• Lost in the Middle: How Language Models Use Long Contexts
• Anthropic: Contextual Retrieval
• Model Context Protocol 2026-07-28: Tools

Good context is not “everything the model might need.” It is a reviewed, traceable input state with enough evidence to complete one task and enough boundaries to prevent unrelated data from becoming instructions.