# LLM Context Window: What It Is, Sizes, and Why Models Forget

> What an LLM context window is, what fills it, window sizes for Claude, GPT-6, Gemini and DeepSeek, what happens on overflow, and why long context costs more.
> Author: Roman Belov · Published: 2026-10-24 · Source: https://futurecraft.pro/blog/context-window/

**A context window is the maximum number of tokens a language model processes in a single request.** It holds the system instructions, the whole conversation history, attached documents, tool definitions and the model's own response. Anything that doesn't fit in the window doesn't exist for the model.

The window isn't training data and it isn't long-term memory. The model stores nothing between requests. In a chat, "memory" comes from the application, which resends the history on every turn. That explains two common surprises. A model with a 1M-token window still "forgets" the start of a conversation. And a long chat costs more than a short one, even when your questions stay just as short.

This article covers how the window works, how big it is as of November 2026, what happens at the limit and why long context is expensive. The numbers on window growth and overflow come from the author's measurements on November 8, 2026, via OpenRouter and Ollama 0.40.14.

## What is a context window in an LLM

Anthropic's documentation describes the context window as "all the text a language model can reference when generating a response, including the response itself" and calls it a "working memory" for the model ([Context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows)). The window is measured in tokens, not words or characters.

A token is a piece of text: a word, part of a word or a symbol. The tokenizer splits the input into these pieces. According to Anthropic, 1M tokens on the current Claude tokenizer (introduced with Opus 4.7) is about 555,000 words or 2.5 million characters, compared with about 750,000 words on earlier models ([Models overview](https://platform.claude.com/docs/en/models/overview)). Language matters too: in the measurements from the article [on tokens](https://futurecraft.pro/blog/tokens-explained/), Russian prose took 1.19-1.28x more tokens on Claude Haiku 5.5 than its English translation.

### What fills the window

Everything in the request goes into the window. The user's latest message is only one part of it. The full list, from Anthropic's documentation:

| What takes up the window | Where it comes from | Notes |
|---|---|---|
| System prompt | the app, the agent, CLAUDE.md | sent with every request |
| `messages` history | all previous turns | grows every turn |
| Files, images, PDFs | user attachments | up to 600 images or PDF pages per request |
| Tool definitions | MCP servers, functions | tool schemas are text too |
| Tool results | file reads, search, logs | tool output goes into the window in full |
| Reasoning (thinking) | the model | on current models, including Haiku 5.5 and Fable 5.1, earlier thinking blocks stay in the window by default |
| Model response | the model | counts toward the same request's window |

Prompt caching doesn't free up space. The `input_tokens`, `cache_read_input_tokens` and `cache_creation_input_tokens` fields all count toward the window. The cache changes what you pay for those tokens, not how many there are.

### Output takes space too

The window is shared between input and output. If the input takes 990K of a 1M window, only 10K is left for the answer, even with `max_tokens` set to 128K. The model's reasoning is part of `max_tokens` and is billed as output tokens.

Google states this directly on the model card: the Gemini 4 Argon window of 1,048,576 tokens is a combined limit for input and output ([Gemini 4 Argon](https://aistudio.google.com/docs/models/gemini-4-argon)).

## Context window sizes in November 2026

Snapshot as of November 8, 2026, from vendor pages. Prices are in US dollars per 1M tokens, standard tier, without caching or batch discounts.

| Model | Window | Max output | Input / output | Long context |
|---|---|---|---|---|
| Claude Fable 5.1 | 1M | 128K | $10 / $50 | one rate |
| Claude Opus 5.5 | 1M | 128K | $4 / $20 | one rate |
| Claude Sonnet 5.5 | 1M | 128K | $2 / $10 | one rate |
| Claude Haiku 5.5 | 1M | 128K | $0.10 / $0.50 | prompt over 100K: $0.50 / $2.50 |
| GPT-6 Astra | 1.05M | 128K | $10 / $50 | input over 272K: input 2x, output 1.5x |
| GPT-6.1 Sol | 1.05M | 128K | $2 / $10 | same |
| GPT-6.1 Terra | 1.05M (input up to 922K) | 128K | $1 / $5 | same |
| GPT-6 Luna | 1.05M | 128K | $0.10 / $0.50 | same |
| Gemini 4 Argon | 1,048,576 for input and output | 65,536 by default | $2 / $10 until 11/30/2026, $4 / $20 from 12/01/2026 | no separate tier |
| DeepSeek `deepseek-flash` | 1M | up to 384K | $0.15 / $0.60 off-peak | 2x at peak hours |
| DeepSeek `deepseek-v4-pro` | 1M | up to 384K | $0.66 / $1.98 off-peak | 2x at peak hours |
| Qwen3.8-27B (open weights) | 262,144 | - | local | depends on your machine's memory |

Sources: [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing), [OpenAI models](https://developers.openai.com/api/docs/models) and the [GPT-6.1 Terra page](https://developers.openai.com/api/docs/models/gpt-6.1-terra), [Gemini pricing](https://aistudio.google.com/docs/pricing), [DeepSeek pricing](https://api-docs.deepseek.com/quick_start/pricing/), Qwen from the [local LLM guide](https://futurecraft.pro/blog/local-llm-guide/). DeepSeek peak hours are 01:00-04:00 and 06:00-10:00 UTC on weekdays. DeepSeek lists the output limit in a single cell for both models.

You can't compare "1M" across vendors directly: tokenizers split the same text into different numbers of tokens. According to Anthropic, models from Opus 4.7 onward produce about 30% more tokens for the same text than earlier Claude models ([Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting)).

### API window vs app window

The window in a chat app can differ from the window in the API. On paid Claude plans, Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 have a 1M window. Some earlier models also get 1M, others get 500K, and the rest get 200K ([Claude Help Center](https://support.claude.com/en/articles/8606394-how-large-is-the-context-window-on-paid-claude-plans)).

This article gives no figures for the ChatGPT app: the primary source couldn't be checked. Don't assume the API window from the table applies to the chat.

## What happens when the context window runs out

Behavior at the limit depends on where the model runs: a cloud API, a local runner or an app built on top of an API.

### Cloud API: a 400 error

Anthropic's rule is simple. If the input alone exceeds the window, the API returns HTTP 400 `invalid_request_error` ("prompt is too long") on every model. If the input fits but input plus `max_tokens` doesn't, Claude 4.5 models and newer accept the request, and generation stops with `stop_reason: "model_context_window_exceeded"` ([Context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows)). Nothing gets cut silently.

In the author's measurement, a cloud endpoint with a 32,768-token window (`mistral-small-24b-instruct-2501` via OpenRouter) received about 41,080 input tokens and returned HTTP 400 with exact numbers:

```text
This endpoint's maximum context length is 32768 tokens. However, you requested
about 41090 tokens (41080 of text input, 10 in the output).
```

The error is annoying but honest. The client knows the request failed and how much to cut.

### Local Ollama: silent truncation

A local runner behaved differently in the same measurement. The author put the code word `PELICAN-47` at the start of a ~4,600-token text, asked about it at the end, and sent it to Ollama 0.40.14 with `num_ctx` set to 2048.

| Model | `num_ctx` | Input | Server response | `prompt_eval_count` | Code word |
|---|---|---|---|---|---|
| qwen3:8b | 2048 | ~4,592 | HTTP 200 | 1026 | not found, answer "Note" |
| gemma4:e4b | 2048 | ~4,400 | HTTP 200 | 1027 | not found: "The text does not mention a secret code word at the very beginning." |
| qwen3:8b | 8192 | 4,592 | HTTP 200 | 4592 | found |
| gemma4:e4b | 8192 | 4,399 | HTTP 200 | 4399 | found |

There was no error. Ollama cut the prompt to about half of `num_ctx`. The start of the text, code word included, was gone, and the model confidently said the word wasn't there. The only warning showed up in the server log:

```text
level=WARN msg="truncating input prompt" limit=1026 prompt=4592 keep=4 new=1026
```

The client never sees that log. The only sign in the response itself is the `prompt_eval_count` field: if it's well below the expected prompt size, the input was truncated. You can check with the same request as in the measurement, written as curl:

```bash
curl -s http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "stream": false,
  "think": false,
  "options": {"num_ctx": 2048, "num_predict": 20},
  "messages": [{"role": "user", "content": "<long text and question>"}]
}' | jq '{prompt_eval_count, answer: .message.content}'
```

This matters for agents on local models. According to the Ollama docs, machines with less than 24 GiB of VRAM get a default context of 4K tokens ([Context length](https://docs.ollama.com/context-length)), so a long agent system prompt can get truncated without a single error. The [Ollama guide](https://futurecraft.pro/blog/ollama-guide/) shows how to set the context and check it with `ollama ps`. The measurement used one Ollama version and two models. Other local runners weren't tested and may behave differently.

### Chat apps and Claude Code: compacting the history

Apps built on an API usually don't show an error. They manage the window themselves. Anthropic's documentation notes that chat interfaces such as claude.ai can manage the window on a rolling "first in, first out" basis. On paid Claude plans with code execution turned on, automatic context management kicks in near the limit: Claude summarizes earlier messages, and the full history stays available to view. Without code execution, this doesn't work ([Claude Help Center](https://support.claude.com/en/articles/8606394-how-large-is-the-context-window-on-paid-claude-plans)).

In Claude Code, auto-compaction does the same job: it "summarizes conversation history when approaching context limits" ([Manage costs](https://code.claude.com/docs/en/costs)). You can also manage it by hand:

- `/context` shows what's taking up the window;
- `/compact Focus on code samples and API usage` compacts the history and tells Claude what to keep; standing instructions can go in a `# Compact instructions` section in CLAUDE.md;
- `/clear` starts a clean session when you switch to unrelated work.

Compaction has a cost. `/compact` reads the whole conversation it summarizes, so compacting a large context is itself a large request. `/clear` costs nothing. For more on working with sessions, see the [complete Claude Code guide](https://futurecraft.pro/blog/claude-code-complete-guide/).

## Why the model "forgets"

"Forgetting" almost always comes down to one of four causes, and only the last one is about the model itself.

| Cause | What it looks like | What to do |
|---|---|---|
| New chat | the model doesn't know what you discussed last session | move the key facts into the first message or a file; product "memory" features are a separate layer with no guarantees |
| The app compacted or dropped old turns | the general thread survives, details from the start are gone | compact with instructions, keep decisions in a separate file |
| Input larger than the window | API: 400 error; local runner: answer without the start of the text | count tokens in advance, trim the input, raise `num_ctx` |
| The model sees everything but uses it worse | a fact from the middle of a long document gets ignored | shorten the context, put important parts near the start or the end |

The model is stateless. Everything that looks like conversation memory is history the app resends in the window each time. That's why a follow-up works better in the same chat: the earlier turns are in the window, and in a new chat they aren't.

The last row of the table is the subject of [Lost in the Middle](https://arxiv.org/abs/2307.03172) (Liu et al., 2023, TACL). Models used information from the start and end of the context better, and noticeably worse from the middle, even models built for long inputs. The study used 2023 models, and current generations may behave differently. Anthropic's documentation puts it directly: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." So the practical rule is: the window gives you capacity, and it doesn't guarantee that every fact gets used.

## Why long context is expensive

Long context costs more for three reasons. You pay for the whole history on every turn. Some models charge a higher rate for long prompts. And inside the model, compute and memory grow with length.

### Every turn pays for the whole history

The model doesn't remember the previous turn, so the app sends the whole conversation every time. The Claude Code docs say it plainly: "Claude Code sends your full conversation with every request" ([Manage costs](https://code.claude.com/docs/en/costs)).

The author's measurement: 10 turns of a short conversation with Claude Haiku 5.5 via OpenRouter, reasoning off, `max_tokens` 120, the full history sent on every turn.

| Turn | Input tokens | Output tokens | Total billed so far |
|---|---|---|---|
| 1 | 20 | 91 | 111 |
| 2 | 127 | 120 | 358 |
| 3 | 262 | 120 | 740 |
| 4 | 399 | 120 | 1,259 |
| 5 | 535 | 65 | 1,859 |
| 6 | 616 | 120 | 2,595 |
| 7 | 754 | 94 | 3,443 |
| 8 | 858 | 120 | 4,421 |
| 9 | 994 | 120 | 5,535 |
| 10 | 1,134 | 95 | 6,764 |

Input on turn 10 is 57 times larger than on turn 1. Over 10 turns, 6,764 tokens were billed, while the questions and answers themselves add up to about 1,230. At this scale, the whole conversation cost $0.0011. This is one model and a short test conversation, but the mechanics are general. For an agent that feeds file contents and logs back into the window every turn, the same arithmetic runs on hundreds of thousands of tokens.

Prompt caching makes the repeated, unchanged prefix cheaper but doesn't remove it. A cache read costs 5% of the input price on Opus 5.5 and Sonnet 5.5, 2.5% on Fable 5.1 and 10% on Haiku 5.5 ([Pricing](https://platform.claude.com/docs/en/about-claude/pricing)). Cached tokens still take up space in the window.

### The cost of a single long request

Calculated from prices as of November 8, 2026, without caching: 50K, 150K and 300K input tokens plus 2,000 output tokens.

| Model | 50K | 150K | 300K |
|---|---|---|---|
| Claude Haiku 5.5 (100K threshold) | $0.006 | $0.080 | $0.155 |
| GPT-6.1 Terra (272K threshold) | $0.060 | $0.160 | $0.615 |
| Claude Sonnet 5.5 (no threshold) | $0.120 | $0.320 | $0.620 |

Thresholds apply to the whole request, not just the tokens above the threshold. On Haiku 5.5, a request with a prompt over 100,000 tokens pays the higher rate on all of it, and cached tokens count toward the prompt length. So a 100,000-token prompt costs $0.01 in input, and a 100,001-token prompt costs about $0.05. Going from 50K to 150K triples the tokens and raises the price 13x.

On GPT-6.1 Terra, a prompt over 272K input tokens is billed at 2x for input and cache and 1.5x for output "for the full request" ([GPT-6.1 Terra](https://developers.openai.com/api/docs/models/gpt-6.1-terra)). At 300K, Terra nearly catches up with Sonnet 5.5, even though it's half the price on short context. Prices change: Gemini 4 Argon's rate doubles on December 1, 2026, so check current prices before running your own numbers. General ways to cut spend are in the article on [LLM cost optimization](https://futurecraft.pro/blog/ai-cost-optimization/).

### Attention and memory

Inside the model, long context costs compute. In the classic transformer, a self-attention layer has O(n²·d) complexity, where n is the sequence length and d is the representation dimension ([Attention Is All You Need](https://arxiv.org/abs/1706.03762), Table 1): every token is compared with every other token. Modern models use optimizations such as sliding windows and hybrid attention, so growth is gentler in practice, but long context never becomes free.

The second cost is the KV cache: intermediate attention data kept in memory for every token in the context. You can see it clearly on local models. Per the measurement in the [local LLM guide](https://futurecraft.pro/blog/local-llm-guide/), the qwen3:8b KV cache takes 144 KiB per token. At a 32,768-token context that's 4.5 GiB, and in Ollama the model took 9.9 GB with a 5.2 GB weights file. On a local machine, a long window runs into memory limits before it reaches the model's stated maximum.

## How to keep context short

Short, clean context is cheaper and more reliable. The techniques below come from Anthropic and OpenAI documentation, and each has a cost.

| Technique | When to use it | What you pay |
|---|---|---|
| RAG: put only retrieved chunks in the window | the knowledge base is larger than the window or changes often | quality depends on retrieval ([RAG vs context engineering](https://futurecraft.pro/blog/context-engineering-vs-rag/)) |
| Compacting the history | a long chat or agent loop approaches the limit | details get lost, and compaction is itself a request |
| Clearing old tool results | the agent reads lots of files and logs | the prompt cache gets rebuilt |
| External memory | facts must survive the session | you need your own storage logic ([AI agent memory](https://futurecraft.pro/blog/ai-agent-memory/)) |
| Subagents | bulky output: tests, logs, code search | the subagent spends its own tokens ([Claude Code subagents](https://futurecraft.pro/blog/claude-code-subagents/)) |
| `/clear` between tasks | switching to unrelated work | the previous task's context leaves the session |
| Deferred tool loading, skills instead of a long CLAUDE.md | many MCP servers and instructions | some instructions load on demand ([MCP and context](https://futurecraft.pro/blog/mcp-for-ai-agents/)) |

Some of these are already built into the APIs. Anthropic offers server-side compaction in beta for Claude 4.6 and later: earlier parts of the conversation are summarized on the server, and the conversation can continue past the window limit. Context editing clears old tool results and thinking blocks. The memory tool runs on the client side: the model requests file operations in a `/memories` directory, and your code carries them out. For long-running agents, the documentation recommends combining the two: compaction keeps the active context small, and memory keeps what must not be lost in summarization ([Memory tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool)).

OpenAI's Responses API turns on server-side compaction through `context_management` with a `compact_threshold`. There's also a separate stateless `/responses/compact` endpoint. The result arrives as an encrypted compaction item ([Compaction](https://developers.openai.com/api/docs/guides/compaction)).

Claude Code gives you the same ideas without writing code. Per the documentation, MCP tool definitions are deferred by default, skills load only when invoked, and a hook can filter a log down to the error lines before Claude reads it ([Manage costs](https://code.claude.com/docs/en/costs)).

## How to count tokens before sending

Counting tokens up front is more reliable than learning about overflow from an error or a truncated answer.

Anthropic has a free `/v1/messages/count_tokens` endpoint for this. It accepts the same `model`, `system`, `messages` and `tools` as a regular request and returns `input_tokens`. Example from the documentation ([Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting)):

```bash
curl -s https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2026-10-21" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-haiku-5-5",
    "system": "You are a code reviewer",
    "messages": [{"role": "user", "content": "<history and new question>"}]
  }'
# {"input_tokens": ...}
```

The documentation notes that this is an estimate, and the actual count in the real request may differ slightly. Count for the model you'll actually call, because tokenizers from different generations give different numbers.

Other places to get numbers:

- the `usage` field in the API response: `input_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens` and `output_tokens` on Anthropic, `prompt_tokens` and `completion_tokens` on OpenAI-compatible APIs;
- `stop_reason` in an Anthropic response: `model_context_window_exceeded` means the answer hit the window limit;
- `/context` and `/usage` in Claude Code: what's in the window and how many tokens the session has used;
- `prompt_eval_count` and `eval_count` in an Ollama response: how many tokens the model actually read and generated.

## What's next

Quality isn't constant inside the window: the more tokens, the higher the chance the model misses the fact you need. That's a separate topic with its own measurements. For this article, one conclusion is enough: a full window doesn't guarantee the model uses everything in it.

Once you know how the window works, the next question is what to put in it: which sources, in what order, how to compact and how to test the result. That's covered in the [context engineering guide](https://futurecraft.pro/blog/context-engineering-guide/).

Where to start:

1. Measure how much your typical request takes: `count_tokens`, the `usage` field or `/context`.
2. Compare the result with the pricing thresholds: 100K on Haiku 5.5, 272K on GPT-6.
3. For local models, check `num_ctx` and compare `prompt_eval_count` with the prompt size.
4. Start a new session for each new task, and keep decisions in a file, not in the chat history.

## Sources

All pages accessed November 8, 2026.

- [Anthropic: Context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows)
- [Anthropic: Pricing](https://platform.claude.com/docs/en/about-claude/pricing)
- [Anthropic: Models overview](https://platform.claude.com/docs/en/models/overview)
- [Anthropic: Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting)
- [Anthropic: Memory tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool)
- [Claude Help Center: context window on paid plans](https://support.claude.com/en/articles/8606394-how-large-is-the-context-window-on-paid-claude-plans)
- [Claude Code: Manage costs effectively](https://code.claude.com/docs/en/costs)
- [OpenAI: Models](https://developers.openai.com/api/docs/models), [GPT-6.1 Terra](https://developers.openai.com/api/docs/models/gpt-6.1-terra), [Pricing](https://developers.openai.com/api/docs/pricing), [Compaction](https://developers.openai.com/api/docs/guides/compaction)
- [Google AI Studio: Gemini 4 Argon](https://aistudio.google.com/docs/models/gemini-4-argon), [Pricing](https://aistudio.google.com/docs/pricing)
- [DeepSeek: Models & Pricing](https://api-docs.deepseek.com/quick_start/pricing/)
- [Ollama: Context length](https://docs.ollama.com/context-length)
- [Liu et al. Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)
- [Vaswani et al. Attention Is All You Need](https://arxiv.org/abs/1706.03762)