Context Window Calculator

The context window is a budget, not a document size limit

The number a provider advertises — 128,000 tokens, 200,000, a million — is the total for everything the model handles in one request. Four things compete for it, and people routinely account for only one.

• The system prompt. Instructions, persona, formatting rules, examples. Often one to three thousand tokens in a mature product, and resent on every call.
• Conversation history. In a chat, every previous turn. This grows without limit until something truncates it.
• Your document or retrieved context. The part people actually think about.
• The response. On most models, generation draws from the same pool. If the input fills the window, there is nowhere for the answer to go.

That last point causes the most confusing failures. A request that fits perfectly can still fail, or return a truncated answer mid-sentence, because no room was reserved for output. As a working rule, keep total input below about seventy-five per cent of the window and leave the rest for the response.

Fitting is not the same as being used well

A million-token window does not mean you should fill it. Retrieval quality degrades as context grows — the well-documented "lost in the middle" effect, where models attend reliably to material at the start and end of a long context and less reliably to what sits in between. A fact buried at the sixty per cent mark of a 400,000-token prompt is measurably less likely to be used than the same fact in a 4,000-token prompt.

Cost and latency scale with input too. Sending an entire codebase because it fits is usually worse on every axis than retrieving the six relevant files: cheaper, faster, and more accurate.

The practical guidance is to use large windows for problems that genuinely need whole- document reasoning — analysing a contract end to end, tracing a change through a long transcript — and to use retrieval for everything else. If you find yourself filling a large window routinely, the retrieval step is usually the thing worth improving.

Chunking when the document does not fit

When a document exceeds the window, it has to be split. The calculator shows how many chunks are needed for a given chunk size and overlap, which is the first thing to know when sizing a pipeline.

Overlap exists because a naive split cuts sentences, paragraphs and arguments in half. Repeating a portion of the previous chunk at the start of the next gives the model enough context to make sense of material near the boundary. Ten to fifteen per cent of the chunk size is a common starting point — enough to preserve continuity without inflating cost much. Note that overlap increases the total token count you pay for: at fifteen per cent overlap you process roughly fifteen per cent more tokens than the document contains.

Where you split matters more than how big the chunks are. Splitting on paragraph or section boundaries produces chunks that each make sense alone; splitting on a fixed character count produces chunks that begin mid-sentence. If your source has structure — headings, sections, function definitions — split on it.

How to use the Context Window Calculator

1. Paste your document

   Or enter a token count directly if you already know it. The token counter gives you that figure for any text.

2. Set your overheads

   Enter the size of your system prompt and any conversation history, and how many tokens you want to reserve for the response.

3. Check which models fit

   Each model shows the full budget broken down, and flags when the reserved response space no longer fits.

4. If it does not fit, size the chunks

   Set a chunk size and overlap to see how many chunks the document needs and how many extra tokens the overlap costs.

Frequently asked questions