Context window
A context window is the maximum number of tokens a large language model can process in a single request. It covers everything the model sees: the system prompt, conversation history, retrieved documents, tool results and the answer it writes. Anything outside the window does not exist for the model on that request.
How it works
The window is a budget shared by input and output. A model with a 128,000-token window that receives a 2,000-token system prompt, 100,000 tokens of documents and 20,000 tokens of history has 6,000 tokens left for its answer. As a rough rule, one English token is about three quarters of a word, so 128,000 tokens is about 96,000 words. If a request goes over, the API rejects it, or the application must drop or summarise content first.
Why it matters in production
Filling the window costs money, since every input token is billed and adds prefill time. Models also use information in the middle of a long context less reliably than information at the start or end. That is why retrieval augmented generation sends a few relevant chunks instead of whole document stores.
See also
On the One2N blog
Further reading
