Context window

What it means

Say you ask an AI assistant to read a 40-page contract and then answer questions about it. Everything the assistant can “see” at that moment has to fit in one space: your request, the contract, the conversation so far, and the answer it is writing. That space is the context window. Anthropic, the company that makes Claude, says it is “all the text a language model can reference when generating a response, including the response itself,” and that it “represents a “working memory” for the model.”¹ This is not the same as what the model learned in training. Training is what it knows in general. The window is what it is looking at right now.¹

The window is measured in tokens. Google explains that a context window “is made up of tokens, which are the building blocks used for processing information.”² A token is roughly a word or part of a word. When a conversation outgrows the window, something has to give. Anthropic's documentation says chat apps “can also manage the context window on a rolling “first in, first out” basis,” which means the oldest material drops away first.¹

Where the word comes from

We could not confirm who first used the phrase. The earliest use we verified in a landmark AI paper is OpenAI's 2020 paper introducing GPT-3, which says: “All models use a context window of n_ctx = 2048 tokens.”³ Windows have grown quickly since then. IBM Research wrote in 2024 that “When ChatGPT made its debut nearly two years ago, its window maxed out at 4,000 tokens.”⁴ Google's documentation says that “Earlier versions of generative models were only able to process 8,000 tokens at a time.”⁵ In February 2024, Google said of its Gemini 1.5 model: “We can now run up to 1 million tokens in production.”² Today Anthropic lists windows of “up to 1M tokens, depending on the model.”¹ As the window grew, the term changed from a narrow technical limit into a selling point and, for AI agents, a resource that has to be managed.

How others define it

• Anthropic: “The “context window” refers to all the text a language model can reference when generating a response, including the response itself.”¹
• IBM Research: “the maximum amount of text the model can consider as it chats with a customer, reviews a contract, or fixes a line of code.”⁴
• Google: “An analogy for the context window is short term memory.”⁵

Where they agree and differ

All three sources agree on the basics. The window limits what a model can consider at once, it is counted in tokens, and it is separate from what the model learned in training. They differ in emphasis. Google and IBM explain it with memory comparisons, while Anthropic stresses that the model's own answer takes up space too.¹⁴⁵

The strongest opposing view is that bigger is simply better. When Google announced Gemini 1.5, Sundar Pichai called its window “the longest context window of any large-scale foundation model yet.”² That view has real support. IBM says a larger window lets a model hold more text “in a kind of working memory.”⁴ A model that can read a whole contract at once does not have to work from pieces.

Research shows, though, that fitting more in does not mean using it well. A peer-reviewed study by Liu and colleagues at Stanford and other institutions found that model performance is “often highest when relevant information occurs at the beginning or end of the input context,” and drops when the key facts sit in the middle.⁶ Anthropic itself warns that “more context isn’t automatically better” and that “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.”¹ IBM adds that larger windows need more computing power, which slows responses and raises costs.⁴ We are fairly confident in our reading: bigger windows are real progress, but what goes into the window matters more than how much fits.

In the news

We found no recent story from a major outlet that we could verify. The most memorable real example is Google's Gemini 1.5 announcement on February 15, 2024. It described a million tokens as enough for “1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code or over 700,000 words.”² For agents, the concern has since moved from size to upkeep. In September 2025, Anthropic wrote that “An agent running in a loop generates more and more data that could be relevant for the next turn of inference, and this information must be cyclically refined,” and that context “must be treated as a finite resource with diminishing marginal returns.”⁷

What it means for your organization

An AI agent acts on what is in its window. Your policies, the task, and the agent's limits all have to be there each time it works. If a rule is not in the window, the agent cannot follow it. That is why it matters what feeds the window: the [knowledge core](/definitions/knowledge-core) an agent draws on should be accurate, current, and short enough to fit. When a vendor advertises window size, ask how well the model performs across the full window and what long requests cost.⁴⁶ When a long task gets summarized to make room, a [person in the middle](/definitions/a-person-in-the-middle) should be able to see what was dropped.

A question for your team: When our agents work on long tasks, what decides what stays in their window, and who checks what gets left out?