Context windows, explained for builders

The context window is the single concept that explains the most confusing model behavior — why it "forgets," why costs spike, why quality drops on long inputs. You don't need the math. You need a builder's mental model, and a few habits that follow from it.

Think of it as the model's desk, not its memory

Everything the model can use to answer — your instructions, the conversation so far, any documents you pasted in — has to fit on the desk at once. It has no memory of anything that isn't currently on the desk. When people say a model "forgot" what they told it earlier, what usually happened is the earlier message slid off the edge to make room.

Bigger isn't free

Two things get worse as you fill the window. Cost and latency rise with everything you put in — you pay for the whole desk every turn. And quality can quietly degrade: bury the one relevant fact in a hundred pages and the model may miss it. More context is a tool, not a default.

Spend the window deliberately

• Retrieve, don't dump. Pull in the few passages that matter instead of the whole document.
• Summarize the old. Compress long histories into a running summary so the desk stays clear.
• Cache the stable. Keep unchanging instructions in a form you can reuse to cut cost and latency.

Once you picture the desk, most "weird" model behavior stops being weird. You're not fighting a black box — you're managing a finite workspace, and the craft is deciding what deserves to be on it.