ScribeOS Press
Context Windows and Cost
I. Penrose
ML
Context Windows and Cost
Attention, caching, and the bill
I. Penrose
A practical monograph on the economics of context: what to put in a prompt, what to retrieve, what to cache, and how a generous window becomes a tax on every call.
Opening
Tokens are rent
A context window is not free space. It is rent, paid on every call, including the ones that fail. Teams discover this after they paste the style guide, the schema, three few-shot examples, and yesterday's thread into a system prompt that now costs more than the generation.
The first editorial act is deletion. What does the model already know? What can a tool fetch? What can a cache hold so the prefix is stable and the bill drops? Prefix stability is not a micro-optimization. It is how you make a product affordable enough to leave on.