Should you be excited about the million tokens in the new Claude Haiku 5.5?

On October 7, Anthropic released Claude Haiku 5.5—the cheapest model in the company's history, featuring a one-million-token context window and a price drop of nearly ten times. The headlines were upbeat: "cheaper," "faster," "smarter." But if you break the numbers down, the picture is far less polished—and this is a good opportunity to have a calm discussion about what tokens, context windows, and agents actually mean technically, rather than just on marketing cards.
A token is not a word, and the rules can change without warning
A token is a chunk of text that a model breaks input into before processing, and the number of characters per token depends on the language and the specific model. In English, one token is roughly three to four characters; in Russian, it is significantly less, which means Russian-language texts usually consume more tokens for the same meaning.
With Haiku 5.5, there is a specific trap: the model has switched to a new tokenizer, and the same text is now broken down into more tokens than before. Independent tests show that the exact same text is counted differently. This is precisely why the advertised 90% price reduction translates to about 75% savings in practice—Anthropic itself admits this. The marketing figure on the cover and the actual bill at the end of the month are different things, and this will almost always be the case when comparing the prices of different model generations.
Context window: the advertised million vs. the million that actually works
A context window is the entire volume of text that a model keeps "in view" simultaneously: the prompt, the conversation history, and uploaded documents. For Haiku 5.5, it is indeed one million tokens, with a maximum response length of 128,000. It sounds like "you can upload an entire library and nothing will be forgotten." In practice, that is not the case.
The Chroma research team ran 18 leading models through long contexts and reached a unified result: each of them loses accuracy as the input text grows, and the decline is non-linear, depending on where the necessary information is located in the text. Models perform worst when the required fact is buried in the middle of a long document: accuracy drops by more than 30 percentage points compared to when the same information is at the beginning or the end.
This is not an abstract laboratory finding. Over the summer, users of OpenAI's Codex noticed that their long refactoring sessions suddenly started throwing context overflow errors—even though the model claimed a 1.05 million token window. An investigation revealed that the actual limit within the product had been quietly cut by nearly a third. The vendor called this a "fix" rather than a reduction—a phrasing that is telling in itself.
Why would a cheap model even need a huge window?
There is some common sense behind the large window in Haiku 5.5—just not the kind the headlines paint. The model is designed as a cheap sub-agent for a smarter Opus or Sonnet: the larger model builds a plan and formulates the task, while Haiku reads documents, summarizes context, classifies tickets, and returns a concise answer.
But even here, there is a catch worth knowing before building an agentic chain: handing off a task to a sub-agent and back costs money in itself, often more than the work itself. In one analysis of such a scheme, the handoff between the orchestrator and the sub-agent cost nearly five times more than the task itself. In other words, the architecture of "a smart model giving commands to cheap assistants" is not free by default—it requires careful design, not just a subscription to a model with a large window.
What to do about this in practice
A million tokens on a model's spec sheet is an upper limit, not a quality guarantee. Before building a workflow on a new model, it is worth checking three things:
- How many tokens the new tokenizer actually spends on your task—comparing prices based on old calculations can be misleading;
- How consistently the model finds the necessary information if it is not located at the beginning or end of a long text;
- How much it costs to hand off tasks between agents if you are building a chain rather than using the model directly.
Healthy skepticism here is not grumbling, but a working tool: figures in announcements should be verified against your own tasks rather than taken on faith.
If you want to compare how different models—from Claude to GPT and Gemini—behave on real long documents and work tasks without extra configuration, you can try them all in GEMERA AI within a single chat or via a Telegram bot, without the hassle of API keys and tariff thresholds.