Why the phrase "think step by step" is no longer a prompt superpower

You have likely seen advice along the lines of, "add the phrase 'think step by step' to your prompt, and your answer accuracy will skyrocket by double digits." This tip has been circulating through guides and Telegram channels for years, sounding like an immutable law of physics. The problem is that this law is long outdated, and the myth of the magic phrase lives a life of its own, disconnected from the reality of modern models.
Where the myth of the magic phrase came from
The story begins in 2022, when Google researchers published a paper on chain-of-thought prompting. It was indeed an impressive leap, but it was this specific breakthrough that turned into an internet legend—a generalized figure of "plus 61% accuracy on everything."
In reality, it was much more specific. It referred to one particular test, on one particular model, under one specific set of conditions. Since then, this figure has been stripped of its context and slapped onto every prompt imaginable—from writing copy to analyzing spreadsheets.
Why the technique works differently in new models
The main reason the old "think step by step" trick doesn't have the same effect today is that the architecture of the models themselves has changed. The model is already thinking step-by-step internally before you even ask it to do so in text.
Because of this, a manual command often becomes redundant. Moreover, there is a flip side: the original technique had its own limitations from the start. Even in 2022, it wasn't a universal "make it smarter" button, but rather an effect that manifested under specific conditions.
There is also a very tangible financial side to this. Every step of reasoning that the model writes out in text consumes additional tokens that you pay for. Add to that the fact that modern models also calculate hidden reasoning tokens that you don't even see, and you end up with a situation where blindly copying old advice can simply inflate your bill without any benefit to quality.
When the old trick still helps
It is too early to write off chain-of-thought prompting entirely—the scope of its usefulness has simply narrowed. Here is where the phrase "think step by step" still genuinely helps:
- Small or medium-sized models that lack a built-in reasoning mode—here, an explicit request to show the reasoning process still improves accuracy.
- Models without specific reasoning fine-tuning—on these, the trigger phrase still works much like it used to.
- Few-shot formats, where you provide the model with several examples that include pre-written reasoning chains.
However, with modern flagship models, the logic is different.
What to do instead of using a magic phrase
If you are working with a modern large model, it is better to spend your energy on other things rather than the "think step by step" incantation:
- Give the model clear context for the task and criteria for a successful result—this has a greater impact than a call to reason.
- If the model supports adjusting reasoning depth (reasoning effort or budget), manage that parameter directly rather than relying on prompt text.
- For truly complex, multi-step tasks, use few-shot prompting with ready-made reasoning examples—this remains a reliable method.
- Don't ask the model to "show its thought process" just for the sake of it—it wastes tokens and time if the task doesn't require transparent reasoning.
The myth isn't that chain-of-thought is useless; it's that a universal magic phrase has long since ceased to be universal. The technology has moved forward, but the advice has stayed the same.
If you want to see for yourself how different models react to these techniques, GPT, Claude, and Gemini are already waiting for you in GEMERA AI, via our Telegram bot and web dashboard. You can quickly test both old and new prompting techniques without extra settings or switching between services.