How Reasoning Models Are Different
How "thinking" AI actually works
① A reasoning model didn't learn a new ability — it's a model trained (and prompted) to write out a long chain of intermediate steps before it gives an answer. That process gets called "thinking," but the mechanism is still next-token prediction.
② Why that lifts accuracy is simple — the intermediate steps the model emits feed back in as input and become material for the next prediction. It's the effect of breaking a hard problem into pieces.
③ It isn't free. The same question costs several times the tokens, runs slower, and costs more. For plain lookups and summaries, a regular model is better. Reasoning models earn their keep only on math, code, and multi-step problems.
① First, untangle the word "reasoning" — it has two meanings
In everyday use, one word covers two ideas that are worth separating.
- Inference — running an already-built model to produce an answer. This is the "inference" set against "pre-training" in how an LLM works, section ⑥. Every AI response comes out of this step.
- Reasoning — working through steps to reach an answer. The "reasoning model" in this piece means this one.
So "reasoning model," spelled out, is "a model that runs by inference, built to be good at reasoning." Product names vary by company (some ship a separate model line, some add it as a "thinking" or "extended thinking" mode on a regular model), so the name alone won't tell you. It's faster to tell them apart by how they work.
② How a regular model answers — a 30-second recap
As covered in how an LLM works, a regular model picks one next token by probability, appends it to the input, and recomputes from the start — over and over. It doesn't plan the whole sentence ahead. Every step, it just picks the next piece.
This structure has one weakness. On a hard multi-step problem it races toward the answer from the very first piece, so a small early slip compounds as it goes. For a person, it's like doing a long problem in your head with no scratch paper. Easy problems are fine; complex ones collapse when there's nowhere to hold the work in progress.
Telling a regular model to "think step by step" also raises its accuracy. Making it emit intermediate steps as tokens means those tokens feed back in as input, giving the next prediction more to work with. A reasoning model just does this habit every time, on its own, in a trained way.
③ What a reasoning model actually does — it writes the working out first
Before producing a final answer, a reasoning model first generates a long stretch of intermediate thinking. Roughly this flow:
- Restates the problem in its own words (pins down what's being asked)
- Breaks it into smaller sub-problems
- Tries candidate approaches and backtracks when stuck
- Checks its own partial results ("wait, this step looks off")
- Pulls it together into a final answer
The key is that all of those intermediate tokens accumulate in the context. So by the time it generates the final answer, the model is picking the next token while looking at a problem it has half-solved, not a blank one. No new thinking ability appeared — it's the same next-token prediction, done from a much better position.
How was the habit instilled? Take problems with known answers (math, code) and do extra training that rewards the reasoning paths that reach the correct answer. It's the same family of reinforcement learning as the RLHF in how AI is made, except the reward signal here is "was the answer right," not "did a human prefer it." As a result the model also learns, to some degree, when to think and how long.
The reasoning shown on screen (or its summary) is not proof that the model actually reached the answer by that logic. It may have written a plausible derivation and decided the answer another way, or fabricated steps as after-the-fact justification. Use the chain of thought as a lead for review, but don't conclude "the steps are visible, so it's trustworthy."
④ Why this lifts accuracy — and where it stops
The effect comes from three things.
| Mechanism | What happens |
|---|---|
| Decomposition | A problem it can't do in one shot is split into several predictions that are each easy |
| A self-correction window | Writing "this step is wrong" as a token makes it input too, fixing the next prediction |
| Math / logic help | Splitting each step into something simple eases the getting-numbers-wrong problem (doesn't fully solve it) |
The limits are just as clear. Thinking longer doesn't create knowledge that isn't there — facts it doesn't know still get plausibly invented (hallucination happens with reasoning models too). Long thinking can even build confidence in a wrong conclusion. And a one-step question ("what does this word mean") has nothing to decompose, so there's no gain.
⑤ The cost — tokens, time, price
The intermediate thinking is all generated tokens too. Even when it's hidden from the screen, it was computed, and it usually counts toward the bill.
| Item | Regular model | Reasoning model |
|---|---|---|
| Response speed | Fast (answers right away) | Slow (needs thinking time) |
| Token use / price | Baseline | Several to tens of times per answer |
| Good fit | Lookups, translation, summaries, drafts | Math, code debugging, multi-step planning, logic puzzles |
| Poor fit | Constraint-heavy optimization, proofs | Simple questions (over-thinking can make it worse) |
The rule of thumb reduces to one question: "Is this a problem I'd work out on paper, step by step?" If so, a reasoning model earns its cost. If it's something you'd answer off the top of your head, a regular model is faster and cheaper.
⑥ An investor's angle — why "reasoning" leads back to chips
A reasoning model spends several to tens of times more tokens per answer. That means inference (model-run) compute rises sharply. Here the two senses of "reasoning" meet — when models good at reasoning take off, inference load in the data center goes up.
That's the backdrop to the recurring story in recent AI-chip results that demand is shifting from "training" to "inference." Training and inference also stress different hardware traits (semiconductor sector deep dive). And this rise in compute is the real-world basis for the debate over AI capex size and AI circular financing.
⑦ In practice — which model, when
| Situation | Pick |
|---|---|
| Fact lookup, translation, summary, brainstorming, short drafts | Regular model (fast, cheap) |
| Math / stats problems, complex code debugging, multi-step schedule / budget planning | Reasoning model |
| Constraint-heavy design / optimization, writing where logical consistency matters | Reasoning model |
| High-volume repetitive work (classifying thousands of items, etc.) | Regular model (the price / speed gap compounds) |
"Think step by step" doesn't need to be added — it's already trained to, so it's redundant. What helps more is spelling out the decision criteria, the constraints, and the output format you want. For example: "List your assumptions first, mark the source of each figure you use per step, and give just the answer on one line at the end." You still need to verify the facts.
A reasoning model is one that writes out its own working before it answers. The mechanism is still next-token prediction, and those working-out tokens feed back in as input, which is why it handles hard problems well. In exchange it's slow and expensive — use it for problems you'd solve on paper, and use a regular model for search and summaries.
※ Written as of September 2026, describing general principles. How reasoning models are implemented, what they're named, and how they're priced vary by service and change often — check the current specs of a given product in its own documentation. This piece is not intended as a basis for investment decisions.
※ This guide is provided for general educational purposes and simplifies technical details for readability.
New guides, when they land
We publish AI literacy guides twice a week. Subscribe and the next one comes to you — free, unsubscribe anytime.