# What is chain-of-thought prompting?

> Chain-of-thought prompting gets a model to write out intermediate reasoning steps before its answer.

Source: https://www.honeyhive.ai/learn/chain-of-thought-prompting (updated 2026-11-01)

Chain-of-thought (CoT) prompting means getting a language model to write out intermediate reasoning steps before it gives a final answer. The [paper that named it](https://arxiv.org/abs/2201.11903) (Wei et al., January 2022) did so by putting a few exemplars with worked reasoning in the prompt. Google Research describes the idea as letting [models "decompose multi-step problems into intermediate steps"](https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/). Nothing is retrained. It is a technique of [prompt engineering](/learn/what-is-prompt-engineering), and the few-shot form builds on [few-shot prompting](/learn/few-shot-prompting).

Its role has changed since 2022. Today's reasoning models think before they answer, so providers now advise asking for steps only in some cases.

## The original research

- **Few-shot CoT.** Wei et al. report gains on arithmetic, commonsense and symbolic reasoning across three models. A [540B-parameter model with eight exemplars reached state of the art accuracy on GSM8K](https://arxiv.org/abs/2201.11903), beating finetuned GPT-3 with a verifier. Google's write-up gives [58% against the prior 55%](https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/), and notes that the authors wrote the chains by hand and that the GSM8K runs used an external calculator.
- **Scale matters.** Per the same post, the [benefits "only materialize with a sufficient number of model parameters (around 100B)"](https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/). On CommonsenseQA, StrategyQA and date understanding, CoT gave ["additional small improvements"](https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/), while sports understanding gained the most.
- **Zero-shot CoT.** Kojima et al. (May 2022) found that [adding "Let's think step by step" before each answer](https://arxiv.org/abs/2205.11916) lifted InstructGPT (text-davinci-002) from 17.7% to 78.7% on MultiArith and from 10.4% to 40.7% on GSM8K.
- **Self-consistency.** Wang et al. (2022) [sample a diverse set of reasoning paths](https://arxiv.org/abs/2203.11171) instead of taking the greedy one, then pick the most consistent answer. They report gains over plain CoT of 17.9% on GSM8K, 11.0% on SVAMP and 3.9% on ARC-challenge.

These results come from 2022-era models. They show why the technique took off, not what it adds on a current model. For how it fits among other techniques, see [prompt engineering techniques](/learn/prompt-engineering-techniques).

## What providers say now

| | Anthropic | OpenAI | Google (Gemini) |
|---|---|---|---|
| Stance | [Manual CoT is a fallback for when thinking is off](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) | For reasoning models, ["think step by step" or "explain your reasoning" is unnecessary](https://developers.openai.com/api/docs/guides/reasoning-best-practices) | [Generally not necessary to detail reasoning steps in the response](https://ai.google.dev/gemini-api/docs/prompting-strategies) |
| What to do instead | On Claude Opus 5.5, Sonnet 5.5 and Fable 5.1, [rely on thinking, at a lower effort level if cost or latency matter](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices); a prompt asking them to write out their reasoning may be declined | [Reasoning models "perform best with straightforward prompts"](https://developers.openai.com/api/docs/guides/reasoning-best-practices) | For harder problems, [ask it to "Think very hard before answering"](https://ai.google.dev/gemini-api/docs/prompting-strategies), which uses more thinking tokens |
| Multi-step tasks | [Prompt chaining when you need to inspect intermediate outputs](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) | | [Make each step a prompt and chain them](https://ai.google.dev/gemini-api/docs/prompting-strategies) |

Anthropic's fallback advice is to ask Claude to [think through the problem before it answers and put the final answer in `<answer>` tags](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices) so you can extract it. Anthropic notes that Claude 4.6 and later use [adaptive thinking, where Claude "dynamically determines when and how much to think"](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices).

## A written chain is not proof of the reasoning

A visible chain is easy to read as an explanation. Lanham et al. (2023) intervened on chains, for example by adding mistakes, and found that [reliance on the chain varied by task](https://arxiv.org/abs/2307.13702). They also report that ["as models become larger and more capable, they produce less faithful reasoning on most tasks we study"](https://arxiv.org/abs/2307.13702), while CoT "can be faithful if the circumstances such as the model size and task are carefully chosen". Treat the text as a useful trace, not a guarantee.

## When to use it

1. On a reasoning model or with thinking on, start without a CoT instruction, as the providers above advise.
2. On a model with no built-in thinking, or with thinking off, try a step-by-step instruction or few-shot examples with worked reasoning.
3. If you need to read or check intermediate results, split the task into chained prompts.
4. Compare each variant on the same cases. A step-by-step instruction adds output tokens, so check cost and latency as well as accuracy. [LLM evaluation](/learn/llm-evaluation) covers how to grade outputs.

## Comparing CoT prompts in HoneyHive

A HoneyHive [experiment](https://docs.honeyhive.ai/v2/evaluation/introduction) runs a function on a dataset with `evaluate()` and scores each output with evaluators. To test CoT, run one experiment per prompt variant (with and without the step-by-step instruction) on the same dataset and compare the aggregated metrics.

## FAQ

### What is chain-of-thought prompting in simple terms?

It is a prompt that leads the model to write out intermediate reasoning steps before the final answer. The [original paper](https://arxiv.org/abs/2201.11903) did this with a few exemplars that show "a series of intermediate reasoning steps".

### What does "Let's think step by step" do?

It is the zero-shot version. Kojima et al. showed that [adding the phrase before each answer](https://arxiv.org/abs/2205.11916) made models "decent zero-shot reasoners", with no examples needed.

### Do reasoning models still need chain-of-thought prompts?

Usually not. OpenAI's guide [calls](https://developers.openai.com/api/docs/guides/reasoning-best-practices) prompting reasoning models to "think step by step" unnecessary, and Google's [says](https://ai.google.dev/gemini-api/docs/prompting-strategies) it is generally not necessary to have the model detail reasoning steps in the response.

### Is a model's written chain of thought its real reasoning?

Not always. Anthropic researchers found it ["unclear if the stated reasoning is a faithful explanation of the model's actual reasoning"](https://arxiv.org/abs/2307.13702), and the answer varied by task and model size.