# What is time to first token (TTFT)?

> Time to first token is the wait between sending a prompt and receiving the first generated token.

Source: https://www.honeyhive.ai/learn/time-to-first-token (updated 2026-11-04)

Time to first token (TTFT) is how long you wait before seeing the model's output: the time from submitting a request to receiving the first token ([NVIDIA](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html)). Anthropic's definition is the same idea, measured from when the prompt was sent, and it calls the metric particularly relevant when you stream ([Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency)).

It is one part of latency. Inter-token latency, throughput and end-to-end latency measure the rest.

## TTFT, inter-token latency, throughput and total latency

These four get mixed up. NVIDIA's [benchmarking docs](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html) define them in one place:

| Metric | What it measures |
|---|---|
| TTFT | Time from sending the query to the first received token |
| Inter-token latency (ITL), also called time per output token | Average time between consecutive tokens; NVIDIA's AIPerf excludes TTFT, though tools differ |
| End-to-end latency | Time from sending the query until the full response arrives: `TTFT + generation time` |
| Tokens per second (TPS) | Total output tokens across all simultaneous requests, divided by the time between the first request and the last response |

Two things follow. A model can have a fast first token and a slow stream, or the reverse, so one number never describes both. And the TPS in that definition is a system-wide throughput figure, not the speed one user sees. Per-user streaming speed is what inter-token latency describes.

## What goes into TTFT

NVIDIA [says](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html) TTFT generally includes request queuing time, prefill time and network latency. Prefill is the model processing your whole input before it generates anything, which is why longer prompts raise TTFT.

That split tells you where to look. If TTFT is high on short prompts, queuing or the network is the likelier cause. If it climbs with prompt length, prefill is.

## Why it matters

Anthropic ties TTFT to responsiveness: it matters most when you stream and want users to see output quickly ([Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency)). For a chat window, the first token is the moment the app stops looking frozen. For a background job or a batch pipeline nobody watches, end-to-end latency and [cost](/learn/llm-cost-optimization) matter more.

## Ways to lower it, and what they do not do

| Lever | Effect on TTFT | Source |
|---|---|---|
| Stream the response | Does not change generation time; the user sees output sooner | [Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency), [OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization) |
| Cache the prompt prefix | Anthropic: improved TTFT "for long documents"; pre-warming removes the cache-miss penalty on the first user request | [Claude docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) |
| Put dynamic content late in the prompt | Keeps the shared prefix long so requests are KV cache-friendly | [OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization) |
| Use a smaller or faster model | OpenAI: smaller models usually run faster and cheaper | [OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization) |
| Shorten the input | OpenAI: cutting 50% of the prompt may give only a 1 to 5% latency gain, mainly worth it for very large contexts | [OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization) |
| Generate fewer output tokens | Cuts total latency (OpenAI: 50% fewer output tokens may cut about 50% of latency), not the wait for the first token | [OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization) |

Prompt caching deserves a caution. Anthropic says a cache hit needs 100 percent identical prompt segments up to the marked block, and that caching has no effect on output token generation: the response is identical to what you would get without it ([Claude docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)). The latency gain Anthropic describes is on time to first token. Any change early in the prompt, such as a timestamp in the system prompt, breaks the match for everything after it. See [context window](/learn/context-window) for what counts as input.

Anthropic also advises building a prompt that performs well first and trimming for latency afterward, because cutting too early can hide what top quality looks like ([Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency)). The same applies to [prompt optimization](/learn/prompt-optimization): check output quality after each latency change.

## Where to start the timer

Where you measure changes the number. NVIDIA's [definition](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html) includes network latency, so a timer in the client reads higher than one inside the model server. Pick one point, write it down, and use it every time. Compare TTFT across prompt lengths and models only under the same conditions, since queuing depends on load.

## Track it in production

Latency problems in an LLM app often live in one step of a chain, not the model alone. HoneyHive's Timeline view draws each span as a bar sized by its duration, so the longest-running spans stand out when you diagnose a slow trace ([HoneyHive docs](https://docs.honeyhive.ai/v2/tracing/timeline-view.md)). The monitoring dashboard charts cost, latency and quality, including LLM call duration and session duration ([HoneyHive docs](https://docs.honeyhive.ai/v2/monitoring/overview.md)). As of November 2026 those docs describe span and call duration, and do not describe a built-in time-to-first-token measure, so if you need TTFT you would record it yourself in your application. For the wider picture see [LLM observability](/learn/llm-observability) and [AI agent monitoring](/learn/ai-agent-monitoring).

## FAQ

### What is time to first token?

It is how long you wait before seeing the model's output, from query submission to the first received token ([NVIDIA](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html)). Anthropic defines it as the time to generate the first token of the response from when the prompt was sent ([Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency)).

### What is the difference between TTFT and inter-token latency?

TTFT covers the wait for the first token. Inter-token latency is the average time between consecutive tokens after that, and NVIDIA's benchmark tool excludes TTFT from it ([NVIDIA](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html)).

### What does TTFT include?

NVIDIA says it generally includes request queuing time, prefill time and network latency, and that longer prompts raise it because the model processes the full input before generating ([NVIDIA](https://docs.nvidia.com/nim/benchmarking/llm/latest/metrics.html)).

### Does prompt caching reduce time to first token?

Often, for long prompts. Anthropic says you generally see improved time-to-first-token for long documents, and that the response is identical to what you would get without caching ([Claude docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)).

### Does streaming make a model faster?

It does not shorten generation, but the user sees output sooner. Anthropic says streaming can significantly improve perceived responsiveness ([Claude docs](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency)), and OpenAI says it cuts the waiting time to a second or less ([OpenAI](https://developers.openai.com/api/docs/guides/latency-optimization)).