TTFT (time to first token)

Time to first token (TTFT) is the time between sending a request to a large language model (LLM) and receiving the first token of its response. It measures how long a user waits before a streamed answer starts to appear. Most of it is the prefill phase, when the model processes the whole prompt before generating output.

How to calculate it

Record the timestamp when the request is sent and the timestamp when the first streamed token arrives, then subtract. Total response time is roughly TTFT plus output tokens divided by the generation rate. For example, a TTFT of 400 ms and 500 output tokens at 50 tokens per second gives about 10.4 seconds in total. Some instrumentation libraries emit it as gen_ai.server.time_to_first_token. Track it as a p95 per model.

Why it matters in production

TTFT is the first latency a chat user feels, separate from how long the full answer takes. Longer prompts and lost prompt cache hits push it up. A rising TTFT with flat output length points at prompt growth or a slower provider rather than your own code.

See also

• LLM observability

• KV cache

• Token (LLM)

• Percentile latency

• Latency and throughput

On the One2N blog

• Token economics, paisa, prompts & production

• AI agent observability: tool calls, turns and runaway loops

• OpenLLMetry vs OpenLIT vs Langfuse vs Phoenix for agents

References

1. OpenTelemetry GenAI semantic conventions