Token (LLM)

A token is the basic unit of text a large language model (LLM) processes. A tokenizer splits input into tokens, often whole words, parts of words or punctuation, and maps each one to a number. The model reads input tokens and generates output tokens one at a time. Context limits, latency and API pricing are all counted in tokens.

How to calculate it

Most modern tokenizers use byte pair encoding (BPE), which merges frequent character sequences into single tokens. A common rule of thumb for English is about 4 characters, or three quarters of a word, per token. A 1,000-word prompt is therefore roughly 1,300 tokens. At an example price of $3 per million input tokens, that prompt costs about $0.004. Output tokens are usually priced higher than input tokens.

Why it matters in production

Token counts set both the bill and the response time. Input tokens are processed in a prefill phase, while output tokens are generated one by one in a decode phase, so long answers add latency. Tracking input, output and cached tokens per request shows where cost comes from, and prompt caching can cut it.

See also

• Time to first token

• KV cache

• AI gateway

• Embeddings

• LLMOps

On the One2N blog

• Token economics, paisa, prompts and production

• Reduce LLM costs with Redis (hands-on workshop)

• Exploring retrieval augmented generation (RAG): enhancing LLMs with external knowledge

• AI observability guide

References

1. Origin of the term

2. Neural machine translation of rare words with subword units (Sennrich et al., 2016)