AI observability

AI observability is the ability to explain why an AI application behaved the way it did in production. It answers which step was slow, what the request cost, which model and prompt produced the answer, and whether the answer was any good. It uses traces, metrics and logs, plus evaluations of output quality.

AI observability is the whole practice. LLM observability covers only the model calls, and agent observability covers only what an agent does inside one request.

How it works

Teams usually observe five layers: HTTP and API, retrieval, the LLM (large language model) calls, the agent loop, and evaluation. The first four produce numbers you can alert on, such as token usage per request or tool call latency. The fifth scores the output itself. A response that invents a refund policy can return HTTP 200 in 1.1 seconds, so only a trace and an evaluation score show the prompt change behind it.

Why it matters in production

An AI application fails in ways a standard dashboard does not show. Wrong, empty or irrelevant answers keep latency and error rates green, and token costs grow with context size and agent turns. Without these signals, the first report of a regression comes from a user or an invoice.