Distributed tracing

Distributed tracing is a way to follow one request as it moves through many services. Each unit of work is recorded as a span with a start time, a duration and attributes. All spans for the request share a trace ID, so they can be assembled into one tree that shows where the time went and where errors happened.

How it works

When a request enters the system, the first service starts a trace and a root span. It passes the trace context to downstream calls, usually in the W3C traceparent HTTP header. Each service creates child spans that point to their parent. For example, a checkout request takes 800 ms. The trace shows 600 ms in the payment service, and 550 ms of that in one database query. The slow step is visible without searching logs on each host.

Why it matters in production

Metrics show that latency rose. Traces show which service and which call caused it. In microservices, that cuts the time to isolate a fault. Context propagation must work end to end, or traces break into fragments.

See also

• OpenTelemetry

• Trace sampling

• Observability

• Agent observability

On the One2N blog

• Building observable Go APIs with OpenTelemetry & Prometheus

• The gotchas of OTEL collector processors for effective observability in K8s

• AI agent observability: tool calls, turns and runaway loops

References

1. Origin of the term

2. Traces (OpenTelemetry documentation)

3. W3C Trace Context specification