Queueing theory

Queueing theory is the mathematical study of waiting lines. It models how work arrives, waits and gets served. Reliability engineers use it to predict how latency changes with load. Its best known result, Little's Law, says the average number of items in a system equals the arrival rate multiplied by the average time each item spends there.

How it works

Utilisation is arrival rate divided by service capacity. In the simplest single-server model, the average time in the system grows in proportion to 1 / (1 - utilisation). If one request takes 10 ms to serve:

At 50% utilisation, a request spends about 20 ms in the system.
At 90%, about 100 ms.
At 95%, about 200 ms.

The curve is flat at low load and rises steeply near full capacity. Real systems differ in detail, but the shape holds.

Why it matters in production

Queueing theory explains why a service that looked fine at 70% CPU falls over at 95%. It gives teams evidence for keeping headroom, for limiting queue sizes, and for shedding load early instead of letting every user wait.