Little's law
Little's law says that the average number of items inside a stable system equals their average arrival rate multiplied by the average time each item spends there. Written L = λW, it ties together concurrency, throughput and latency. It applies to request handlers, thread pools, connection pools and any other queue.
How it works
Say a service receives 200 requests per second, and each takes 250 ms on average. Then L = 200 x 0.25 = 50 requests in flight at any moment.
If the service has 40 workers, 10 requests wait in line. Now a slow dependency raises latency to 1 second. At the same arrival rate, L = 200 x 1 = 200 requests in flight, four times the previous 50. The queue grows until clients time out.
You can also flip it around. A pool of 40 workers with 250 ms latency can sustain at most 40 / 0.25 = 160 requests per second.
The law holds for averages over a long run, with a stable system where arrivals roughly equal departures. It needs no assumption about how arrivals are spread.
Why it matters in production
It explains why a slow downstream service makes memory, threads and connections climb together. It also gives a quick check for pool sizes and autoscaling targets before a load test.
See also
On the One2N blog
Further reading
