Services

Resources

Company

/

Load testing

Load testing

Load testing means sending traffic you control at a system and measuring how it responds. The usual question is whether the service holds its SLO at the busiest traffic you expect. The same tooling answers a few related questions, and each kind of test is named after one:

  • A load test holds expected peak, for example 450 requests a second for 20 minutes.

  • A stress test keeps ramping past peak until something breaks, so you learn the limit and how the service fails at it: fast 503s, slow timeouts, or a crash.

  • A soak test runs normal traffic for 8 hours or more to catch slow leaks such as memory growth, connections that never close or a disk filling with logs.

  • A spike test jumps from idle to peak in seconds to see whether autoscaling keeps up.

How it works

How the tool generates traffic matters more than which tool you pick. In a closed model, a fixed number of virtual users each send a request, wait for the answer, then send the next. If 100 users get answers in 200 ms, that is 500 requests a second. When the service slows to 1 second, the same 100 users can only send 100 a second, so the test quietly backs off at the moment real users would not. That is Little's law: throughput equals concurrency divided by response time.

In an open model, requests arrive at a set rate whatever the response time, the way users on the internet do. When the service cannot keep up, a queue builds. To test a public API, use an open model. In k6 that is an arrival-rate executor:

import http from 'k6/http';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate', // open model
      startRate: 50,
      timeUnit: '1s',
      preAllocatedVUs: 600,
      stages: [
        { target: 450, duration: '5m' },  // ramp to peak
        { target: 450, duration: '20m' }, // hold peak
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<300'],
    http_req_failed: ['rate<0.001'],
  },
};

export default function () {
  http.post('https://staging.example.com/checkout', '{"cart":"c-123"}');
}
import http from 'k6/http';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate', // open model
      startRate: 50,
      timeUnit: '1s',
      preAllocatedVUs: 600,
      stages: [
        { target: 450, duration: '5m' },  // ramp to peak
        { target: 450, duration: '20m' }, // hold peak
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<300'],
    http_req_failed: ['rate<0.001'],
  },
};

export default function () {
  http.post('https://staging.example.com/checkout', '{"cart":"c-123"}');
}
import http from 'k6/http';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate', // open model
      startRate: 50,
      timeUnit: '1s',
      preAllocatedVUs: 600,
      stages: [
        { target: 450, duration: '5m' },  // ramp to peak
        { target: 450, duration: '20m' }, // hold peak
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<300'],
    http_req_failed: ['rate<0.001'],
  },
};

export default function () {
  http.post('https://staging.example.com/checkout', '{"cart":"c-123"}');
}
import http from 'k6/http';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate', // open model
      startRate: 50,
      timeUnit: '1s',
      preAllocatedVUs: 600,
      stages: [
        { target: 450, duration: '5m' },  // ramp to peak
        { target: 450, duration: '20m' }, // hold peak
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<300'],
    http_req_failed: ['rate<0.001'],
  },
};

export default function () {
  http.post('https://staging.example.com/checkout', '{"cart":"c-123"}');
}
import http from 'k6/http';

export const options = {
  scenarios: {
    checkout_peak: {
      executor: 'ramping-arrival-rate', // open model
      startRate: 50,
      timeUnit: '1s',
      preAllocatedVUs: 600,
      stages: [
        { target: 450, duration: '5m' },  // ramp to peak
        { target: 450, duration: '20m' }, // hold peak
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<300'],
    http_req_failed: ['rate<0.001'],
  },
};

export default function () {
  http.post('https://staging.example.com/checkout', '{"cart":"c-123"}');
}

Make the traffic look like production too. Use the real mix of endpoints: if checkouts are 10% of requests, test them at 10%. Use real data, so that not every request hits the same cached product, and an environment sized like production.

Take the checkout service from the root cause analysis example. Six pods with 20 database connections each give 120 connections. A new query holds a connection for 400 ms, so the pool can finish at most 120 / 0.4 = 300 checkouts a second. A load test at 100 a second uses 40 connections and every graph looks fine.

Real peak is 450 a second, which would need 180 connections. Every second, 150 more checkouts arrive than the pool can finish, so the wait for a connection keeps growing. A closed test with 100 users never gets near 450. The open test above does, and its p99 threshold fails as the ramp passes 300 a second, about three minutes in.

Why it matters in production

Results lie in a few predictable ways. Read p99 over the steady part of the run only, since the ramp and warm-up drag it around. Read it per endpoint too, because a fast health check can hide a slow checkout. Count errors as failures, since a 500 that returns in 5 ms makes latency look better. Check that the achieved rate matched the target (k6 reports dropped iterations) and that the load generator itself was not out of CPU.

Then compare against the SLO: if the target is p99 under 300 ms at 450 a second and the test shows 280 ms at 300 a second, you have not tested the target.

One2N hit this on an AI chatbot running on AWS ECS. Load tests in a preview environment showed each task, with 2 vCPU and 4 GB of RAM, hitting its limits after 10 to 15 chat rooms. New tasks took 200 to 300 seconds to start, because the 6 GB image downloaded models on boot. That was found before launch, while there was still time to change how the bot ran. For the numbers behind queues and saturation, see latency and throughput.

See also

On the One2N blog

References

  1. Grafana k6 docs: Open and closed models

  2. Google SRE book: Addressing cascading failures

Line chart for a load test: connections in use equal requests per second times 0.4 s. A dashed line marks the pool limit, 6 pods x 20 = 120 connections. A test at 100 requests a second uses 40 connections and looks fine. The line crosses the limit at 300 a second, then turns red up to the real peak of 450 a second, which needs 180 connections, so the queue grows and p99 fails.