Latency percentiles deep dive

Latency is not a single number. Every request your system serves takes a slightly different amount of time — some because of network jitter, some because of garbage collection pauses, some because of lock contention. A meaningful performance measurement captures this distribution, not just its center.

Why average latency misleads

Suppose your API processes 1 000 requests in a minute. 990 of them complete in 50 ms. The remaining 10 — triggered by a cache miss that causes a database round trip — take 5 000 ms each.

Metric	Value
Average	~98 ms
p50 (median)	50 ms
p95	50 ms
p99	5 000 ms

The average (98 ms) is misleading in two directions: it makes the fast requests look slower than they are, and it hides the 1% of requests that are 100× slower. A user who lands in that 1% — your worst-case experience — is invisible to the average.

The p99 (5 000 ms) surfaces exactly that worst-case experience. At 1 000 requests per minute, 1% means 10 users per minute experiencing a five-second wait.

Never use average latency as an SLO target

Average latency is easy to hit with a skewed distribution. A system can have a great average and terrible tail behavior. Define your SLO on p95 or p99 — whichever your users actually feel.

What each percentile tells you

Percentile	What it represents	When it matters
p50 (median)	Half of requests are faster, half are slower. The “typical” experience.	Good for understanding the center of the distribution.
p90	90% of requests complete within this time.	A common intermediate threshold.
p95	95% of requests complete within this time.	The standard SLO boundary for most web APIs.
p99	99% of requests complete within this time.	Critical for high-traffic APIs where 1% is still many users.
p99.9	99.9% of requests complete within this time.	Used in financial and safety-critical systems.

MaxoPerf reports p50, p90, p95, and p99 on the Overview tab for every run. The glossary entry for p95 and p99 latency has the formal definitions.

Reading the percentile chart in MaxoPerf

The latency chart on the run-detail Overview tab shows all four percentile lines over time. Here is how to read them:

Lines move together, similar gap throughout — the distribution shape is stable. The system responds consistently at all percentiles throughout the run. This is a healthy pattern.

Lines spread apart as the run progresses — tail latency (p99) is climbing faster than the median (p50). This often means a slow resource leak: a connection pool filling up, a memory cache hitting its limit, or a database acquiring more contention over time.

p99 spikes, p50 stays flat — isolated slow requests. Possible causes: GC pause, cache miss storm, or an async task completing slowly behind one request.

All lines climb together — the system is approaching its capacity ceiling. Every request is slowing, not just the outliers. Compare to the throughput chart: if throughput has plateaued, this is a clear capacity signal.

Percentiles in SLOs

A service level objective (SLO) for latency should name the percentile explicitly:

“p95 latency for the POST /v1/orders endpoint must be under 400 ms at 200 concurrent users.”

This is precise and testable. “Average latency under 200 ms” is neither — you can satisfy it with 990 fast requests and 10 very slow ones.

See Baselines, SLOs, and error budgets for how to turn percentile targets into MaxoPerf failure criteria.

Histogram vs percentiles

Some tools expose a latency histogram — the full distribution bucketed into ranges (e.g., 0–50 ms, 51–100 ms, 101–250 ms). A histogram gives the most complete picture but is harder to scan quickly. Percentiles are a summary of the histogram: p95 is the value at the 95th bucket.

MaxoPerf reports percentiles (the practical summary) rather than raw histograms. If you need the full histogram for detailed analysis, use the metrics export or a connected observability platform.

The impact of outliers on infrastructure planning

At low traffic (100 RPM), a p99 of 5 000 ms means about 1 slow request per minute — probably ignorable. At high traffic (10 000 RPM), the same p99 means 100 slow requests per minute, each tying up a connection or thread for five seconds. Thread pools, connection limits, and request queues all feel the tail, not just the median.

Plan your infrastructure around the percentiles that your actual traffic makes significant — not just what looks acceptable in a low-traffic smoke test.

Do / don’t

Do	Don’t
Set SLOs on p95 or p99 — whichever your infrastructure and user experience require.	Set SLOs on average latency.
Look at percentile spread (gap between p50 and p99) to detect tail issues.	Celebrate a good p50 without checking p99.
Monitor all four percentile lines over the run duration to detect latency drift.	Only check the end-of-run summary number — time-series reveals patterns the summary hides.
Cross-reference latency climb with throughput — if both move, it’s capacity; if only p99 moves, it’s tail behavior.	Treat all latency climbs the same way.

Where to go next

• p95 and p99 latency — formal definition and quick reference.
• Core metrics explained — the full set of metrics on the Overview tab.
• Baselines, SLOs, and error budgets — turning percentile targets into testable SLOs.
• Anatomy of a load test result — a tour of every panel in a MaxoPerf run-detail page.
• Read run results and logs — practical guidance on reading a finished run in MaxoPerf.