Results analysis and reporting

Use case

Load test results are only valuable if the right people can read them, act on them, and refer back to them. A test that produces a wall of percentiles in a format no one outside the SRE team understands is not a release-decision tool — it is a box-ticking exercise.

Results analysis and reporting bridges the gap between raw metrics and decisions. The goal is to answer three questions quickly: did we pass the thresholds we agreed on? Which specific part of the system is responsible for any breach? What changed since the last run?

These questions need answers at two different time horizons: immediately after a run (release-candidate decision) and weeks or months later (incident retrospective, regression bisection, capacity planning). Good result storage serves both.

How MaxoPerf supports it

Every MaxoPerf run produces a structured result that includes: overall pass/fail based on configured thresholds; per-label latency percentiles (p50, p95, p99), error rate, and throughput; runner-health metrics (CPU, memory) for each active runner; request logs with timestamps; and a human-readable summary.

The per-label breakdown is the most actionable view. When a run fails on p95, the label breakdown tells you whether every endpoint degraded proportionally (suggesting infrastructure-level saturation) or one endpoint degraded while others were unaffected (suggesting a code-level regression in a specific path).

The compare view lets you place two runs side by side — typically the current release candidate and the previous production baseline — and see the delta for every label and metric. A regression shows up as a red delta; an improvement shows as green. The comparison is exportable for stakeholders who review releases outside the console.

Run history is permanent (within your retention policy). Because runs are stored with the test file and configuration that produced them, you can reproduce any historical run exactly, not just observe its result.

A concrete reporting workflow

A SaaS company’s engineering manager reviews performance results as part of every release-candidate approval:

• Run tagging: every CI-triggered run is tagged with env:staging, branch:<name>, and version:<semver>
• Comparison: the release manager compares the RC run against the previous production-equivalent run using the compare view; regressions > 10% on any label surface automatically
• Threshold summary: the pass/fail summary is linked in the pull request description (run ID → direct link to MaxoPerf result)
• Stakeholder report: for significant releases the engineering manager exports the per-label breakdown as a PDF and attaches it to the change-management ticket
• Retrospective reference: after the last incident (a 3× latency spike on the search endpoint) the team used run history to identify that the regression had been present but below threshold for three releases — the label breakdown showed it; the aggregate p95 had not crossed the alert threshold

Practical rollout

1. Tag every run — add environment, version, and branch tags from the start. Tags are cheap; searching through untagged runs is expensive.
2. Set and record thresholds explicitly — a run without configured thresholds cannot produce a pass/fail. Configure thresholds in the test definition, not in a separate document that drifts from the actual run.
3. Establish a baseline run — before the first gate, run the test manually and save the result as a named baseline. The first comparison has to compare against something.
4. Wire the result link into your release process — add the MaxoPerf run ID or result link to your PR template, deployment ticket, or release checklist. One extra line in a template creates a permanent audit trail.
5. Review the label breakdown, not just the summary — train the team to open the label view before deciding a run “passed.” The summary can be green while one label is quietly approaching a threshold.