Skip to content
MaxoPerf MaxoPerf docs

Failure criteria pass/fail gates

Problem: Without explicit thresholds, a MaxoPerf run always ends with status Finished — you have to look at the charts and decide yourself whether it passed. When you automate tests (scheduled or CI-triggered), you need an objective, machine-readable signal. Failure criteria encode your SLOs directly into the test so the run verdict is automatic.

Test type: Any — failure criteria apply to all test types.

Prerequisites

Section titled “Prerequisites”
  • A MaxoPerf account and a test with at least one baseline run.
  • Your SLO thresholds: p95 latency target, error rate budget, and optionally throughput floor.

How failure criteria work

Section titled “How failure criteria work”

MaxoPerf evaluates failure criteria against the entire run (not per-second). When any criterion is violated, the run status changes from Finished to Failed. Multiple criteria are combined with AND — all must pass for the run to be considered Finished.

You can set criteria at:

  • Request label level — POST /checkout p95 < 300ms
  • Global level — overall error rate < 1 %
  • Throughput floor — RPS > 50 (useful to catch cases where the test itself did not exercise enough load)

Step by step in MaxoPerf

Section titled “Step by step in MaxoPerf”

1. Open the failure criteria section

Section titled “1. Open the failure criteria section”
  1. Open the test in the MaxoPerf console and switch to the Configuration tab.
  2. Scroll to the Failure criteria section.
  3. Click Add criterion.

2. Add a latency criterion

Section titled “2. Add a latency criterion”
  1. Select metric: p95 latency.
  2. Select label: All labels (or a specific request label for per-endpoint precision).
  3. Operator: greater than.
  4. Value: your SLO threshold in milliseconds — for example 500.
  5. Click Add.

3. Add an error-rate criterion

Section titled “3. Add an error-rate criterion”
  1. Click Add criterion again.
  2. Select metric: Error rate.
  3. Operator: greater than.
  4. Value: 1 (meaning 1 %).
  5. Click Add.

4. (Optional) Add a throughput floor

Section titled “4. (Optional) Add a throughput floor”

A throughput floor protects against false “pass” results when the test runner did not actually generate meaningful load (e.g. if a startup error limited traffic to 1 RPS):

  1. Click Add criterion.
  2. Select metric: Throughput (RPS).
  3. Operator: less than.
  4. Value: your expected minimum — for example 80 (80 % of target RPS).
  5. Click Add.

5. Save and run

Section titled “5. Save and run”
  1. Review the criteria list and click Save.
  2. Click Run now (or let a scheduled run fire).
  3. When the run finishes, the status badge reflects the verdict.

6. Inspect a failed run

Section titled “6. Inspect a failed run”
  1. Open a run with status Failed.
  2. The run overview shows a Failure criteria panel listing which criterion was violated and by how much.
  3. Use the metric charts to pinpoint when during the run the violation occurred.

Verify

Section titled “Verify”
  • A run that meets all criteria ends with status Finished (green badge).
  • A run that violates one or more criteria ends with status Failed (red badge).
  • The failure criteria panel on a failed run names the violated criterion and shows the measured value vs the threshold.
  • CI pipelines that poll the run status correctly read failed and exit non-zero — see CI-gated performance test.

Variations

Section titled “Variations”
  • Per-label criteria: apply tighter criteria to critical endpoints (POST /payment p95 < 200ms) and looser criteria to background endpoints (GET /analytics p95 < 2000ms).
  • Criteria-only failure, no threshold on VUs: leave VUs/throughput criteria off for exploratory runs where the goal is pure observation.
  • Failure without stop: by default a criterion violation marks the run failed but does not stop it early. Enable stop on first failure in the criteria settings if you want the run to terminate immediately.

Where to go next

Section titled “Where to go next”