Safe and ethical load testing

Load testing is one of the few development activities that can cause real, immediate harm to a production system — or to systems you do not own. This page explains the authorization requirements, blast-radius controls, and responsible practices that must be in place before you run any significant load test.

import { Aside } from ‘@astrojs/starlight/components’;

**Only run load tests against systems you are explicitly authorized to test.**

Running a load test against a system without authorization — even your own company’s production environment without SRE sign-off — is equivalent to launching a denial-of-service attack. It can cause real outages for real users, trigger incident responses, and in some jurisdictions may have legal consequences. Always obtain written authorization before testing any system you did not build and personally operate.

The authorization rule

Before running any load test, answer yes to all three of these questions:

1. Do you own or have explicit written permission to load-test this system? If the answer is “I think so” or “probably”, it is not yet a yes. Get explicit sign-off.

2. Have you notified everyone who will be affected by the test? This includes your SRE/ops team, the teams responsible for dependent services, and your on-call rotation. A load test looks identical to a DDoS attack in monitoring.

3. Do you have a runbook for stopping the test if it causes unintended impact? Know exactly which button in MaxoPerf stops the test and who to notify if you need to stop it immediately.

If you cannot answer yes to all three, do not run the test.

Choosing a safe test environment

The safest environment for load testing is a dedicated performance environment — a staging or pre-production environment that mirrors production infrastructure but carries no real user traffic and no real data.

Environment	Safety level	Notes
Dedicated performance environment	Highest	Best choice; no real user impact
Shared staging	Medium	Coordinate with other teams; no prod data
Production (authorized, off-peak)	Low	Only with SRE sign-off + maintenance window
Production (uncoordinated)	Never	Do not do this

When a dedicated environment is not available, see Choosing safe test environments in Foundations for a detailed guide on staging vs production trade-offs.

Blast radius controls

Even in a safe environment, a poorly designed load test can cascade to systems you did not intend to stress. Apply these controls before every significant test:

Limit the scope of your test

• Test only the specific endpoints and flows you need to validate. A broad “test everything” approach amplifies the blast radius unnecessarily.

• Use MaxoPerf failure criteria with conservative thresholds to auto-stop the test if error rates exceed a threshold early in the run:

  pass-criteria:
    - subject: http.failures
      condition: '>'
      value: 0.10   # stop if error rate exceeds 10 %
      stop-test: true

Control the ramp rate

• A sudden jump to peak VUs can cause a cascade failure before you have a chance to react. Always ramp up gradually (minimum 2–3 minutes) so you can observe and stop if the system starts to degrade earlier than expected.

Protect dependent and third-party services

• Mock or stub third-party services that would receive real traffic during the test. Your load test should not send 10,000 real emails, charge 10,000 real payment transactions, or trigger 10,000 real SMS messages.
• Use test payment credentials (Stripe test mode, PayPal sandbox, etc.) when payment flows are in scope.
• Disable real notification delivery during the test window — or use a dedicated test account that routes notifications to a sink.

Know your MaxoPerf account limits and runner capacity

• Check your MaxoPerf plan limits for maximum concurrent VUs and maximum concurrent runners before a large test. Exceeding them causes the test to error out rather than start cleanly.
• If you are running a multi-region test, confirm that the locations you select are within your plan.

Data safety during testing

• Never use real production data (PII) as test data. See Test data dos and don’ts.
• Never create real financial transactions, real orders, or real user accounts during a load test. Use test credentials and test data that map to no-op handlers in your system.
• Purge test-generated records from your database after the test if your environment is shared with other teams.

Stopping a test safely

MaxoPerf allows you to stop a running test at any time from the run-detail page or from the account-level active runs view. If you observe any of the following during a test, stop immediately and investigate before continuing:

• Error rate climbing above your agreed blast-radius threshold.
• Alerting firing in your production monitoring system (if testing in or near production).
• A third-party service showing unexpected load in their dashboard.
• Your system becoming unresponsive to health checks.

The stop action in MaxoPerf sends a graceful termination to all runners. Runners drain in-flight requests (up to a configured drain timeout) before stopping.

Responsible disclosure

If you discover a performance vulnerability during a load test — a crash at a specific load level, a memory leak that causes OOM kills, or an unauthenticated endpoint that can be saturated trivially — follow your organization’s responsible disclosure process before publishing or sharing the findings publicly.

Checklist before testing a system you do not own

• Written authorization from the system owner is on file.
• SRE / operations team notified with test window and expected load.
• On-call rotation informed and available during the test window.
• Third-party services mocked, stubbed, or using test credentials.
• Failure criteria configured to auto-stop on early error-rate breach.
• Runbook for emergency stop is documented and distributed.
• Post-test cleanup plan for test-generated records is agreed.

Where to go next

• Choosing safe test environments — staging vs prod
• Pre-flight checklist — comprehensive run checklist
• Common pitfalls and anti-patterns — anti-pattern #1 in detail
• Failure criteria pass/fail gates — auto-stop thresholds