Skip to content

Choosing safe test environments

import { Aside } from ‘@astrojs/starlight/components’;

Load testing is intentionally stressful. That is the point. But “intentionally stressful” in the wrong environment — or without proper authorization — can cause real outages, corrupt production data, send unwanted emails to real users, or trigger rate limits on third-party APIs you did not intend to hit. Choosing the right environment and getting proper authorization are the first decisions in every load test design.

The choice between staging and production depends on what you are trying to learn and what you can afford to risk.

FactorStagingProduction
Risk to real usersNone (no real traffic)High — load test traffic competes with real users
Data integritySafe — test data can be resetRisky — test data may pollute real records
External APIsUsually mocked or sandboxedReal — may trigger costs, emails, notifications
RealismLower — caches colder, data smallerHigher — real traffic patterns, full data volume
Authorization neededSimpler — usually just your teamRequires cross-team coordination and change control

Staging is the default environment for performance testing and is where MaxoPerf is designed to shine. Use staging when:

  • You are developing and iterating on test scripts.
  • You are running regression tests in CI.
  • You have not yet established a production performance baseline.
  • Any risk of service degradation is unacceptable.

The trade-off: staging results may not perfectly predict production behavior, especially when staging databases are much smaller (fewer cache misses, different query plans) or when staging hardware is significantly under-provisioned.

Production load testing is sometimes the only way to get accurate results — especially for systems where staging is too different from production to be predictive. Before testing in production:

  1. Notify all stakeholders — including on-call teams and the business (marketing, support).
  2. Schedule during a low-traffic window — nights or weekends.
  3. Start with a fraction of the target load — ramp slowly and monitor.
  4. Have an abort plan — know exactly how you will stop the MaxoPerf run if real users are affected.
  5. Isolate traffic if possible — use a separate subdomain, a canary target, or a shadow traffic pattern to avoid mixing test traffic with real user traffic in your logs and analytics.

Managed cloud locations vs private locations

Section titled “Managed cloud locations vs private locations”

MaxoPerf offers two types of runner locations:

  • Managed cloud locations — MaxoPerf’s own runner infrastructure in AWS/GCP regions. Traffic originates from known MaxoPerf IP ranges. Use for external-facing services. See the Private locations concept page for the full list of managed regions.
  • Private locations (Outpost) — your own runners, deployed in your infrastructure, behind your firewall. Traffic originates from inside your network. Use for internal services, behind-firewall targets, and VPN-only endpoints. See Private datacenter Outpost test for the setup recipe.

For internal APIs (accessible only from inside the corporate network or VPC), managed cloud locations cannot reach the target. You need private locations.

A subtle source of incidents: your application under load may call third-party APIs, send emails, charge payment methods, or trigger webhooks — and load testing amplifies all of these.

Before running a load test in any environment, audit your scenario for:

  • External API calls: payment processors, email providers, SMS gateways, analytics SDKs. These may have rate limits, per-request costs, or real-world side effects. Replace them with mocks or sandbox accounts in staging.
  • Email and notifications: a 1 000-VU test that each triggers a welcome email will attempt to send 1 000 emails. Make sure staging email senders are sandboxed.
  • Database writes: writes to a shared database can corrupt test data or affect other teams’ tests. Use isolated schemas or test-data namespaces.
  • Webhooks to external systems: if your system sends webhooks to CRM, ticketing, or ERP systems, your load test will trigger them in bulk.

Every MaxoPerf load test should be authorized at two levels:

  1. Technical authorization: you have confirmed that the target host (IP, domain, or internal endpoint) is one you are permitted to test. If the target is behind a WAF or a DDoS-protection service, allowlist MaxoPerf’s IP ranges for the test window to avoid false positives.

  2. Organizational authorization: you have notified the target team, your on-call rotation, and any stakeholders who would be paged or affected. A brief Slack message and a calendar invite is usually enough for a staging test; a formal change ticket is standard for production.

DoDon’t
Use staging as the default environment for all development-phase and CI performance tests.Assume staging results perfectly predict production — document the known differences.
Get explicit authorization from the system owner before running any load test.Run load tests at “low enough” VU counts without authorization, assuming small tests are harmless.
Audit the scenario for third-party calls, email sends, and payment events before running at scale.Run a load test that exercises payment flows without confirming sandbox mode is active.
Use private locations (Outpost) for internal or behind-firewall targets.Attempt to reach internal services from managed cloud locations — they will not be reachable.
Have an abort plan: know how to stop the MaxoPerf run quickly if something goes wrong.Start a 2-hour soak test in production without a monitoring dashboard and an on-call engineer watching.