CDN testing do and don't
CDN load testing has failure modes that do not appear in typical origin load testing. A misconfigured test can bypass the cache entirely, give you misleading results, or send traffic that violates your CDN provider’s terms of service. This page collects the rules that matter most for CDN testing.
Always warm the cache before measuring
Section titled “Always warm the cache before measuring”Do: Run a low-concurrency warm-up pass before your measurement phase. CDN edge nodes start cold. They have no cached copy of your assets until the first request arrives. A cold-cache run measures origin performance, not CDN performance.
Don’t: Start a high-VU load test immediately against a CDN URL you have never tested. Cache misses dominate the first few minutes of every CDN test, so those minutes do not show steady-state edge performance.
A two-phase MaxoPerf test handles this automatically: configure a concurrency: 5, hold-for: 3m warm phase before the main load phase.
Test with realistic URL distributions
Section titled “Test with realistic URL distributions”Do: Use a representative sample of URLs from your actual traffic mix. CDN performance varies a lot across asset types (JS bundles, images, API responses) and URL patterns. If your production traffic hits 1,000 distinct image URLs, a test hitting only 3 URLs will show an unrealistically high hit ratio.
Don’t: Test with a single URL repeated millions of times. That measures CDN edge performance for one warm object, not your CDN under real workload.
Assert on cache state, not just status codes
Section titled “Assert on cache state, not just status codes”Do: Include assertions on X-Cache, Age, Cache-Control, or your CDN’s equivalent header in every CDN load test. A 200 OK response tells you the request succeeded. It does not tell you whether the cache or the origin served it. Without cache-state assertions, a warm-cache test and a cold-cache test look identical in the MaxoPerf results overview.
Don’t: Rely on status code assertions alone to validate CDN behaviour. An uncached origin response returns the same 200 as a cached edge response.
Do not bypass the cache to measure CDN performance
Section titled “Do not bypass the cache to measure CDN performance”Do: Test with realistic URLs as your users would generate them.
Don’t: Add cache-busting query parameters (e.g. ?cb=<timestamp>) in a test you describe as a CDN performance test. Cache-busting parameters make every request miss the cache, so you measure origin performance, not CDN performance. Cache-busting tests only fit origin offload and pressure testing.
Test from multiple geographic regions
Section titled “Test from multiple geographic regions”Do: Use at least two MaxoPerf load locations when testing CDN edge performance. A CDN exists to serve users from nearby. A test from one region only measures one edge.
Don’t: Draw conclusions about global CDN performance from a single-region test. A 15 ms TTFB from us-east-1 tells you nothing about what users in ap-southeast-1 or eu-west-1 experience.
See Multi-region edge performance for the full pattern.
Respect your CDN provider’s terms of service
Section titled “Respect your CDN provider’s terms of service”Do:
- Read your CDN provider’s acceptable-use policy before running load tests. Most major providers (CloudFront, Cloudflare, Fastly, Akamai) permit load testing of customer-owned origins with prior notice or under specific conditions.
- If your provider requires advance notice before load testing, submit the request at least 48 hours in advance with the expected peak RPS and the test window.
- Test against a staging or non-production CDN distribution where you can. This removes the risk to real users.
Don’t:
- Run load tests against shared CDN infrastructure (e.g. a shared S3 static website distribution used by multiple customers) without explicit authorisation.
- Generate load at rates meant to test the CDN provider’s infrastructure capacity. You are testing your own origin and cache behaviour, not the CDN’s backbone.
- Issue mass purges during a load test in production without an incident plan. Purge testing belongs in staging.
Distinguish CDN-layer failures from origin-layer failures
Section titled “Distinguish CDN-layer failures from origin-layer failures”Do: Monitor both MaxoPerf results (CDN-to-runner perspective) and origin metrics (origin access logs, APM) during CDN load tests. A 200 response in MaxoPerf results may be the CDN serving stale content while your origin fails. You need both views.
Don’t: Assume all errors in the MaxoPerf Log tab are CDN failures. A high 5xx rate may mean origin saturation (the CDN forwards requests correctly but origin cannot handle them), or CDN misconfiguration. Look at the error breakdown and cross-reference with origin metrics.
Validate Vary headers before load testing
Section titled “Validate Vary headers before load testing”Do: Check Vary headers on your cached responses with curl -I before running a load test. Vary: User-Agent or Vary: Cookie on a public static asset fragments the cache across every unique User-Agent string in your VU pool. The hit ratio drops near zero and your CDN test tells you nothing.
Don’t: Skip header inspection before testing. A 10-second curl check can spare you a 30-minute test that says nothing about edge performance.
See Cache key and Vary testing for the full Vary testing pattern.
Do not test CDN purge behaviour against production hot paths during peak hours
Section titled “Do not test CDN purge behaviour against production hot paths during peak hours”Do: Test purge behaviour against a staging distribution or against low-traffic assets during off-peak hours. A purge test that sends unexpected traffic to origin during peak production hours can cause an outage.
Don’t: Trigger a batch purge during a load test on a production distribution without a tested incident response plan, origin scaling in place, and stakeholders aware of the test window.
See Cache invalidation and purge testing for the right sequence.
Set failure criteria before every CDN test
Section titled “Set failure criteria before every CDN test”Do: Define success before the test runs. Typical CDN failure criteria:
- p95 TTFB < 50 ms (cached small assets, same-continent region)
- p95 TTFB < 150 ms (cached API responses)
- Assertion failure rate < 1 % (proxy for cache miss rate)
- Error rate < 0.1 %
Configure these as MaxoPerf failure criteria so MaxoPerf marks the run as failed if any threshold is breached.
Don’t: Declare a CDN test passed because there are “no obvious problems in the charts.” With vague success criteria, regressions slip through between releases.
Quick reference
Section titled “Quick reference”| Rule | Do | Don’t |
|---|---|---|
| Cache warm-up | Two-phase test: warm then measure | Start high-VU load against cold cache |
| URL realism | Representative URL sample | Single URL or cache-busting params |
| Assertions | Assert on X-Cache, Age, Cache-Control | Assert on status code only |
| Geography | Test from ≥ 2 MaxoPerf regions | Single-region CDN conclusions |
| Provider ToS | Review policy, notify provider if required | Load-test shared CDN infrastructure |
| Purge testing | Staging or off-peak, low-traffic assets | Production hot paths during peak hours |
| Vary headers | Check Vary headers with curl first | Skip header inspection |
| Failure criteria | Define p95 TTFB + error thresholds upfront | Pass/fail on visual chart inspection only |
Where to go next
Section titled “Where to go next”- CDN testing overview: full section map.
- Cache hit/miss testing: the warm-up rule in practice.
- Multi-region edge performance: multi-region setup.
- Cache key and Vary testing: Vary header validation.
- Cache invalidation and purge testing: purge testing rules in detail.