CDN testing do and don't
CDN load testing has failure modes that do not appear in typical origin load testing. A misconfigured test can defeat the cache entirely, give you misleading results, or generate unintended traffic that violates your CDN provider’s terms of service. This page consolidates the most impactful rules for CDN testing.
Always warm the cache before measuring
Section titled “Always warm the cache before measuring”Do: Run a low-concurrency warm-up pass before your measurement phase. CDN edge nodes start cold — they have no cached copy of your assets until the first request arrives. A cold-cache run measures origin performance, not CDN performance.
Don’t: Start a high-VU load test immediately against a CDN URL you have never tested. The first few minutes of every CDN test are dominated by cache misses and do not represent steady-state edge performance.
A two-phase MaxoPerf test handles this automatically: configure a concurrency: 5, hold-for: 3m warm phase before the main load phase.
Test with realistic URL distributions
Section titled “Test with realistic URL distributions”Do: Use a representative sample of URLs from your actual traffic mix. CDN performance varies significantly across different asset types (JS bundles, images, API responses) and URL patterns. If your production traffic hits 1,000 distinct image URLs, a test hitting only 3 URLs will show an unrealistically high hit ratio.
Don’t: Test with a single URL repeated millions of times. The result is a measurement of CDN edge performance for one warm object — not a realistic picture of your CDN under real workload.
Assert on cache state, not just status codes
Section titled “Assert on cache state, not just status codes”Do: Include assertions on X-Cache, Age, Cache-Control, or your CDN’s equivalent header in every CDN load test. A 200 OK response tells you the request succeeded; it does not tell you whether it was served from cache or origin. Without cache-state assertions, a warm-cache test and a cold-cache test look identical in the MaxoPerf results overview.
Don’t: Use status code assertions alone to validate CDN behaviour. An uncached origin response returns the same 200 as a cached edge response.
Do not bypass the cache to measure CDN performance
Section titled “Do not bypass the cache to measure CDN performance”Do: Test with realistic URLs as your users would generate them.
Don’t: Add cache-busting query parameters (e.g. ?cb=<timestamp>) in a test you describe as a CDN performance test. Cache-busting parameters ensure every request misses the cache — you are measuring origin performance, not CDN performance. Cache-busting tests are only appropriate for origin offload and pressure testing.
Test from multiple geographic regions
Section titled “Test from multiple geographic regions”Do: Use at least two MaxoPerf load locations when testing CDN edge performance. A CDN’s job is geographic proximity; a test from one region only measures one edge’s performance.
Don’t: Draw conclusions about global CDN performance from a single-region test. A 15 ms TTFB from us-east-1 tells you nothing about what users in ap-southeast-1 or eu-west-1 experience.
See Multi-region edge performance for the full pattern.
Respect your CDN provider’s terms of service
Section titled “Respect your CDN provider’s terms of service”Do:
- Read your CDN provider’s acceptable-use policy before running load tests. Most major providers (CloudFront, Cloudflare, Fastly, Akamai) permit load testing of customer-owned origins with prior notice or under specific conditions.
- If your provider requires advance notice before load testing, submit the request at least 48 hours in advance with the expected peak RPS and the test window.
- Test against a staging or non-production CDN distribution where possible. This eliminates risk to real users.
Don’t:
- Run load tests against shared CDN infrastructure (e.g. a shared S3 static website distribution used by multiple customers) without explicit authorisation.
- Generate load at rates designed to test the CDN provider’s infrastructure capacity — you are testing your own origin and cache behaviour, not the CDN’s backbone.
- Issue mass purges during a load test in production without an incident plan. Purge testing belongs in staging.
Distinguish CDN-layer failures from origin-layer failures
Section titled “Distinguish CDN-layer failures from origin-layer failures”Do: Monitor both MaxoPerf results (CDN-to-runner perspective) and origin metrics (origin access logs, APM) during CDN load tests. A 200 response in MaxoPerf results may be the CDN serving stale content while your origin is failing — you need both perspectives.
Don’t: Assume all errors in the MaxoPerf Log tab are CDN failures. A high 5xx rate may indicate origin saturation (the CDN is correctly forwarding requests but origin cannot handle them), or it may indicate CDN misconfiguration. Look at the error breakdown and cross-reference with origin metrics.
Validate Vary headers before load testing
Section titled “Validate Vary headers before load testing”Do: Check Vary headers on your cached responses with curl -I before running a load test. Vary: User-Agent or Vary: Cookie on a public static asset will fragment the cache across every unique User-Agent string in your VU pool, producing near-zero hit ratio — making your CDN test useless.
Don’t: Skip header inspection before testing. A 10-second curl check can save you from running a 30-minute test that tells you nothing about edge performance.
See Cache key and Vary testing for the full Vary testing pattern.
Do not test CDN purge behaviour against production hot paths during peak hours
Section titled “Do not test CDN purge behaviour against production hot paths during peak hours”Do: Test purge behaviour against a staging distribution or against low-traffic assets during off-peak hours. Purge testing that triggers unexpected origin traffic during peak production hours can cause an outage.
Don’t: Trigger a batch purge during a load test on a production distribution without a tested incident response plan, origin scaling in place, and stakeholders aware of the test window.
See Cache invalidation and purge testing for the right sequence.
Set failure criteria before every CDN test
Section titled “Set failure criteria before every CDN test”Do: Define what success looks like before the test runs. Typical CDN failure criteria:
- p95 TTFB < 50 ms (cached small assets, same-continent region)
- p95 TTFB < 150 ms (cached API responses)
- Assertion failure rate < 1 % (proxy for cache miss rate)
- Error rate < 0.1 %
Configure these as MaxoPerf failure criteria so the run is automatically marked as failed if any threshold is breached.
Don’t: Run CDN tests and declare them passing based on “no obvious problems in the charts.” Vague success criteria mean regressions go unnoticed between releases.
Quick reference
Section titled “Quick reference”| Rule | Do | Don’t |
|---|---|---|
| Cache warm-up | Two-phase test: warm then measure | Start high-VU load against cold cache |
| URL realism | Representative URL sample | Single URL or cache-busting params |
| Assertions | Assert on X-Cache, Age, Cache-Control | Assert on status code only |
| Geography | Test from ≥ 2 MaxoPerf regions | Single-region CDN conclusions |
| Provider ToS | Review policy, notify provider if required | Load-test shared CDN infrastructure |
| Purge testing | Staging or off-peak, low-traffic assets | Production hot paths during peak hours |
| Vary headers | Check Vary headers with curl first | Skip header inspection |
| Failure criteria | Define p95 TTFB + error thresholds upfront | Pass/fail on visual chart inspection only |
Where to go next
Section titled “Where to go next”- CDN testing overview — full section map.
- Cache hit/miss testing — applying the warm-up rule in practice.
- Multi-region edge performance — multi-region setup.
- Cache key and Vary testing — Vary header validation.
- Cache invalidation and purge testing — purge testing rules in detail.