Browser vs protocol load — the tradeoff
Protocol load testing and browser-level load testing answer different questions. If you pick the wrong one for your question, you waste money (browser VUs are expensive), get misleading data (protocol metrics miss client-side failures), or both. This page lays out the full tradeoff so you can choose one, or combine them.
Before you start
Section titled “Before you start”- Read Browser performance testing overview for the basic difference in what each approach measures.
- Read Selenium performance testing to learn how browser VUs work in MaxoPerf.
The tradeoff table
Section titled “The tradeoff table”| Dimension | Protocol load (JMeter / k6 / Taurus HTTP) | Real-browser (Taurus selenium / wdio) |
|---|---|---|
| What it measures | Raw HTTP response time: server processing + network | Full end-user experience: server + render + JS + third-party |
| VU cost | Low: thousands of VUs per CPU core | High: 1 full Chrome process per VU; 10–100× more resource-intensive |
| Max practical VU count | Hundreds to thousands on a single runner | 5–50 VUs per runner before CPU/memory saturates |
| Startup time | Near-instant | 10–30 s per Chrome instance cold-start |
| Realism | Simulated, HTTP level only | High: JS execution, rendering, third-party scripts, web fonts |
| Core Web Vitals | Not available (no browser pipeline) | Available: LCP, CLS, INP, TTFB from the browser |
| JavaScript errors | Not detected | Detected as assertion failures or uncaught exceptions |
| SPA hydration correctness | Not testable | Testable |
| Third-party script impact | Not captured | Captured |
| Scenario complexity | High: multi-step flows, correlation, data parameterization | High: full browser scripting with explicit waits |
| Result interpretation | Simple: response time = server time | Harder: response time includes render; compare to protocol baseline |
| CI suitability | Excellent: fast, deterministic, low cost | Moderate: slower start, more resource-intensive, less deterministic |
| Best for | Throughput ceiling, soak/endurance, API regression | Real UX, Core Web Vitals gate, SPA journey correctness |
What each approach catches
Section titled “What each approach catches”Protocol load catches
Section titled “Protocol load catches”- Throughput ceiling. The maximum RPS the server can sustain before latency climbs or errors appear.
- Latency under concurrency. How server-side response time degrades as the VU count rises.
- Error rate at scale. HTTP 4xx/5xx rates under realistic or peak load.
- Database / queue / cache bottlenecks. These show up as rising latency in protocol metrics.
- API regression. Whether a new deploy changed response time compared with a baseline.
Protocol load does not catch rendering failures, JavaScript errors, slow LCP from a large hero image, or the 500 ms a third-party chat widget adds to TTI.
Browser testing catches
Section titled “Browser testing catches”- Real page load time, including JS parsing, hydration and render.
- Core Web Vitals. LCP, CLS, INP and TTFB as a real browser sees them.
- Front-end regressions. A new JS bundle size or layout change that pushes LCP past your SLO.
- SPA navigation correctness. Client-side route transitions, lazy-loaded chunks, hydration mismatches.
- Third-party script overhead. Analytics, chat widgets, consent banners, A/B testing scripts.
- Visual bugs under load. Elements that fail to render when the API slows down.
Browser testing does not scale to high VU counts at a reasonable cost. It is the wrong tool for finding the server throughput ceiling.
When protocol load is the right choice
Section titled “When protocol load is the right choice”- You want to know how many concurrent users the API can handle.
- You are running a soak or endurance test over hours.
- You are finding the breakpoint / capacity ceiling.
- You need 500+ VUs.
- Your CI time budget is tight (< 5 min).
- Your application is mainly a JSON API used by a mobile app or another service.
When browser testing is the right choice
Section titled “When browser testing is the right choice”- You need to measure or gate on Core Web Vitals.
- Your application is a content-heavy SPA where rendering takes significant time.
- You need to check that a user journey completes correctly, not only that HTTP returns a 200 status.
- You have third-party scripts that matter for UX and cannot be simulated.
- You want a pre-release UX gate at a low VU count (2–10 browsers).
When to use both
Section titled “When to use both”The hybrid load architecture pattern uses both at once: many protocol VUs stress the backend while a few browser VUs sample the front-end experience. One coordinated MaxoPerf run answers both questions: “how much load can the server handle?” and “what does a user see during peak load?”
Recommended split
Section titled “Recommended split”| Test type | Protocol VUs | Browser VUs | Rationale |
|---|---|---|---|
| Pre-release UX gate | 0 | 3–5 | Pure browser check, no load needed |
| Hybrid peak simulation | 200–500 | 3–10 | Backend stress + UX measurement |
| Core Web Vitals baseline | 0 | 2–3 | Minimal browser runs for reliable CWV data |
| Throughput / breakpoint | 200–2000 | 0 | Pure capacity; a browser adds no value here |
| SPA journey regression | 0 | 5–10 | Correctness check, not a throughput measurement |
Do / don’t
Section titled “Do / don’t”Do:
- Use browser VUs when the question is about UX or Core Web Vitals.
- Start with protocol load for any scale or throughput question. It is faster, cheaper and more stable.
- Use the hybrid pattern when you need both answers in one run.
Don’t:
- Run 100+ browser VUs to generate load. Use protocol VUs for that.
- Compare browser test response times directly with protocol test response times. They measure different things.
- Drop browser testing because it is expensive. 3–5 browser VUs give you real UX data that protocol tests cannot.
Where to go next
Section titled “Where to go next”- Hybrid load architecture: the standard pattern for combining browser and protocol VUs.
- Frontend web vitals under load: how to measure Core Web Vitals while the backend is under stress.
- Selenium performance testing: how to run browser VUs in MaxoPerf.
- Test types: Frontend browser performance test: test type context and load profile guidance.