Startup time and rebuffering QoE

Load tests measure HTTP latency. Quality of Experience (QoE) is what viewers actually notice. The core skill in streaming load testing is connecting the two: knowing which latency numbers predict buffering, startup failure or a degraded experience before your real viewers tell you.

This page covers the main streaming QoE metrics, the math that links segment download time to buffer starvation, and how to read MaxoPerf latency results in QoE terms.

Before you start

• Read HLS and DASH manifest and segment testing for how segment requests map to load-test labels.
• Read Concurrent viewers load for the segment request rate math.

The key QoE metrics

Startup / join time

Definition: The time from a viewer pressing play to the first frame of video appearing.

In HTTP terms, startup time is dominated by:

1. Time to fetch and parse the master manifest.
2. Time to fetch and parse the variant playlist.
3. Time to download the first segment (or first few chunks for LL-HLS).

In MaxoPerf terms, startup time ≈ master-manifest latency + variant-playlist latency + first-segment latency. If your master manifest p95 is 300ms, variant playlist p95 is 200ms, and first segment p95 is 1,200ms, then your p95 startup time proxy is ~1,700ms. Most streaming QoE benchmarks target startup time under 3 seconds; premium live services target under 1 second.

Segment download time vs segment duration: the buffer equation

No QoE relationship matters more in streaming load testing.

For a viewer to watch without buffering, the time to download each segment must be less than the playback duration of that segment:

segment_download_time < segment_duration

If segment_download_time ≥ segment_duration, the player’s buffer runs out before the next segment arrives, and the viewer sees buffering. This is called buffer starvation.

In percentile terms:

• If p50 segment latency < segment_duration and p95 < segment_duration → healthy, no buffering in most viewers.
• If p95 segment latency > segment_duration → 5% of your viewers are buffering. At 100,000 concurrent viewers, that is 5,000 viewers with buffering events.
• If p99 segment latency > segment_duration → tail-percentile buffering, likely acceptable if p95 is fine.

Rebuffer ratio

Definition: The fraction of total playback time spent buffering (waiting for segments) rather than playing.

A load test cannot measure rebuffer ratio directly, because that needs client-side player instrumentation. You can derive a proxy instead:

rebuffer_proxy = max(0, segment_download_time - segment_duration) / segment_duration

When segment_download_time exceeds segment_duration, the excess time is exactly the rebuffer duration for that segment. Across all segment requests in your run, you can compute:

rebuffer_ratio_proxy = (sum of max(0, download_time - segment_duration)) / (total_segments × segment_duration)

Industry benchmarks: a rebuffer ratio above 0.5% measurably reduces viewer retention, and above 1% causes significant churn.

Manifest fetch time (live streams)

For live streams, the variant playlist must be re-fetched every segment interval. If variant-playlist latency climbs above half the segment interval, viewers may stall waiting for updated playlists. For a 4-second segment interval, keep manifest p95 under 2,000ms.

How to read MaxoPerf results for QoE

Setting up QoE-oriented failure criteria

Configure failure criteria in MaxoPerf before the run so the test fails automatically when a QoE threshold is breached:

# In MaxoPerf test configuration (Taurus reporting section)
reporting:
  - module: passfail
    criteria:
      # Startup time components
      - avg-rt of master-manifest > 500ms for 1m: stop as failed
      - p95(master-manifest) > 1000ms for 1m: stop as failed

      # Buffer starvation gate — the critical QoE criterion
      # Segment duration = 4000ms; fail if p95 exceeds it
      - p95(segment) > 4000ms for 2m: stop as failed

      # Live manifest re-fetch
      - p95(variant-playlist) > 2000ms for 2m: stop as failed

      # Error rate
      - fail rate of segment > 0.5% for 1m: stop as failed

These criteria map directly to viewer experience. As soon as p95 segment latency crosses the segment duration, the test fails automatically. You do not need a manual review to catch the most critical streaming failure mode.

Analyzing the results chart

After a run, look at the latency chart in the Overview tab with these QoE benchmarks in mind:

Metric	Healthy	Warning	Critical
master-manifest p95	< 300ms	300–800ms	> 800ms
variant-playlist p95	< 300ms	300–1000ms	> segment_duration / 2
segment p50	< segment_duration × 0.3	< segment_duration × 0.6	> segment_duration × 0.8
segment p95	< segment_duration × 0.7	70–90% of segment_duration	≥ segment_duration
segment p99	< segment_duration	n/a	≥ segment_duration
Error rate (segments)	< 0.1%	0.1–0.5%	> 0.5%

Deriving startup time from the run

MaxoPerf does not compute startup time directly. You can derive a proxy from the run data:

1. Filter the latency chart to show only the first master-manifest, variant-playlist, and first segment requests.
2. Sum the p50 (or p95) values for those three labels.
3. This gives you the estimated startup time at the 50th (or 95th) percentile for a viewer session that started mid-test.

For accurate startup time, include a “cold start” VU scenario in your test that starts from the beginning of the player sequence (no cached manifests).

Do / don’t

Do:

• Always compare segment latency against segment duration. Raw milliseconds without that context say nothing about streaming QoE.
• Set failure criteria on segment p95 < segment_duration before running. No gate matters more for streaming tests.
• Run the test long enough to see steady-state QoE, not only CDN warm-up behavior.

Don’t:

• Report only average latency. The tail (p95, p99) drives streaming QoE. A viewer who buffers once in 30 minutes remembers it, and the average hides those events.
• Mix up manifest latency and segment latency. They point to different infrastructure components and need different fixes.
• Ignore error rates. A 1% segment 404 rate means 1% of every viewer’s segments are missing, which causes buffering at once, whatever the latency.

Where to go next

• Concurrent viewers load: the full viewer throughput and latency test setup.
• Live event streaming load: how startup time behaves during the cold-cache kickoff window.
• Video streaming do and don’t: a short checklist of the most important practices.
• Failure criteria / pass-fail gates: configure p95 segment latency gates in MaxoPerf.