Startup time and rebuffering QoE

Load tests measure HTTP latency. Quality-of-Experience (QoE) is what viewers actually feel. Bridging those two is the core skill in streaming load testing — knowing which latency numbers predict buffering, startup failure, or a degraded viewer experience before your real viewers tell you.

This page explains the key streaming QoE metrics, the mathematical relationship between segment download time and buffer starvation, and how to read MaxoPerf latency results through a QoE lens.

Before you start

• Read HLS and DASH manifest and segment testing for how segment requests map to load-test labels.
• Read Concurrent viewers load for the segment request rate math.

The key QoE metrics

Startup / join time

Definition: The time from a viewer pressing play to the first frame of video appearing.

In HTTP terms, startup time is dominated by:

1. Time to fetch and parse the master manifest.
2. Time to fetch and parse the variant playlist.
3. Time to download the first segment (or first few chunks for LL-HLS).

In MaxoPerf terms, startup time ≈ master-manifest latency + variant-playlist latency + first-segment latency. If your master manifest p95 is 300ms, variant playlist p95 is 200ms, and first segment p95 is 1,200ms, then your p95 startup time proxy is ~1,700ms. Most streaming QoE benchmarks target startup time under 3 seconds; premium live services target under 1 second.

Segment download time vs segment duration — the buffer equation

This is the most important QoE relationship in streaming load testing.

For a viewer to watch without buffering, the time to download each segment must be less than the playback duration of that segment:

segment_download_time < segment_duration

If segment_download_time ≥ segment_duration, the player’s buffer runs out before the next segment arrives — the viewer buffers. This is called buffer starvation.

In percentile terms:

• If p50 segment latency < segment_duration and p95 < segment_duration → healthy, no buffering in most viewers.
• If p95 segment latency > segment_duration → 5% of your viewers are buffering. At 100,000 concurrent viewers, that is 5,000 viewers with buffering events.
• If p99 segment latency > segment_duration → tail-percentile buffering, likely acceptable if p95 is fine.

Rebuffer ratio

Definition: The fraction of total playback time spent buffering (waiting for segments) rather than playing.

Rebuffer ratio is not directly measurable from a load test — it requires client-side player instrumentation. However, you can derive a proxy:

rebuffer_proxy = max(0, segment_download_time - segment_duration) / segment_duration

When segment_download_time exceeds segment_duration, the excess time is exactly the rebuffer duration for that segment. Across all segment requests in your run, you can compute:

rebuffer_ratio_proxy = (sum of max(0, download_time - segment_duration)) / (total_segments × segment_duration)

Industry benchmarks: rebuffer ratio above 0.5% measurably reduces viewer retention; above 1% causes significant churn.

Manifest fetch time (live streams)

For live streams, the variant playlist must be re-fetched every segment interval. If variant-playlist latency climbs above half the segment interval, viewers may stall waiting for updated playlists. For a 4-second segment interval, keep manifest p95 under 2,000ms.

How to read MaxoPerf results for QoE

Setting up QoE-oriented failure criteria

Configure failure criteria in MaxoPerf before the run to automatically fail the test when QoE thresholds are breached:

# In MaxoPerf test configuration (Taurus reporting section)
reporting:
  - module: passfail
    criteria:
      # Startup time components
      - avg-rt of master-manifest > 500ms for 1m: stop as failed
      - p95(master-manifest) > 1000ms for 1m: stop as failed

      # Buffer starvation gate — the critical QoE criterion
      # Segment duration = 4000ms; fail if p95 exceeds it
      - p95(segment) > 4000ms for 2m: stop as failed

      # Live manifest re-fetch
      - p95(variant-playlist) > 2000ms for 2m: stop as failed

      # Error rate
      - fail rate of segment > 0.5% for 1m: stop as failed

These criteria translate directly to viewer experience: the moment p95 segment latency crosses the segment duration, the test fails automatically. No manual result review needed to catch the most critical streaming failure mode.

Analyzing the results chart

After a run, look at the latency chart in the Overview tab with these QoE benchmarks in mind:

Metric	Healthy	Warning	Critical
master-manifest p95	< 300ms	300–800ms	> 800ms
variant-playlist p95	< 300ms	300–1000ms	> segment_duration / 2
segment p50	< segment_duration × 0.3	< segment_duration × 0.6	> segment_duration × 0.8
segment p95	< segment_duration × 0.7	70–90% of segment_duration	≥ segment_duration
segment p99	< segment_duration	—	≥ segment_duration
Error rate (segments)	< 0.1%	0.1–0.5%	> 0.5%

Deriving startup time from the run

MaxoPerf does not compute startup time directly, but you can derive a proxy from the run data:

1. Filter the latency chart to show only the first master-manifest, variant-playlist, and first segment requests.
2. Sum the p50 (or p95) values for those three labels.
3. This gives you the estimated startup time at the 50th (or 95th) percentile for a viewer session that started mid-test.

For accurate startup time, include a “cold start” VU scenario in your test that starts from the beginning of the player sequence (no cached manifests).

Do / don’t

Do:

• Always compare segment latency against segment duration — raw milliseconds without this context are meaningless for streaming QoE.
• Set failure criteria on segment p95 < segment_duration before running. This is the single most important gate for streaming tests.
• Run the test long enough to see steady-state QoE, not just CDN warm-up behavior.

Don’t:

• Report only average latency — streaming QoE is dominated by the tail (p95, p99). A viewer who buffers once in 30 minutes remembers it; the average masks these events.
• Conflate manifest latency with segment latency — they indicate different infrastructure components and require different remediation.
• Ignore error rates — a 1% segment 404 rate means 1% of every viewer’s segments are missing, causing immediate buffering regardless of latency.

Where to go next

• Concurrent viewers load — the full viewer throughput and latency test setup.
• Live event streaming load — how startup time behaves during the cold-cache kickoff window.
• Video streaming do and don’t — a condensed checklist of the most important practices.
• Failure criteria / pass-fail gates — configuring p95 segment latency gates in MaxoPerf.