Scalability test

A scalability test answers the question: “If I add more instances, does the system actually perform proportionally better?” It is not about finding the breaking point — it is about verifying that the architecture scales horizontally as intended.

Before you start

• You can control the number of replicas of the service under test (Kubernetes replicas, ECS tasks, VM count, etc.).
• Your system passes a load test at your baseline VU count.
• You have the ability to resize the environment and rerun the test without changing the test definition.

What is a scalability test?

A scalability test runs the same fixed load profile (same VUs, same duration, same scenario) across multiple infrastructure configurations. Typical pattern:

Run	Replicas	Expected throughput (ideal linear scaling)
Run 1 (baseline)	1	RPS₁
Run 2	2	~2 × RPS₁
Run 3	4	~4 × RPS₁

If throughput doubles when replicas double, your architecture scales linearly. If throughput grows less than expected, there is a bottleneck — database, shared cache, network, or a singleton component that does not scale out.

Scalability testing is also called scale-out validation or horizontal scaling verification.

How to run a scalability test in MaxoPerf

Load profile

Parameter	Value
Virtual users (VUs)	Enough to saturate the baseline (e.g., 200 VUs for 1-replica)
Duration	15 min (same across all runs)
Ramp-up	2–3 min (same across all runs)
Stop mode	Duration
Locations	Same single location across all runs

Console walk-through

1. Create a test named api-scalability-baseline with your fixed load profile. Keep the VU count high enough that the single-replica run is noticeably constrained:

   execution:
     - executor: jmeter
       concurrency: 200
       ramp-up: 3m
       hold-for: 12m
       scenario: scale-check
   
   scenarios:
     scale-check:
       requests:
         - url: https://api.staging.example.com/v1/products
           label: list-products

2. Run Run 1 with 1 replica deployed. Tag the run replicas=1.

3. Scale your infrastructure to 2 replicas (e.g., kubectl scale deployment api --replicas=2).

4. Run the same test again. Tag the run replicas=2.

5. Scale to 4 replicas and run once more. Tag the run replicas=4.

6. Use MaxoPerf’s run comparison feature to place Run 1 and Run 2 side by side and check throughput and latency deltas.

Comparing runs

In MaxoPerf, open any run and click Compare to select another run as the reference baseline. The comparison view shows per-metric delta chips (green = improvement, red = regression) across throughput, p50/p95/p99 latency, and error rate.

How to read the result

Look for:

• Throughput (RPS) — should scale proportionally with replica count. Calculate the scaling efficiency: (RPS at 2× replicas) / (2 × RPS at 1 replica). Values above 0.85 (85 % efficiency) are generally good.
• p95 latency — should remain flat or improve as replicas increase. If latency rises with more replicas, a shared resource (load balancer, database, cache) is becoming the bottleneck.
• Error rate — should stay near zero across all runs if scaling works correctly.

If throughput growth is sub-linear, investigate:

• Database connection pool limits (the DB may not scale with the app tier).
• Session affinity / sticky routing reducing effective parallelism.
• Shared cache (Redis) becoming a hotspot.
• External API rate limits.

Do / don’t

Do	Don’t
Keep the test definition identical across all runs	Change the VU count or scenario between runs — results are not comparable
Tag each run with the replica count for easy identification	Rely on memory to track which run had which configuration
Scale replicas cleanly and wait for pods to be Ready before running	Run immediately after scaling; a pod in Pending state distorts results
Compare using MaxoPerf’s run comparison view	Compare throughput numbers across different test types or durations

Where to go next

• Breakpoint / capacity test — find the ceiling before scaling, to understand the current single-instance limit.
• Cookbook: Comparing runs and baselines — how to use the run comparison view to quantify the scaling gain.
• Foundations: Virtual users and concurrency — understand how VU count relates to throughput.