Scalability test

A scalability test answers one question: “If I add more instances, does the system actually perform proportionally better?” It does not look for the breaking point. It checks that the architecture scales horizontally the way you designed it to.

Before you start

• You can control the number of replicas of the service under test (Kubernetes replicas, ECS tasks, VM count, etc.).
• Your system passes a load test at your baseline VU count.
• You can resize the environment and rerun the test without changing the test definition.

What is a scalability test?

A scalability test runs the same fixed load profile (same VUs, same duration, same scenario) across multiple infrastructure configurations. Typical pattern:

Run	Replicas	Expected throughput (ideal linear scaling)
Run 1 (baseline)	1	RPS₁
Run 2	2	~2 × RPS₁
Run 3	4	~4 × RPS₁

If throughput doubles when replicas double, your architecture scales linearly. If throughput grows less than expected, something is a bottleneck: the database, a shared cache, the network, or a singleton component that does not scale out.

Scalability testing is also called scale-out validation or horizontal scaling verification.

How to run a scalability test in MaxoPerf

Load profile

Parameter	Value
Virtual users (VUs)	Enough to saturate the baseline (e.g., 200 VUs for 1-replica)
Duration	15 min (same across all runs)
Ramp-up	2–3 min (same across all runs)
Stop mode	Duration
Locations	Same single location across all runs

Console walk-through

1. Create a test named api-scalability-baseline with your fixed load profile. Set the VU count high enough that the single-replica run is clearly constrained:

   execution:
     - executor: jmeter
       concurrency: 200
       ramp-up: 3m
       hold-for: 12m
       scenario: scale-check
   
   scenarios:
     scale-check:
       requests:
         - url: https://api.staging.example.com/v1/products
           label: list-products

2. Run Run 1 with 1 replica deployed. Tag the run replicas=1.

3. Scale your infrastructure to 2 replicas (e.g., kubectl scale deployment api --replicas=2).

4. Run the same test again. Tag the run replicas=2.

5. Scale to 4 replicas and run once more. Tag the run replicas=4.

6. Use MaxoPerf’s run comparison to put Run 1 and Run 2 side by side and check the throughput and latency deltas.

Comparing runs

In MaxoPerf, open any run and click Compare to select another run as the reference baseline. The comparison view shows per-metric delta chips (green = improvement, red = regression) across throughput, p50/p95/p99 latency, and error rate.

How to read the result

Look for:

• Throughput (RPS). It should grow in proportion to the replica count. Calculate the scaling efficiency: (RPS at 2× replicas) / (2 × RPS at 1 replica). Values above 0.85 (85 % efficiency) are generally good.
• p95 latency. It should stay flat or improve as you add replicas. If latency rises with more replicas, a shared resource (load balancer, database, cache) is turning into the bottleneck.
• Error rate. If scaling works, it stays near zero across all runs.

If throughput grows sub-linearly, check:

• Database connection pool limits (the DB may not scale with the app tier).
• Session affinity / sticky routing reducing effective parallelism.
• Shared cache (Redis) becoming a hotspot.
• External API rate limits.

Do / don’t

Do	Don’t
Keep the test definition identical across all runs	Change the VU count or scenario between runs, which makes results incomparable
Tag each run with the replica count for easy identification	Rely on memory to track which run had which configuration
Scale replicas cleanly and wait for pods to be Ready before running	Run immediately after scaling; a pod in Pending state distorts results
Compare using MaxoPerf’s run comparison view	Compare throughput numbers across different test types or durations

Where to go next

• Breakpoint / capacity test: find the ceiling before scaling, so you know the current single-instance limit.
• Cookbook: Comparing runs and baselines: use the run comparison view to measure the scaling gain.
• Foundations: Virtual users and concurrency: how VU count relates to throughput.