Autoscaling
Also called auto scaling, elastic scaling.
Automatically adding or removing computing capacity, such as servers or containers, as the load on a system goes up or down.
Last updated
What it is.
An autoscaler watches a signal, such as CPU use, request count or queue length, and starts more instances when it passes a threshold and removes them when load falls. It works best when the app is stateless, so any instance can handle any request.
Scaling out (more instances) is the usual pattern. Scaling up (bigger machines) is slower and has a ceiling. serverless platforms autoscale for you, down to zero when there is no traffic.
Why it matters
Without autoscaling you either pay for peak capacity all day or fall over at peak. With it, cost follows usage. But it does not fix bottlenecks further back: if the database is the limit, more app servers only make it worse.
In a product build
We keep app servers stateless, put sessions and files in shared stores, and scale on the signal that matches the bottleneck. We also set a maximum, so a bug or an attack cannot scale the bill without limit, and load test before big launches.
Common mistakes.
- 01
Scaling the app tier while the database runs out of connections.
- 02
No upper limit, which turns a traffic spike into a surprise bill.
- 03
Slow startup times, so new instances arrive after the spike is over.
Related terms.
- Load balancerA service that spreads incoming traffic across several servers, and stops sending traffic to any that fail their health checks.
- ServerlessA cloud model where the provider runs your code on demand and manages the servers, and you pay for what you use rather than for idle machines.
- Kubernetes (K8s)An open source platform that runs and manages containers across a group of machines: starting them, restarting them, scaling them and routing traffic to them.
- Connection poolingKeeping a set of open database connections and sharing them across requests, instead of opening a new connection for each one.
- Cold startThe extra delay when a serverless function or a new server instance has to start up before it can handle a request.
