What Is Load Balancing? The Basics Explained

Here is why a website can absorb a sudden traffic spike without going down, and how a load balancer actually decides where each request goes.

What load balancing is

Load balancing spreads incoming traffic across multiple servers instead of sending it all to one, so a single overloaded or failed server does not take the whole service down. A load balancer sits in front of a pool of servers, accepts every request, and routes each one to whichever server is best suited to handle it at that moment.

Round robin

The simplest and most common method: incoming requests are handed to servers in strict rotating order, server A, then B, then C, then back to A. It is easy to implement and works well when every server has similar capacity, but it ignores the fact that some servers may already be busier than others.

Least connections

Instead of rotating blindly, this method sends each new request to whichever server currently has the fewest active connections, in other words, the least busy one. It produces a more even distribution than round robin when request processing times vary a lot, such as when some requests involve a slow file upload.

IP hash (session persistence)

This method always routes a given client's requests to the same server, based on a hash of the client's IP address. It matters for services that store session data, like login state, in one server's memory rather than sharing it, since without IP hashing a user could get logged out simply because a later request landed on a different server. The trade-off is that many users behind the same shared IP, such as an office network, can end up piling onto one server.

Layer 4 vs. Layer 7 load balancers

A Layer 4 load balancer only looks at network-level information, IP addresses and ports, so it is fast but limited to simple routing rules. A Layer 7 load balancer inspects the actual request content, such as the URL path or browser type, and can route '/images' traffic one way and '/api' traffic another. That makes it especially useful for microservice architectures split by function.

Health checks

A load balancer periodically pings each server, or calls a dedicated status-check URL, to confirm it is responding normally. If a server stops responding or keeps erroring out, it is automatically and temporarily pulled from the pool so users never see its failures, then added back once it recovers.

Load balancers vs. CDNs

A load balancer distributes traffic among servers that usually sit in the same data center; a content delivery network (CDN) distributes traffic across edge servers spread around the world. Large-scale services typically use both together: a CDN routes a request to the nearest region first, and a load balancer then splits it among the servers within that region.

Load balancing plus autoscaling

In cloud environments, a traffic surge can trigger autoscaling to automatically launch new server instances, which the load balancer immediately starts routing traffic to. When traffic drops, unneeded instances are scaled back down to save cost, so the two systems are commonly paired in cloud infrastructure.

Load balancing algorithms beyond the basics

Round robin and least connections are the two most common starting points, but production systems often use refinements on top of them: weighted round robin, giving more powerful servers a larger share of requests, least response time, factoring in how quickly a server has been answering recently, and consistent hashing, used heavily in caching layers so the same key almost always maps to the same server even as servers are added or removed.

Load balancing in modern, cloud-native infrastructure

Beyond a single dedicated appliance, load balancing today often happens at several layers at once: a cloud provider's managed load balancer in front of the whole application, an ingress controller distributing traffic inside a Kubernetes cluster, and a service mesh handling load balancing between individual internal services. Each layer solves the same basic problem, not overloading any one instance, at a different scope.

Frequently Asked Questions

What happens if the load balancer itself fails?

To avoid the load balancer becoming a single point of failure, real-world deployments usually run at least two load balancers in an active-standby or active-active setup, so if one goes down the other keeps handling traffic without interruption.

Does a small website need load balancing?

Not if a single server comfortably handles its traffic. Once traffic grows enough to require multiple servers, though, a load balancer becomes essential, since there is no other way to split incoming requests across them.