High Level Design
Load Balancing
Load balancers, routing algorithms, Layer 4 vs Layer 7 balancing, health checks, and how traffic flows through a real system.
A load balancer distributes incoming client requests to computing resources such as application servers and databases. It can be hardware-based or software-based.
Benefits#
- Scalability — Add more servers behind the load balancer without redesigning your architecture
- High Availability — If one server goes offline, traffic automatically reroutes to healthy servers
- Performance Optimization — Prevents any single server from being overloaded while others sit idle
- Maintainability — Perform maintenance on individual servers without taking the whole system down
Types of Load Balancers#
Hardware vs. Software#
| Type | Description | Examples |
|---|---|---|
| Hardware | Specialized physical devices; powerful but expensive and less flexible | F5, Citrix ADC |
| Software | Runs on standard servers/VMs; open-source, highly configurable | Nginx, HAProxy, Envoy |
Layer 4 vs. Layer 7#
| Type | Routing Basis | Use Case |
|---|---|---|
| Layer 4 (Transport) | IP address + port; doesn't inspect application data | Fast, simple traffic routing |
| Layer 7 (Application) | HTTP headers, cookies, URL path; application-aware | Advanced routing, A/B testing, sticky sessions |
Routing Algorithms#
| Algorithm | Description |
|---|---|
| Round Robin | Requests distributed sequentially to each server in a loop |
| Weighted Round Robin | Servers with higher weights receive proportionally more requests |
| Least Connections | Request goes to the server with fewest active connections |
| IP Hash | Hash of client IP always routes to the same server (sticky sessions) |
| Random | Server selected randomly |
How Load Balancers Work#
Step 1: Traffic Reception#
All incoming requests arrive at the load balancer's public IP or domain.
Step 2: Routing Decision#
The load balancer picks a server based on the chosen routing algorithm.
Step 3: Health Checks#
Load balancers periodically check if servers are alive (e.g., GET /health).
- Unhealthy servers are removed from rotation automatically
- Healthy servers are re-added when they recover
Step 4: Response Handling#
The server processes the request and returns a response to the load balancer, which forwards it to the client.