High Level Design
Scalability: The Basics
Core concepts of scalability — vertical vs horizontal scaling, components that improve it, and trade-offs every system designer must reason about.
Scalability is the ability of a system to handle increased load by adding resources. It ensures that as demand grows, the system can maintain performance and reliability.
Why Scalability Matters#
- Managing Growth — Scalable systems handle more users, data, and traffic without losing speed or reliability.
- Increasing Performance — Distributing load across multiple servers boosts processing speed and response times.
- Ensuring Availability — Maintains uptime during traffic spikes or component failures.
- Cost-effectiveness — Resources can scale up or down based on demand, preventing overprovisioning.
- Encouraging Innovation — With fewer infrastructure limits, teams can build and ship new features faster.
How to Achieve Scalability#
1. Vertical Scaling (Make It Bigger)#
Add more power to the same server — more CPU, memory, or storage. Good for smaller apps but has hardware limits.
2. Horizontal Scaling (Get More Servers)#
Add more servers and spread the load. Great for large apps with lots of users.
3. Microservices (Divide and Conquer)#
Break the app into independent services and scale only the parts that need it.
4. Serverless (No Servers, No Problems)#
The platform automatically handles scaling. Cost-efficient for unpredictable workloads (e.g., AWS Lambda).
Factors Affecting Scalability#
| Factor | Description |
|---|---|
| Performance Bottlenecks | Slow queries, inefficient algorithms, or resource contention |
| Resource Utilization | Poor CPU/memory/disk use creates bottlenecks |
| Network Latency | High latency slows communication in distributed systems |
| Data Storage & Access | Distributed databases, sharding, and caching improve performance at scale |
| Concurrency & Parallelism | Running multiple tasks simultaneously increases throughput |
| System Architecture | Modular, loosely coupled systems scale more efficiently |
Components That Improve Scalability#
| Component | How It Helps |
|---|---|
| Load Balancer | Distributes incoming traffic; prevents single-server overload |
| Caching | Stores frequently accessed data; reduces latency and backend load |
| Database Replication | Copies data to multiple nodes; enhances read performance |
| Database Sharding | Splits data into smaller partitions; distributes load |
| Microservices | Each service scales independently based on its own workload |
| Data Partitioning | Divides data by region, user ID, etc.; distributes storage load |
| CDNs | Serve content from geographically closer edge servers |
| Queueing Systems | Decouple components; smooth out traffic spikes |
Challenges and Trade-offs#
Cost vs. Scalability#
Scaling requires additional resources, increasing infrastructure cost. Balance improved performance against financial impact.
Complexity#
As systems scale, architectural and operational complexity increases — harder to maintain and debug.
Latency vs. Throughput#
Lowering latency may reduce maximum throughput and vice versa. Design must prioritize the metric aligned with business requirements.
Data Partitioning Trade-offs#
Partitioning improves scalability but requires careful selection of partition keys, balancing shard sizes, and minimizing cross-shard communication.