Chapter 2 of 2
Course Progress0 of 2 completed (0%)

Modern System Design & API Architecture · Chapter 2 · ~15 min · Beginner

Scaling Up: Vertical vs Horizontal & Load Balancing

Learn how systems handle millions of simultaneous users without crashing: the Single Point of Failure, Vertical Scaling vs Horizontal Scaling, and how Load Balancers distribute traffic.

  • Identify Single Points of Failure (SPOF) in early-stage architectures
  • Understand the tradeoffs of Vertical Scaling (Scale Up)
  • Master Horizontal Scaling (Scale Out) used by tech giants
  • Distribute web traffic with Round Robin Load Balancers

1The Scaling Problem: What Happens When You Go Viral?

Imagine you build a website for your startup and host it on one computer. When you have 50 friends testing it, the server is fast and responsive.

Then your product gets featured on the news or goes viral on TikTok. Suddenly, 100,000 visitors arrive at the exact same minute.

Your single server's CPU hits 100%, RAM fills up completely, and incoming connection requests begin queuing up. The server crashes with a dreaded 500 Internal Server Error.

This is called a Single Point of Failure (SPOF): a single component whose failure brings down the entire system.

Concept Check

What is a 'Single Point of Failure' (SPOF) in system design?

2Vertical Scaling (Scale Up): Buying a Bigger Box

When your server is running out of horsepower, the easiest initial fix is Vertical Scaling: upgrade the machine to more CPU cores, faster clocks, and 512 GB of RAM.

The Pros

Dead Simple

Zero code changes required. Your app stays identical, running on one big computer.

The Cons

Hard Physical Ceiling

You hit physical hardware limits. Monster machines cost exponentially more money, and it is still a Single Point of Failure!

Concept Check

What is the primary limitation of relying solely on Vertical Scaling?

3Horizontal Scaling (Scale Out): The Power of Numbers

Instead of buying one giant supercomputer, Horizontal Scaling means adding dozens or hundreds of smaller, inexpensive commodity servers side by side.

If 1 server can handle 1,000 users/second, 10 servers can handle 10,000 users/second. Need to handle 50,000? Just spin up 50 servers!

Concept Check

Why do large companies like Netflix prefer Horizontal Scaling over Vertical Scaling?

4The Load Balancer: The Traffic Cop

When you have 5 backend servers running, how does a user's phone know which server to send its request to?

You place a Load Balancer in front of your server pool. The Load Balancer acts like a traffic cop:

1. Single Public IP Address

Users connect directly to the Load Balancer's public IP address, keeping your actual application servers private and safe from direct attacks.

2. Round-Robin Distribution

The load balancer sends Request 1 to Server A, Request 2 to Server B, Request 3 to Server C, repeating in a circle so no server gets overwhelmed.

3. Health Checks

The load balancer pings each server every 5 seconds (GET /healthz). If Server B stops responding, the load balancer automatically stops sending traffic to Server B until it recovers!

Concept Check

What does a Load Balancer do when one of the servers in its pool fails a health check?

5Hands-On: Test Traffic Scaling Live

Use the interactive simulator below. Drag the traffic slider to see when a single server fails, then toggle the Load Balancer on to balance the load!

Live Traffic Scaling & Load Balancer Simulator

Single Server (SPOF)
500 req/s
Light (200 req/s)Medium (1,500 req/s)Viral Spike (4,000 req/s)
Standalone Server Instance 200 OK — 100% Availability
Server #133% CPU
Handling:500 req/s

Notice: At low traffic, 1 server is fine. Drag traffic past 1,500 req/s to see it struggle.

6Chapter Knowledge Challenge

Test your mastery of scaling strategies, Single Points of Failure, and load balancer algorithms:

Concept Check 1 of 2
1

What is the critical danger of relying solely on Vertical Scaling (Scale Up) for a production system?

Concept Check 2 of 2
2

How does Round Robin load balancing distribute traffic across 3 servers?

7Recap

  • A Single Point of Failure (SPOF) is any individual component whose failure takes down the whole system.
  • Vertical Scaling (Scale Up) adds CPU/RAM to a single machine, but hits cost and physical limits.
  • Horizontal Scaling (Scale Out) adds multiple machines in parallel for infinite growth and redundancy.
  • A Load Balancer distributes incoming user traffic across servers and routes around failed machines via health checks.

Congratulations on completing Chapter 2! Mark as complete to update your syllabus:

Enjoying this free lesson? Your support helps us build more free animated chapters.

Support Us