System Design Introduction

By the end of this lesson you will know what system design means, how it differs from writing code, and a simple eight-step method you can use for any design question, in an interview or at work.

What is system design?

Writing code answers the question "how do I make this feature work?". System design answers a bigger question: "how do I make this feature work for a lakh users at the same time, without losing data, and without a huge bill?".

System design is the process of choosing the parts of a software system and deciding how they work together. The parts are things like web servers, databases, caches and queues. You decide where data lives, how requests travel, and what happens when something breaks.

Think of a railway station. One ticket counter works fine for a village halt. A big junction like Lucknow Charbagh needs many counters, a display board, an enquiry desk, separate queues for reserved and general tickets, and a plan for festival rush. Nobody changed what a "ticket" is. The design of the station changed because the load changed.

Software is the same. A college project with one server and one database is a perfectly good design for fifty users. It is a poor design for five crore users.

High-level and low-level design

You will hear two terms:

This course is mostly about high-level design, with enough low-level detail (APIs, tables, keys) to make the boxes real. If you want the data-structure side, the DSA course shows where B-trees, heaps and hash rings live inside real software.

Why companies ask design questions

Product companies ask design questions because real engineering work is full of trade-offs. There is rarely one correct answer. The interviewer wants to see how you think:

• Do you ask what the system must do before drawing boxes?
• Can you estimate the load with simple maths?
• Do you know the common building blocks and their costs?
• Can you explain why you chose one option over another?

A design that is simple and well explained beats a complex design you cannot defend.

The eight-step method

Use the same steps every time. They keep you calm, and they make sure you never forget something important.

1. Clarify requirements. What must the system do (functional requirements)? How fast, how reliable, how big (non-functional requirements)?
2. Estimate the scale. Users per day, requests per second, storage per year. Rough numbers are fine.
3. Define the API. List the main operations, such as POST /orders or GET /feed.
4. Sketch the data model. Which entities exist, and which fields do you look things up by?
5. Draw the high-level design. Clients, load balancer, services, databases, caches, queues.
6. Deep-dive into the hard parts. The interviewer often picks one: the database, the feed, the hot key.
7. Find bottlenecks and failure points. What breaks first at 10 times the load? What if a server dies?
8. Summarise the trade-offs. Say what you chose, what you gave up and what you would do next.

You will practise these steps in every worked design later in the course, starting with Design a URL Shortener.

Seeing why design matters

Let us see how a single server behaves as more users arrive. A server can only work on a few requests at a time. When requests come faster than it can finish them, they wait in a queue, like passengers at one ticket window.

Step 1: work out how many requests one server can finish per second. If it has 8 worker processes and each request takes 50 ms (0.05 seconds), each worker finishes 20 requests per second, so the server finishes about 160.

Step 2: compare that with the expected traffic. Run the code below.

workers = 8
seconds_per_request = 0.05
capacity = workers / seconds_per_request      # requests per second
print(f"One server handles about {capacity:.0f} requests/second")

# A food app in Pune grows 40% every month
traffic = 30.0
for month in range(1, 9):
    servers_needed = -(-traffic // capacity)     # ceiling division
    print(f"Month {month}: {traffic:6.0f} req/s -> {servers_needed:.0f} server(s)")
    traffic *= 1.4

One server handles about 160 requests/second
Month 1:     30 req/s -> 1 server(s)
Month 2:     42 req/s -> 1 server(s)
Month 3:     59 req/s -> 1 server(s)
Month 4:     82 req/s -> 1 server(s)
Month 5:    115 req/s -> 1 server(s)
Month 6:    161 req/s -> 2 server(s)
Month 7:    226 req/s -> 2 server(s)
Month 8:    316 req/s -> 2 server(s)

By month 6 one server is not enough. That is the moment the app needs a load balancer and more servers, which is the topic of Vertical & Horizontal Scaling.

Step 3: look at waiting time, not just capacity. A server running at 95% capacity is technically "coping", but users feel the queue. This small simulation sends random requests to one server and measures how long each one waits.

import random

def average_wait(arrivals_per_sec, service_time=0.05, n=20000, seed=7):
    rng = random.Random(seed)
    clock = free_at = total_wait = 0.0
    for _ in range(n):
        clock += rng.expovariate(arrivals_per_sec)   # next request arrives
        start = max(clock, free_at)                  # wait if the server is busy
        total_wait += start - clock
        free_at = start + service_time
    return total_wait / n * 1000                     # in milliseconds

for load in [5, 10, 15, 18, 19]:
    busy = load * 0.05
    print(f"{load:>2} req/s, server busy {busy:.0%}: average wait {average_wait(load):7.1f} ms")

 5 req/s, server busy 25%: average wait     8.3 ms
10 req/s, server busy 50%: average wait    25.5 ms
15 req/s, server busy 75%: average wait    75.6 ms
18 req/s, server busy 90%: average wait   217.4 ms
19 req/s, server busy 95%: average wait   420.0 ms

This model has one worker that takes 50 ms per request. Look at the jump: from 75% to 95% busy, the average wait grows more than five times. This is why engineers keep servers well below 100% and add capacity before the rush, the same way railways add extra coaches before Diwali, not during it.

How this course is organised

• Getting Started: requirements, estimation, networking and performance numbers.
• Scaling Building Blocks: load balancers, caches, CDNs and consistent hashing.
• Data at Scale: databases, replication, sharding, consistency and storage.
• Communication: APIs, queues, idempotency, rate limiting and microservices.
• Running in Production: observability, reliability and security.
• Worked Designs: six full designs explained step by step, then a project where you design a train booking system for Tatkal-style traffic.

Every runnable example is a small Python simulation. It is not the real system, but it lets you see an idea work, change a number and watch the result.

Common mistakes

• Drawing boxes first. Without requirements you cannot judge any design. Always start with questions.
• Using buzzwords as answers. "Use Kafka" is not a design. Say what problem the queue solves and what it costs.
• Designing for Google scale on day one. Match the design to the numbers you estimated.
• Hiding trade-offs. Every choice gives something up. Saying so shows maturity.

Key points

• System design means choosing the parts of a system and how they work together at the expected scale.
• High-level design is about boxes and connections; low-level design is about what is inside each box.
• Use the same eight steps: requirements, estimates, API, data model, diagram, deep dive, bottlenecks, trade-offs.
• Waiting time grows sharply as a server gets close to full, so plan capacity early.
• A simple design you can explain beats a complex one you cannot defend.

workers = 4
seconds_per_request = 0.04
capacity = workers / seconds_per_request          # 100 requests/second
peak = 600 / 60                                   # 600 requests in one minute
servers = -(-peak // capacity)
print(f"capacity {capacity:.0f} req/s, peak {peak:.0f} req/s, servers {servers:.0f}")

capacity 100 req/s, peak 10 req/s, servers 1