A software system is not a program. A program reads input and produces output. A system is a collection of components — each with its own boundary, responsibility, and failure mode — that work together to serve a user need over time. The gap between those two words is where system design lives.
Start with the box, not the pieces
Before you draw a database or a queue, draw the box. What does the system do for its user? What crosses its boundary, and what stays inside? A web service's boundary is its API: requests enter through HTTP endpoints, responses leave through the same door. Everything behind that door — caching, databases, background workers — is an internal implementation detail that the user never directly touches.
The boundary tells you what you can change without breaking the outside world, and what you cannot change without a coordination cost. That distinction is the first design question every system answers.
Components, connections, and the cost between them
Inside the boundary, a system is components connected by communication. A component owns some state or some logic. A connection is how one component asks another to do work. That connection has a price: serialization, network latency, retries, timeouts, and the possibility of failure. Most system design is deciding which costs are worth paying and which can be avoided.
Read the full request journey in a browser request reaches a server, and see how cache decisions play out in a cache hit and miss trace.
Failure is not an exception, it is the default
A single program either runs or crashes. A system is always partially failing. Network requests time out. Databases slow down. Processes restart. The job of system design is to make those failures degrade gracefully: a missing cache should slow things down, not bring the whole service down; a slow database should trigger timeouts and fallbacks, not pile up requests until everything collapses.
Start every design by asking: when this piece fails, what does the caller see?
The three questions that drive every decision
Every system design reduces to three questions, asked in order:
- What must the system do? — Functional requirements and the user-facing API. No amount of scaling helps if you built the wrong thing.
- How well must it do it? — Latency, throughput, availability, durability. These become concrete constraints, not vague aspirations.
- What can fail, and how do we live with it? — Failure modes and the acceptable degradation when they happen.
The answer to question three is never "nothing." It is "this specific thing can fail, and here is how we detect it and recover."
The discipline, not the diagram
System design is often taught as a gallery of architecture diagrams — microservices, event sourcing, CQRS, microservices on a service mesh. Those are tools, not goals. Good system design is the discipline of making the cheapest choice that satisfies the requirements today, and knowing what trigger will force the next, more expensive choice. The simplest system that works is better than the most sophisticated system that is hard to operate.
This topic builds that discipline, one trace at a time. Start with the foundations learning path.