A strong answer
I would start by contrasting consistent hashing with the naive modulo approach, then explain why the difference matters in production.
Naive modulo: Assign shard hash(key) % N to server pool[N]. Adding one server changes N to N+1, and every key's shard assignment changes. You must rehash the entire dataset — a 10 TB database means copying terabytes of data across the network.
Consistent hashing: Place servers on a hash ring. Assign each key to the next server clockwise from its hash position. When a server is added, only the keys in the arc between the new server and its predecessor move. When a server is removed, only its keys move. The movement is bounded to roughly 1/N of the key space, not 100%.
The interviewer usually asks: "But what if the hash distribution is uneven?" That's where virtual nodes come in — each physical server gets many positions on the ring, smoothing out the distribution and ensuring that when one server fails, its load spreads across many others, not just one neighbor.
The shard distribution article covers the full design, including the shard-key choice and the query-cost implications. The load balancer article explains how the same technique applies to routing requests.
Follow-up directions
- Compare consistent hashing with rendezvous hashing, which avoids the ring entirely and supports weighted distribution.
- Discuss what happens when a virtual node fails — the remaining nodes must absorb its load, and if the load was uneven, some nodes may exceed capacity.
- Explain how replication interacts with the shard map — the replica placement must avoid putting all replicas in the same failure domain.