A processor may execute simple arithmetic quickly, yet spend much of its time waiting for instructions or data. Computer architecture explains the structures that bridge this gap: pipelines, registers, caches, memory controllers, and the protocols that keep shared data coherent.
Instructions move through stages
A simplified pipeline fetches an instruction, decodes it, executes an operation, accesses memory when needed, and writes a result. Modern processors overlap work from multiple instructions. Dependencies can prevent that overlap: an instruction may need a value that has not been produced yet, a branch may make the next instruction uncertain, or a resource may be busy. Real CPUs use techniques such as prediction and out-of-order execution, but these do not remove the cost of every dependency.
Caches exploit locality
Caches keep recently or nearby used memory close to the processor. Temporal locality means recently used data is likely to be used again; spatial locality means nearby addresses are likely to be accessed. On a cache miss, the machine fetches a block (often called a cache line) from a lower level. The exact line size, hierarchy, and latency depend on the processor.
This is why two loops with the same number of arithmetic operations can run at different speeds. A sequential scan touches neighboring addresses; a pointer-heavy walk may wait for each next address before it can request the following one. Cache-friendly code makes useful work from each fetched line.
Branches and parallel work
The processor can predict a branch and speculatively execute likely instructions. If the prediction is wrong, work may be discarded and the pipeline refilled. Data-dependent branches with unpredictable outcomes can therefore matter, but removing a branch is not automatically faster: extra memory traffic or instructions may cost more.
Treat processor claims as hypotheses about a workload. Inspect compiler output or hardware counters when needed, then measure on the actual target. The CS:APP Cache Lab makes locality visible by asking learners to simulate cache behavior and optimize memory access.