A1.1.6 · Computer hardware and operation
Pipelining and multi-core processors
By the end you can
Describe how pipelining overlaps instructions, work out how many cycles it saves, how cores run pipelines independently and in parallel, and what sharing cache and memory costs them.
Think of doing laundry. Washing takes an hour, drying an hour, folding an hour. Doing four loads strictly one after another takes twelve hours. But as soon as the first load moves to the dryer, the washer is free: put the second load in. Now the washer, dryer and folding table all work at once, and four loads take six hours. Nothing got faster; the work just overlapped.
does the same with instructions. The work of one instruction is split into , each done by its own part of the hardware in one clock cycle:
- Fetch the instruction from memory.
- Decode it.
- Execute it (the ALU does its work).
- Memory access: read or write any data in memory.
- Write back the result into a register.
Between stages, results wait in small buffer registers (latches), and the clock moves everything along one stage per cycle. Once the pipeline is full, one instruction finishes every cycle.
| Cycle → | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Instr 1 | F | D | E | M | W | |||||
| Instr 2 | F | D | E | M | W | |||||
| Instr 3 | F | D | E | M | W | |||||
| Instr 4 | F | D | E | M | W | |||||
| Instr 5 | F | D | E | M | W | |||||
| Instr 6 | F | D | E | M | W |
k + ⌈n ÷ cores⌉ − 1 = 5 + 6 − 1 = 10 cycles
3.00× faster than one core with no pipeline (30 cycles).
Counting the cycles#
With k stages and n instructions, each stage taking one cycle:
- Without pipelining, each instruction occupies the whole processor for k cycles:
k × n. - With pipelining, the first instruction takes k cycles to come out of the end, and after that one more finishes every cycle:
k + n − 1.
For 5 stages and 100 instructions that is 500 cycles against 104: nearly five times the throughput. Note that each single instruction still takes 5 cycles; pipelining increases how many finish per second, not how fast any one of them goes.
Pipelines in a multi-core processor#
Every core in a multi-core processor has its own complete pipeline, which opens two ways to speed things up:
- Independent execution: different programs (or threads) run on different cores, each core pipelining its own instructions without waiting for the others. A heavy job on one core does not hold up the rest.
- Parallel execution: one big task is split into parts that run on several cores at the same time, like four people each painting one wall of a room. This only works when the parts do not depend on each other's results.
What the cores share, and why it matters#
Each core has its own pipeline, registers and L1 cache (often L2 too), but the cores share the L3 cache, the memory buses and RAM. That brings two problems a single core never has:
- . Two cores may each hold a copy of the same value in their own cache. If one core changes its copy, the other must not carry on with the old one, so the hardware marks the other copy out of date and fetches the new value. Keeping copies consistent costs time, especially when cores keep updating the same data.
- Contention. When several cores need the shared cache or RAM at the same moment, some have to wait their turn. Writes to shared data also need coordinating, so that two cores do not overwrite each other's results.
This is one reason four cores rarely give four times the speed: the more the cores share data, the more time goes on keeping them in step.
Check yourself
Pipelines and multi-core
Quick questions, made fresh every time and marked here. Do a few until they feel easy.
Exam practice
Paper 1-style questions
5 questions · 17 marks
Write your answer as you would in the exam, then show the mark points and tick each one your answer made. These are our own questions in the style of the exam, not past-paper questions.
1.Describe the process of pipelining in a CPU.
[3]Describe: A detailed account: say what happens, step by step if it is a process.
2.A processor uses a 4-stage pipeline, each stage taking one clock cycle. Calculate how many cycles it takes to process 10 instructions with pipelining, and how many without.
[2]Calculate: Work out the number, showing the working.
3.Explain how a multi-core processor combines pipelining and parallel execution to improve performance.
[3]Explain: Give reasons: say what happens AND why, or how one thing causes another.
4.Explain why cache coherency is needed in a multi-core processor.
[3]Explain: Give reasons: say what happens AND why, or how one thing causes another.
5.Evaluate pipelining and adding more cores as ways of increasing a processor’s performance.
[6]Evaluate: Weigh the strengths against the weaknesses (or the benefits against the costs) and reach a judgement that you back up.
In a nutshell
- Pipelining overlaps instructions: while one executes, the next is decoded and the one after is fetched.
- A typical pipeline has five stages: fetch, decode, execute, memory access, write back, each taking one clock cycle.
- With k stages and n instructions: k × n cycles without pipelining, k + n − 1 with it (ideal case).
- Each instruction still takes k cycles; what improves is throughput: up to one instruction finishing per cycle.
- In a multi-core CPU each core runs its own pipeline, independently (separate tasks) or in parallel (parts of one task).
- Cores share the L3 cache and RAM: cache coherency keeps their copies of data consistent, and contention for shared resources makes them wait.