Computer Architecture interview questions

10 questions, easiest first

1. 1Imagine an L1 cache with a 1-cycle hit time and a 3% miss rate. Every miss costs an additional 100 cycles to fetch data from lower levels of the memory hierarchy. Walk me through how you'd compute the average memory access time (AMAT), and then explain in plain terms why that miss penalty dominates the average so heavily even though the miss rate is small.intro

   Answer this one, graded

   Or just show what a strong answer covers

   AMAT = hit time + miss rate × miss penalty. Here that's 1 + 0.03 × 100 = 4 cycles. So even though 97% of accesses only take 1 cycle, the 3% that miss add 3 cycles on average to every access, quadrupling the effective latency. The point is that miss penalty and miss rate multiply together, so a 'small' miss rate can still dominate if the penalty is large — this is why architects care about reducing miss rate (bigger/associative caches, prefetching) and reducing penalty (faster lower levels, non-blocking caches) rather than just one or the other.

   • States AMAT formula — Candidate writes or derives AMAT = hit time + (miss rate × miss penalty) or an equivalent expression
   • Correct numeric result — Computes AMAT ≈ 4 cycles (1 + 0.03×100)
   • Explains dominance intuitively — Explains that a 3% miss rate multiplied by a 100-cycle penalty adds 3 cycles, quadrupling the base 1-cycle hit cost, showing penalty size matters as much as rate
   • Mentions hierarchy implication — Notes this motivates deeper cache hierarchies, prefetching, or larger/associative caches to cut miss rate or penalty

   Where answers go wrong: Forgets to add hit time, only computes miss rate × penalty · Confuses miss penalty with total access time on a miss · Assumes miss rate alone determines performance without weighting by penalty size

2. 2You have a large 2D array of doubles stored in row-major order (each row contiguous in memory). One version of your code sums every element by looping over rows in the outer loop and columns in the inner loop. Another version swaps the loop order, iterating columns in the outer loop and rows in the inner loop. On a typical cache-based machine, the first version runs dramatically faster. Why?intro

   Assume a standard cache with, say, 64-byte lines and no special prefetching beyond what a cache line fetch gives you for free.

   Answer this one, graded

   Or just show what a strong answer covers

   In row-major storage, elements of a row are contiguous in memory, so array[i][j] and array[i][j+1] sit right next to each other. When the inner loop walks across a row, the first access to a cache line pulls in, say, 8 doubles at once, and the next several accesses are free cache hits -- great spatial locality. When you swap the loop order and walk down a column instead, each step jumps by the width of a row in memory, which is almost always bigger than a cache line. So nearly every single access misses and pulls in a fresh line, most of which you never use again before it's evicted. The result is roughly one cache miss per element instead of one miss per cache-line's worth of elements, which can easily be an order of magnitude more traffic to lower levels of the hierarchy, and that's where the slowdown comes from.

   • Identifies row-major layout means consecutive elements in a row are adjacent in memory — States that in row-major storage, elements of the same row occupy contiguous addresses.
   • Connects to cache line fetch behavior — Explains that accessing one element pulls in a whole cache line, so subsequent nearby accesses become hits.
   • Explains why column-major traversal breaks locality — Notes that striding down a column jumps by the row length each step, likely landing outside the previously fetched line, causing a miss on nearly every access.
   • Notes the performance implication is a large increase in miss rate, not just 'it's slower' — Ties the reasoning back to a concrete effect: many more cache misses, not merely 'bad locality' as a vague phrase.

   Where answers go wrong: Saying 'row-major is just faster' without explaining the cache line mechanism. · Confusing spatial locality with temporal locality (reuse of the same address over time). · Ignoring that the effect depends on array size relative to cache size -- if the whole array fits in cache, the difference shrinks.

3. 3You read a 32-bit timestamp register over a serial link into a byte buffer, then reinterpret that buffer as a uint32_t and print it. On one board it prints the correct value; on a second board with a different processor, it prints a value that looks byte-shuffled — for example 0x12345678 shows up as 0x78563412. Walk me through why this happens and how you would confirm your explanation without just guessing.easy

   Answer this one, graded

   Or just show what a strong answer covers

   This is a classic endianness mismatch — the two processors disagree on which byte is most significant when a 4-byte value is loaded from memory. One board's load instruction treats the first byte as least significant (little-endian), the other treats it as most significant (big-endian), so reinterpreting the identical byte buffer as an integer gives you the reverse value. I wouldn't just assume that though — I'd first dump the raw bytes as received, in order, before any cast, and separately send a known pattern like 0x01020304 across the link so I can see exactly which byte arrives first and how each side assembles it. That confirms it's a byte-order issue rather than data corruption or a framing bug, and tells me exactly where to add a byte-swap.

   • Identifies endianness as root cause — States that the two processors disagree on byte order (little-endian vs big-endian) when reinterpreting the same byte stream as a multi-byte integer
   • Explains the byte reversal pattern — Shows that 0x12345678 becoming 0x78563412 is exactly a full byte-order reversal, consistent with swapping between the two conventions
   • Proposes concrete verification — Suggests printing the raw bytes in order before casting, or sending a known asymmetric pattern (e.g. 0x01020304) and checking which byte lands first, rather than assuming

   Where answers go wrong: Blaming it on a hardware fault or noisy link instead of byte order · Assuming both boards must use the same endianness because they're 'the same architecture family' · Fixing it by guessing which swap to apply instead of confirming with raw byte inspection first

4. 4You're told a five-stage pipelined processor has a data hazard: an instruction needs a register value that the immediately preceding instruction hasn't written back yet. Explain two different ways the hardware could handle this — one that costs no extra cycles and one that costs at least one stall cycle — and say which situations force the slower option.easy

   Answer this one, graded

   Or just show what a strong answer covers

   The fast fix is forwarding (bypassing): the ALU result sitting in the EX/MEM or MEM/WB latch gets routed directly into the ALU input of the next instruction's EX stage, so it never has to wait for the value to actually be written back to the register file. That costs zero extra cycles for a simple ALU-to-ALU dependency. But if the producing instruction is a load, the value isn't available until after the MEM stage completes, and the very next instruction's EX stage happens before that — forwarding can't reach back in time, so the hardware has to insert at least one stall cycle (a bubble) to wait for the load to finish, even with full forwarding support. That's the classic load-use hazard.

   • Describes forwarding correctly — Explains that the ALU result can be routed from EX/MEM or MEM/WB pipeline registers directly to the EX stage of the dependent instruction, avoiding waiting for writeback
   • Describes stalling correctly — Explains that a bubble/no-op is inserted into the pipeline to delay the dependent instruction until the needed value is available
   • Identifies the load-use case as forcing a stall — Recognizes that when the producing instruction is a load, the data isn't ready until after the memory stage, so even with forwarding at least one stall cycle is unavoidable

   Where answers go wrong: Says forwarding eliminates all hazards including load-use hazards · Confuses data hazards with control/branch hazards · Describes stalling without explaining what specifically triggers the need for it

5. 5You're looking at a simple 5-stage scalar pipeline (IF, ID, EX, MEM, WB) that has only a single unified memory with one port, shared by instruction fetch and data loads/stores -- there's no separate instruction cache. You notice extra one-cycle stalls happening that aren't explained by any data hazard between instructions. Digging into the trace, the stalls always occur when a load or store instruction is in its MEM stage at the same cycle another instruction is trying to do IF. What's going on, and how would a real design typically resolve it?easy

   Assume no forwarding issues are involved -- data hazards have already been ruled out by inspecting the register dependencies.

   Answer this one, graded

   Or just show what a strong answer covers

   This is a structural hazard, not a data hazard -- it's a resource conflict, not a register dependency. IF and MEM both need to talk to memory in the same cycle, but there's only one memory port, so one of them has to wait. That's exactly why real processors split instruction and data access onto separate ports or separate caches -- a Harvard-style split at the cache level -- so fetch and load/store can happen simultaneously without contention. If you can't afford a second port, the usual fallback is to stall the IF stage (bubble the pipeline) whenever a MEM-stage memory access is active, which is safe but costs a cycle every time it happens, so it's worth checking how often loads/stores actually occur in the instruction mix before deciding it's cheap enough to live with.

   • Names it as a structural hazard, not a data or control hazard — Explicitly distinguishes this resource conflict from the register-dependency hazards already ruled out.
   • Explains the resource conflict mechanically — States that IF needs to read an instruction from memory in the same cycle MEM needs to read/write data, and one port can only service one request per cycle.
   • Proposes a concrete fix — Suggests separate instruction and data memory ports/caches, or a memory arbitration/stall scheme, as the practical resolution.
   • Notes the cost of the fix or the stall — Mentions that the stall wastes a cycle each time it happens, or that adding a port/cache costs area -- shows awareness this isn't free.

   Where answers go wrong: Calling this a data hazard because it involves the MEM stage touching memory. · Assuming forwarding paths would fix it (forwarding solves data hazards, not resource conflicts). · Proposing to just 'add more pipeline stages' without addressing the actual port conflict.

6. 6Three masters on an on-chip bus — a CPU, a DMA engine, and a display controller — all want access to the same memory port, and the display controller has a hard real-time requirement (it must never stall or you get a visible glitch on screen). Describe an arbitration scheme that protects the display controller without starving the CPU or DMA, and explain what metric you'd measure on a logic analyzer or bus trace to prove your scheme actually works under load.medium

   Answer this one, graded

   Or just show what a strong answer covers

   I'd give the display controller a high, but not unconditional, priority — for example, fixed priority for its requests, but capped in burst length or duty cycle so it can't monopolize the port indefinitely, or alternatively a reserved time-division slot guaranteed every N cycles so its worst-case latency is bounded regardless of what the CPU or DMA are doing. The CPU and DMA would then arbitrate between themselves with round-robin or weighted fairness so neither starves. To prove it works, I wouldn't just eyeball the screen — I'd instrument the bus with a logic analyzer or trace capture, generate a synthetic worst-case load where CPU and DMA are both saturating the bus, and measure the actual request-to-grant latency for the display controller across many cycles, comparing the worst observed value against the FIFO depth on the display side to confirm it can never underrun.

   • Proposes a concrete arbitration scheme — Names a specific mechanism, e.g. fixed priority for the display controller with a bandwidth cap, or reserved time-division slots, not just 'give it priority'
   • Addresses starvation of other masters — Explains how CPU/DMA still get bus access, e.g. round-robin among themselves, a cap on how much bandwidth the display controller can consume even at high priority, or bounded burst lengths
   • Names a measurable verification metric — Identifies a specific thing to measure such as worst-case latency from request to grant for the display controller, or FIFO/buffer occupancy over time, under a synthetic worst-case contention test

   Where answers go wrong: Says 'give the display controller top priority' with no mechanism to prevent it from starving everyone else · Proposes strict round-robin for all three masters, which doesn't guarantee bounded latency for the real-time consumer · Names no concrete measurement, just says 'test it and see if it glitches'

7. 7A benchmark running on a new processor design shows the CPU pipeline stalling far more than expected, and profiling shows most stalls happen right after branch instructions. The branch predictor's reported accuracy looks fine in isolation tests. What are the possible explanations for the mismatch between 'predictor looks accurate' and 'lots of stalls after branches,' and how would you narrow it down?medium

   Answer this one, graded

   Or just show what a strong answer covers

   First I'd separate two different things: prediction accuracy and total stall cycles caused by branches, because they're not the same. Even a 95% accurate predictor causes real stalls if branches are frequent in this workload or if the pipeline's misprediction penalty — the flush depth — is large; a small number of mispredictions times a big penalty can dominate. Second, I'd question whether the isolated accuracy test really matches the real workload's branch behavior — the benchmark might have branch patterns, like correlated or context-dependent branches, that alias differently in the predictor's history table than whatever pattern the isolated test used, so real accuracy in context could be lower than reported. To narrow it down, I'd add hardware counters that measure misprediction rate and stall cycles specifically during the actual benchmark run rather than trusting the isolated number, and I'd also check if stalls correlate with a small set of hot branches, which would point at predictor aliasing rather than a fundamental accuracy problem.

   • Distinguishes accuracy from total penalty — Explains that even a 95%+ accurate predictor can cause many stall cycles if branches are frequent and/or the misprediction penalty (flush depth) is large
   • Considers predictor aliasing/pathological patterns — Suggests that the real workload's branch history or address patterns might alias in the predictor's table differently than the isolated test, causing effective accuracy to differ from reported
   • Proposes a narrowing strategy — Describes a concrete debugging step such as adding hardware counters for per-branch misprediction rate and stall cycles in the real workload, or comparing predictor accuracy measured in-situ during the actual benchmark versus the isolated test

   Where answers go wrong: Assumes the predictor accuracy number must be wrong/lying, rather than considering it can be true and still cause stalls · Doesn't distinguish between prediction being wrong versus prediction being right but still causing a bubble (e.g. taken-branch target delay) · Jumps straight to 'redesign the predictor' without first isolating whether the issue is frequency, penalty, or accuracy on this specific workload

8. 8A shared on-chip bus runs at 200 MHz and transfers 8 bytes (64 bits) per cycle when fully utilized. Three agents need simultaneous access: a display controller needs a sustained 1500 MB/s to avoid frame underrun, a DMA engine averages 800 MB/s, and the CPU needs about 400 MB/s on average for cache refills. What percentage of the bus's peak bandwidth would be required to satisfy all three simultaneously, and what does that tell you about whether this bus can support the workload as-is?medium

   Assume the bus can sustain its peak transfer rate every cycle with no protocol overhead, to keep the arithmetic focused on the core bandwidth budgeting question.

   Answer this one, graded

   Or just show what a strong answer covers

   First, peak bus bandwidth is 200 MHz times 8 bytes per cycle, which is 1600 MB/s, or 1.6 GB/s. The three consumers need 1500 plus 800 plus 400, which is 2700 MB/s simultaneously. That's 2700 divided by 1600, about 169% of the bus's capacity. Since that's over 100%, the bus is oversubscribed -- no arbitration scheme can make 2700 MB/s of demand fit through a 1600 MB/s pipe. Arbitration only decides who gets to wait, it doesn't create bandwidth. The real-time display controller is the one that will suffer visibly if it doesn't get priority, since a missed deadline there means visible glitches, whereas the CPU and DMA can usually tolerate some stalling. So the actual fix has to be architectural: widen the bus, raise its clock, add a second port, or reduce one of the demands, not just retune the arbiter.

   • Computes peak bus bandwidth correctly — Calculates 200 MHz * 8 bytes = 1600 MB/s (1.6 GB/s) as the bus ceiling.
   • Sums the three demands correctly — Adds 1500 + 800 + 400 = 2700 MB/s as total simultaneous demand.
   • Computes utilization percentage and concludes over capacity — Divides total demand by peak bandwidth (2700/1600 ≈ 168.75%) and concludes the bus cannot satisfy all three at once.
   • Draws the practical consequence — States that since the bus is oversubscribed, at least one consumer (most critically the real-time display controller) will miss its deadline no matter how arbitration is tuned, and that either bandwidth must be reduced, the clock/width increased, or priorities must sacrifice a non-real-time consumer.

   Where answers go wrong: Forgetting to convert MHz and byte-width into a bandwidth figure before comparing to MB/s demands. · Assuming clever arbitration alone can fix an oversubscribed bus -- arbitration only decides who waits, not whether total demand fits. · Treating the demands as compatible without adding them, since they are described as needing to happen simultaneously.

9. 9You're designing the memory hierarchy for a small embedded processor and have a fixed silicon area budget. You must choose between (a) a larger, more associative single-level cache, or (b) a smaller L1 backed by a modest L2, given that the target workload has a large but only moderately reused working set with occasional bursts of sequential access. Walk me through how you'd decide, including what you'd actually measure to justify the choice.hard

   Answer this one, graded

   Or just show what a strong answer covers

   I'd start from what the workload tells me: a large working set with only moderate reuse means a single cache big enough to hold the whole working set would need a lot of area and, worse, more ways/larger arrays typically increase hit latency, which hurts every access, not just misses. The occasional sequential bursts suggest a simple stream prefetcher or an L2 optimized for sequential fetches could capture a lot of value cheaply. So my instinct is a split hierarchy: keep L1 small and fast for the moderately-reused hot subset, and let L2 (or a prefetcher in front of it) absorb the sequential bursts and the rest of the working set, accepting a higher L2 miss penalty in exchange for lower average hit latency on the common case. But I wouldn't commit without checking the numbers — I'd run representative traces through a simulator for both configurations, computing miss rate per level, AMAT, and area cost, and compare. If the moderate reuse means L1 hit rate stays reasonably high even at modest size, the split hierarchy wins on area-normalized AMAT; if reuse turns out lower than assumed, the calculus could flip toward a bigger single-level cache.

   • Reasons about workload fit — Connects the described access pattern (large working set, moderate reuse, sequential bursts) to which structure better exploits it — e.g. bursts favor prefetching-friendly sequential access in L2, moderate reuse limits benefit of huge associativity in L1
   • Identifies the real tradeoff, not just 'bigger is better' — Explains that a bigger single-level cache increases hit latency (longer tag/data array access, more ways to check) while a split hierarchy keeps L1 fast but adds a miss path through L2 with its own latency and complexity
   • Proposes concrete measurement/evaluation — Describes computing/simulating AMAT, miss rate at each level, and area cost for both configurations on a representative trace before committing to a choice
   • Reaches a defensible decision with reasoning, not a guess — States a preference (e.g. split hierarchy given moderate reuse and area constraint) and ties it explicitly back to the tradeoffs discussed rather than asserting it without justification

   Where answers go wrong: Assumes bigger/more associative is always strictly better without considering hit-time cost · Picks an answer without proposing any way to measure or validate it · Ignores the 'moderate reuse' detail and reasons as if the workload had either no locality or perfect locality · Forgets that area budget is fixed, so 'just make both bigger' isn't a valid answer

10. 10A system has a CPU with a write-back data cache and a DMA engine that reads sensor data and writes it directly into main memory, bypassing the CPU's cache entirely. After a DMA transfer completes and the CPU is signaled to read the new data, it sometimes reads stale values instead of what the DMA just wrote. Walk me through the mechanism causing this, and describe at least two different ways a system could actually fix it.hard

    Assume the DMA engine has no visibility into the CPU's cache contents and there is no hardware cache-coherence protocol between them.

    Answer this one, graded

    Or just show what a strong answer covers

    The DMA write goes straight to main memory and never touches the CPU's cache, so if the CPU had previously read that address and still has a valid cached line for it, the cache now holds stale data while memory holds the fresh DMA-written data -- the two views have diverged. When the CPU reads again, if it hits in cache, it gets the old value, even though memory itself is correct. This is a coherence problem, not a timing or ordering bug -- the DMA genuinely finished before the read, but the CPU's cache wasn't told anything changed. Two ways to fix it: first, purely in software, invalidate the cache lines covering that buffer right after the DMA completes and before the CPU reads, which forces the next CPU access to miss and fetch the real data from memory -- cheap but relies on software discipline and is easy to forget in a driver. Second, mark the DMA target buffer as non-cacheable, so the CPU always goes to memory for it directly -- simpler and foolproof, but slower for any code path that re-reads that buffer often. A more expensive but transparent option is a hardware coherence/snooping mechanism where the DMA write is observed by the cache controller and automatically invalidates or updates the matching line, at the cost of extra bus snoop traffic and complexity.

    • Correctly explains the root cause — States that the CPU cache holds an old copy of the line from before the DMA write, and because the DMA write didn't go through the cache, the cache is now inconsistent with memory.
    • Distinguishes this from a simple ordering/synchronization bug — Notes this is a coherence problem specifically -- the data arrived correctly in memory, but the CPU's view of it via the cache is wrong -- not a race in when the DMA finished.
    • Proposes software cache invalidation as one fix — Describes invalidating the relevant cache lines (or the whole cache/region) after DMA completion and before the CPU reads, forcing a fetch from memory.
    • Proposes a second, distinct fix — Describes either marking the buffer non-cacheable/uncached, or relying on a hardware coherence/snooping mechanism, as an alternative to software invalidation.
    • Notes a tradeoff between the fixes — Mentions a cost such as invalidation being software overhead and error-prone if forgotten, versus non-cacheable memory being simpler but slower for repeated CPU access, or hardware coherence costing area/complexity.

    Where answers go wrong: Blaming the DMA engine for writing 'wrong' data, when the data in memory is actually correct. · Confusing this with a data hazard or pipeline stall, which are unrelated pipeline-internal issues. · Proposing only 'add a memory barrier' without addressing that a barrier alone doesn't invalidate stale cache lines -- ordering and coherence are different problems.