Test Automation interview questions

10 questions, easiest first

1. 1You're asked to take a manual bench check -- power up a board, probe a rail with a multimeter, and eyeball whether it's in range -- and turn it into an automated script that will run on hundreds of boards. Before writing any code, how would you structure it?intro

   Answer this one, graded

   Or just show what a strong answer covers

   I'd split it into three pieces. First, an instrument control layer -- thin wrappers around the power supply and DMM that just know how to set voltage, read a measurement, handle timeouts, and report errors. Second, a sequencing layer that says 'power on, wait for settle, take measurement, power off' -- the actual test flow, independent of which specific instruments are behind it. Third, an assertion/limits layer that takes the measured value and compares it against a defined range to decide pass/fail, with those limits stored somewhere configurable, not hardcoded in the middle of the sequence. That way if the DMM model changes, only the instrument layer changes; if the spec changes, only the limits change; and the sequence logic stays untouched either way.

   • Identifies at least three logical layers — Answer separates instrument control (talking to the power supply/DMM), sequencing (what order steps run, setup/teardown), and assertions (comparing measured value to limits) rather than one monolithic script.
   • Explains why separation matters — Gives a concrete reason: swapping instruments, reusing the measurement across other tests, or being able to change limits without touching instrument code.
   • Mentions limits as data not hardcoded logic — States that pass/fail thresholds should live outside the control flow (config, constants, or a table) so they can change without editing the test logic.

   Where answers go wrong: Describing only 'write a Python script that measures voltage' with no structural separation. · Focusing entirely on which instrument/library to use instead of architecture. · Not mentioning that limits/thresholds should be configurable rather than baked into if-statements.

2. 2You've written an automated test script for a voltage regulator board: it opens a connection to a power supply, sets a voltage, reads back a current from a multimeter, and prints PASS or FAIL directly in the same function using a hardcoded threshold. Your manager asks you to add a second board variant that needs a different multimeter model and a different pass threshold. What's wrong with the current script's structure, and how would you restructure it before bolting on the new variant?intro

   Answer this one, graded

   Or just show what a strong answer covers

   Right now the instrument commands, the sequencing, and the pass/fail decision are all smashed into one function, so any change to hardware or limits means editing test logic directly -- that's why adding a variant is painful. I'd split it into three layers: an instrument driver layer that exposes generic calls like set_voltage() or read_current() regardless of which multimeter model is underneath, a sequencer that just calls those generic methods in order, and an assertion layer that compares results against limits pulled from a config file keyed by board variant. To add the new board, I'd write a new driver for the second multimeter implementing the same interface, add a new entry to the limits config, and the sequencer code wouldn't change at all.

   • Identifies coupling problem — Explicitly names that instrument commands, sequencing, and pass/fail logic are tangled in one function, which is why adding a variant is painful.
   • Proposes instrument abstraction layer — Suggests a driver/interface layer so a different multimeter model can be swapped in without touching the test sequence.
   • Externalizes limits — Suggests moving thresholds into a config file, dictionary, or per-board-variant data structure rather than hardcoding them.
   • Reasons about extensibility — Explains how the new structure lets a second board variant be added by changing config/driver only, not the core test logic.

   Where answers go wrong: Suggesting an if/else branch on board type inside the same function instead of abstracting instruments · Only moving the threshold to a variable without addressing the instrument coupling · Proposing a full rewrite without explaining what specifically breaks in the current design

3. 3You write a script that sends a SCPI query like MEAS:VOLT? to a power supply right after commanding it to a new setpoint, and reads back the result immediately. Most of the time it works, but occasionally the value returned looks like the previous setpoint, not the new one. What's going on, and how would you fix it?easy

   Answer this one, graded

   Or just show what a strong answer covers

   This smells like a race condition -- the supply needs time to slew its output to the new setpoint, and if I query for a measurement immediately after sending the SET command, I might catch it mid-transition or before it's even started, which explains why it's intermittent rather than every time. The fix is to synchronize instead of guessing: either poll an operation-complete style query supported by the instrument, poll the measurement until it stabilizes within a tolerance, or at minimum use a settle delay based on the supply's documented slew rate rather than an arbitrary sleep. I'd verify by logging the setpoint, the delay used, and the read value with timestamps across a few hundred runs to confirm the failure disappears and understand the margin, rather than just adding a delay and hoping.

   • Identifies timing/race condition as root cause — States that the supply hasn't finished slewing or processing the SET command before the MEAS query is sent, so the read races the setpoint change.
   • Proposes a synchronization mechanism — Suggests using an operation-complete check (e.g., *OPC?/*WAI equivalent), polling until the value stabilizes, or an appropriate settle delay rather than a fixed guess.
   • Notes verification, not just a fix — Mentions confirming the fix by logging timestamps/values across many runs, not just adding a delay and assuming it's solved.

   Where answers go wrong: Blaming the instrument as 'broken' without considering command/query ordering. · Adding an arbitrary large sleep() without justifying the delay or checking it against the instrument's actual slew time. · Assuming it's a wiring/connection issue rather than a timing issue since it's intermittent.

4. 4An automated test runs overnight on 500 boards and 3 of them fail. By morning the boards have been unpowered and swapped out already. What would you want your test script to have logged at the moment of failure so you actually have a shot at diagnosing what happened?easy

   Answer this one, graded

   Or just show what a strong answer covers

   A pass/fail bit alone tells me almost nothing after the fact, so I'd want the raw measured values for every parameter checked, not just the one that failed, plus which step in the sequence it failed at and what happened in the steps right before it. I'd log a timestamp, the board's serial number, the test station or fixture ID, and the test script/firmware version, since 'it always fails on station 3' or 'it started after we updated firmware' are common real explanations. If it's cheap to do, I'd also snapshot related signals -- like all rail voltages, not just the failing one -- since a marginal supply or a thermal issue often shows up as a pattern across multiple measurements rather than a single value.

   • Logs raw measured values, not just pass/fail — Specifies capturing the actual measured numbers (voltage, current, timing values) rather than a boolean result, so trends and margins can be examined later.
   • Captures identifying/traceability info — Mentions board serial number, station/fixture ID, timestamp, and test/firmware version so the failure can be tied to a specific unit and conditions.
   • Captures surrounding context at time of failure — Notes logging which step failed, what step preceded it, and ideally a snapshot of related signals (e.g., all rail voltages, not just the one that failed) to catch a symptom that fed into the failure.

   Where answers go wrong: Saying only 'log pass or fail' without raw data. · Not mentioning any way to trace a failure back to a specific board or station. · Assuming re-running the test later will reproduce the issue instead of capturing enough at the time to not need to.

5. 5Your automated test suite talks to an oscilloscope over VISA to capture a rising-edge screenshot for every board in a queue. About 1 run in 50, the script throws a VISA timeout error and the entire suite crashes, taking down the rest of that day's queue with it. How would you change the script's interaction with the instrument so a single timeout doesn't kill the whole run, and what would you want to know about why the timeout is happening in the first place?easy

   Answer this one, graded

   Or just show what a strong answer covers

   I'd wrap each instrument transaction -- the acquisition and read-back -- in its own try/except with a bounded retry, so a single timeout logs the failure, marks that board as inconclusive with its ID and timestamp, and lets the suite move on to the next board instead of crashing entirely. Before just bumping the timeout value, I'd want to know if the scope actually triggered on those runs -- checking trigger status or a raw waveform capture would tell me if this is a real intermittent signal problem versus the instrument occasionally being slow to respond to the VISA query. If it's a trigger issue, that's actually more interesting than a software bug -- it might be telling me something about board-to-board signal timing variation.

   • Wraps individual calls, not the whole run — Proposes try/except or equivalent error handling around each instrument transaction, not just a single outer guard around the whole suite.
   • Continues past the failure — Describes logging the failed board as inconclusive/needs-rerun and proceeding to the next board rather than aborting everything.
   • Investigates root cause of the timeout — Suggests checking whether the timeout correlates with the scope actually not triggering (real signal issue) versus the acquisition simply being slow, rather than just increasing the timeout blindly.
   • Logs diagnostic context — Mentions capturing board ID, timestamp, and instrument state at the time of the failure for later analysis.

   Where answers go wrong: Just increasing the timeout value without investigating why it's occasionally too short · Wrapping only the top-level loop instead of the specific instrument call that fails · Silently swallowing the error without logging enough to diagnose it later

6. 6A test in your suite fails about 1 time in 10 with no code changes and no obvious pattern. Walk me through how you'd figure out whether this is a flaky test -- something wrong in the test script or its assumptions -- versus a real intermittent hardware fault on the board.medium

   Answer this one, graded

   Or just show what a strong answer covers

   First I'd try to separate test-side and board-side variables: run the same failing test many times on the same board to see the failure rate, then run it on a couple of other boards and on a different fixture slot if possible. If the failure rate stays roughly the same regardless of which board is under test, that points at the script, fixture, or instrument timing rather than the board itself. On the script side, I'd specifically look for race conditions -- am I reading a measurement before the instrument or the board has settled, is there shared state left over from a previous test run, is a timeout too tight. On the hardware side, I'd want more instrumentation at the moment of failure rather than just re-running blind: extra logging of intermediate signals, or a scope with a trigger set up to catch the failing condition if it's a timing-sensitive signal. The goal is to get evidence pinpointing which side it's on before assuming either one.

   • Proposes running the same test on multiple boards/fixtures — Suggests running the failing test many times on the same board and on different boards/fixtures to see if the failure follows the board or is fixture/script-wide.
   • Considers test-script causes explicitly — Names concrete test-side suspects: race conditions/timing assumptions in the script, shared/stale state between test runs, or an instrument settling issue -- not just 'maybe it's the hardware.'
   • Adds instrumentation to catch it in the act — Proposes capturing more detail at/around the failure (e.g., scope trigger, extra logging, tighter timestamps) rather than just re-running and hoping, to distinguish the two hypotheses with evidence.

   Where answers go wrong: Jumping straight to 'it's probably a bad solder joint' without first ruling out the test script. · Re-running the test many times as the entire debugging strategy, without adding any new instrumentation. · Not considering that the fixture or connector itself, not the board, could be the intermittent element.

7. 7You're setting the pass/fail limits for an automated rail-voltage test: say the rail is specified at 3.3V and you're deciding how tight to make the acceptable range. Set it too tight and you'll fail good boards; set it too loose and you'll ship boards that fail in the field. How do you actually decide where to draw the line, and what does a 'PASS' on this test really guarantee?medium

   Answer this one, graded

   Or just show what a strong answer covers

   I wouldn't just pick a round number like ±5%; I'd look at what actually consumes that rail -- the datasheet operating range of the downstream ICs, plus any system-level margin analysis for things like brown-out thresholds -- and set the limit based on what those parts genuinely need, with some margin. I'd also account for the test instrument's own measurement uncertainty, because setting a limit tighter than what the DMM can reliably resolve just produces false failures. But I'd be upfront that a PASS on this test only proves the rail was within range at the moment I measured it, at room temperature, under whatever load condition the test applied -- it doesn't prove the rail stays in range over temperature extremes, under different load, or over the product's lifetime, unless I've specifically built tests for those conditions too. Test coverage is about being honest with yourself about what you didn't check.

   • Anchors limits to actual requirements, not arbitrary percentages — Explains that the limit should come from what downstream components can tolerate (their datasheet operating range) and/or system-level margin analysis, not just a default like '±5%.'
   • Accounts for measurement uncertainty — Mentions that the test instrument's own accuracy/resolution must be factored in so the limit isn't tighter than what can actually be measured reliably.
   • States clearly what a PASS does and doesn't prove — Explicitly states that passing this test only confirms the rail was within range at that moment, under those conditions (e.g. room temp, that load), and does not guarantee correctness across temperature, load, or over time -- naming at least one condition the test doesn't cover.

   Where answers go wrong: Picking a limit purely by convention (e.g., '±5% because that's typical') without tying it to what the board actually needs. · Ignoring the test instrument's measurement uncertainty when setting a tight limit. · Claiming a passing voltage test proves the board 'works' rather than that one specific parameter was in range under one specific condition.

8. 8You've automated a test for a sensor interface board. It sweeps three fixed input voltages spanning the sensor's specified range and checks the output reading is within tolerance at each point. Every board run through this test for the last month has passed. A batch of those boards later shows occasional bad readings in the field. What kinds of real defects could sail through 100% of this fixed-point test, and how would you change the test to catch them?medium

   Answer this one, graded

   Or just show what a strong answer covers

   Three fixed points only tell you the sensor is correct at those exact voltages -- a nonlinearity or dead zone between them, or right at the range edges, would pass every time. It's also a purely static test, so anything related to noise immunity, step response, or settling time after a fast input change would never show up, even though 'occasional bad readings' sounds exactly like a noise or transient issue. And it's a single pass at presumably nominal temperature and supply voltage, so temperature-dependent drift or an intermittent solder issue that only appears after thermal cycling or many operating hours wouldn't be caught either. I'd add a denser or randomized voltage sweep instead of three fixed points, add a step-input test to check settling and noise rejection, and run at least a subset of boards through a temperature or repeated-cycle stress test before calling the coverage adequate.

   • Identifies inter-point gaps — Explains that checking only 3 fixed voltages can miss nonlinear regions or discontinuities between those points.
   • Identifies missing dynamic behavior — Notes the test never exercises transient response, noise immunity, or settling time, which static checks can't reveal.
   • Identifies missing environmental/repetition coverage — Notes the test runs once at presumably room temperature/nominal supply, missing temperature, supply variation, or intermittent faults that show up over many cycles.
   • Proposes concrete test changes — Suggests specific fixes such as a denser or randomized voltage sweep, a step-response/noise test, or repeated/environmental stress runs, tied to the gaps identified.

   Where answers go wrong: Assuming the field failures must be a bad component rather than a coverage gap in the test · Suggesting only 'add more test points' without addressing dynamic or environmental gaps · Vague answer like 'test more thoroughly' without naming a specific defect class it would catch

9. 9A controller board passes its hardware-in-the-loop test suite, where a simulator feeds it synthetic sensor signals and checks its outputs against expected values, but fails when connected to the real plant. What kinds of problems can HIL testing structurally miss, and how would you change the HIL setup to catch more of them, given that you still can't test on the real plant every time?hard

   Answer this one, graded

   Or just show what a strong answer covers

   HIL only proves the controller behaves correctly with respect to the simulator's model of the plant, so anything the model gets wrong or leaves out is invisible no matter how many HIL cases pass. Concretely, simulated sensor signals are often cleaner than reality -- missing real noise, nonlinearity, or cross-talk -- and the simulator may not replicate real electrical loading on the controller's outputs, timing jitter, or physical dynamics like thermal lag or mechanical inertia that only show up with an actual plant attached. To close the gap without testing on real hardware every time, I'd try to capture real signals -- including their noise -- from an actual plant and replay those into the HIL rig instead of purely synthetic idealized waveforms, add realistic electrical loading to the simulator's outputs, and periodically run a smaller set of real-plant tests specifically to validate that the simulator's model still tracks reality, updating the model when it drifts. The point is treating the simulator itself as something that needs its own validation, not just trusting it because it's convenient.

   • Names specific gaps between simulated and real signals — Identifies concrete differences HIL can miss: real sensor noise/nonlinearity, electrical loading effects, timing/latency differences, or physical dynamics (thermal, mechanical) not captured by the simulator.
   • Explains why a passing HIL result doesn't guarantee real-world behavior — States clearly that HIL only proves correctness relative to the simulator's model of the plant, so any inaccuracy in that model is invisible to the test regardless of how many HIL cases pass.
   • Proposes concrete ways to close the gap — Suggests at least one actionable improvement: capturing and replaying real recorded signals (including noise) into the HIL rig, validating/calibrating the simulator's model against real plant data, adding realistic electrical loading, or running periodic limited real-plant tests to check HIL fidelity itself.

   Where answers go wrong: Saying HIL is 'basically the same as the real thing' and not identifying any structural gap. · Only suggesting 'just test on real hardware more' without addressing why that's often impractical or proposing how to improve HIL fidelity instead. · Vague answers like 'the simulation isn't accurate' without naming a specific mechanism (noise, loading, timing, thermal, etc.).

10. 10You build a hardware-in-the-loop rig for a motor controller: a host computer runs a plant model, drives DACs to feed the controller board synthetic sensor voltages, and reads the controller's PWM outputs back through ADCs to check they match the model's expected control response. The controller passes every HIL scenario you throw at it. What categories of real-world failure would this rig be structurally unable to catch, no matter how accurate your plant model is, and what's the biggest gap you'd close first?hard

    Answer this one, graded

    Or just show what a strong answer covers

    Because everything the controller sees comes through a DAC and everything we check comes back through an ADC, the rig can never exercise real sensor electrical characteristics like noise, EMI, or genuine wiring faults, and the DAC/ADC conversion and settling times add their own timing that's different from a real sensor, which can hide or fake timing-sensitive bugs. It also never touches real power-stage behavior -- there's no actual motor, so back-EMF nonlinearities, switching losses, thermal stress on the driver, or mechanical wear are completely invisible no matter how good the model is, because the model only encodes what we thought to represent. Passing every HIL scenario really just means the controller matches our model's predictions, not reality. The biggest gap to close first is the power stage and real actuator physics, so I'd add a power-hardware-in-the-loop stage or a bench test with an actual motor for final validation, and I'd log HIL predictions alongside real bench measurements over time to catch when the model itself has drifted from what the real system does.

    • Names signal-path limitations — Identifies that DAC/ADC bandwidth, conversion time, and settling introduce timing artifacts not present with real sensors, which can mask or fabricate timing bugs.
    • Names missing power/physical stage — Identifies that signal-level HIL never exercises real motor electrical behavior (back-EMF, switching, thermal) or mechanical faults, since it's not driving an actual motor.
    • Names model fidelity ceiling — Explicitly states that any real-world behavior absent from the plant model can never be caught, regardless of scenario count, because HIL only checks the controller against the model's own predictions.
    • Proposes a concrete next validation step — Suggests a specific follow-on such as power-hardware-in-the-loop or bench test with a real motor/actuator, plus logging to compare HIL predictions against real hardware behavior to track model fidelity.

    Where answers go wrong: Saying HIL is 'basically the same as real hardware' with no caveats · Only mentioning 'software bugs might still exist' without naming a concrete mechanism · Not distinguishing between the rig's electrical/bandwidth limits and the plant model's conceptual limits