research methods
Usability testing vs heuristic evaluation
Usability testing or heuristic evaluation - which finds more problems?
Two ways to find usability problems. Usability testing watches real people attempt tasks and surfaces what actually trips them up. Heuristic evaluation has experts inspect the interface against known principles. One reveals reality; the other predicts it - and they catch strikingly different things.
Also known as: heuristic evaluation vs usability testing, expert review vs user testing
The demo
Five real problems hiding in a checkout. Switch between the two methods and see which each one would catch - and, more tellingly, which it would miss entirely.
Switch methods and watch the coverage change.
What this demo shows (text version)
Five usability problems in a checkout: low-contrast labels and inconsistent button styles (which experts catch against guidelines), users expecting "Basket" instead of "Bag" and missing the checkout button below the fold (which only show up when real people try the tasks), and a missing error message on failed payment (which both methods catch).
Choosing heuristic evaluation highlights the three an expert review would find and greys out the two only real users reveal; choosing usability testing does the reverse. Each method catches three of five and misses two - different two. The demonstration is that the methods are complementary: run a cheap heuristic pass first, then usability testing for what only real use exposes.
Run the experts and they flagged the contrast and consistency issues fast - but sailed past the wrong word and the missed button that only real users hit. Run the test and you caught those, but not the principle violations the experts spotted in minutes. Neither method saw everything; together they did. That's the whole case for using both.
Heuristic evaluation is fast, cheap and needs no participants, so it's brilliant early and for catching textbook violations - but it's expert opinion, prone to flagging "problems" real users never notice and missing surprises no principle predicts. Usability testing finds the real, often unguessable problems, but costs time and recruiting and only covers the tasks you test.
So sequence them: a heuristic pass first to clear the obvious faults cheaply, then usability testing to find what only real use reveals - don't waste a paid session on a bug an expert would have caught for free. They are complements, not rivals, and "we did a heuristic review" is never a substitute for watching someone actually use the thing. See [usability testing](/entries/usability-testing/) and [heuristic evaluation](/entries/heuristic-evaluation/).