cognitive psychology
Confirmation bias
What is confirmation bias?
Confirmation bias is the pull to notice, trust and remember whatever fits what you already believe - and to wave away whatever does not. It quietly turns research into a hunt for "yes" rather than an honest test of "is this true?".
Also known as: confirmatory bias, my-side bias
The demo
A belief to test. Open the evidence in whatever order you like - then notice which pieces you reached for first, and which you would happily have skipped.
The belief: "Our new onboarding is working."
-
Plenty of five-star reviews praising how quick the setup feels. Nice to read - and exactly what you hoped to find.
-
Watching people who completed onboarding, it looks effortless. Of course it does - you only watched the ones who made it through.
-
People who quit partway through - and several stall on the very same step. This is the evidence that could prove you wrong.
-
Among the reasons people give for leaving: "couldn't get set up." The claim does not survive contact with this one.
Open some evidence. Notice whether you reach for what agrees with you first.
What this demo shows (text version)
A belief is stated - "our new onboarding is working" - alongside four pieces of evidence you can open. Two support the belief (positive reviews, sessions where users finished) and two challenge it (sessions where users dropped out, the reasons people cancelled).
Opening only the supportive evidence confirms the belief without ever testing it - the comfortable path confirmation bias pulls you down. Genuinely testing the claim means deliberately opening the evidence that could prove it wrong. If a belief only survives because you avoided looking, it was never really tested.
You opened the evidence that agreed with you and skimmed past the rest, then walked away sure. But the real test was the evidence you avoided - the session where it failed, the person who left. Confirmation bias feels good precisely because it only ever shows you that you were right.
It is most dangerous in user research, where it is easy to ask leading questions, recruit friendly users and read an ambiguous result as support. The guard is to decide in advance what would prove you wrong, then go looking for exactly that. If you only ever find confirmation, you were not testing - you were campaigning.
Watch it in your own analytics habits: filtering to the segment that worked, quoting the one glowing comment, re-running a test because the first result "must" be a fluke. A belief worth holding is one that survives a genuine attempt to break it.