research methods
Synthetic users
What are synthetic users, and can they replace real research?
Synthetic users are AI-generated stand-ins for research participants: prompt a model to "be" a persona and answer your interview or survey questions. They produce fluent, plausible, instant responses - which is exactly the danger. They reflect patterns in training data and the assumptions in your prompt, not the lived experience of a real person, so their confidence is untethered from truth.
Also known as: synthetic users, ai users, synthetic research, simulated users
Prefer to watch?
Watch the recap 0:58
The demo
Two answers to the same research question - one from a real person, one from an AI "synthetic user". Pick the synthetic one. (Harder than it sounds, which is the whole point.)
What this demo shows (text version)
For each round, a research question is shown with two answers: one written the way a real participant might (specific, a bit messy, sometimes contradictory or off-topic), and one the way an AI synthetic user tends to (fluent, tidy, plausible, agreeably on-theme). You guess which is the synthetic one, and the reveal points out the tells and keeps score. Most people find it genuinely hard.
That difficulty is the lesson: synthetic answers are convincing by construction, which is precisely why they're dangerous. They reflect training data and your prompt's assumptions rather than lived experience, confabulate specifics, and echo what you expect - amplifying confirmation and false-consensus bias. They may help you rehearse or draft a study, but any decision affecting real users needs real users. Plausible is not true.
Synthetic users are fast, cheap and dangerously convincing - they sound like research without being research. Because they're generated from training data and your own prompt, they tend to confirm what you already expect and invent specifics no one verified, with none of the friction, surprise or contradiction real people bring. Use them at most for warming up questions or rehearsing a study; never to make decisions that affect real users. Talk to real people.
Could you reliably tell the AI answer from the human one? Most people can't - synthetic responses are fluent and plausible by design. That's the trap: they pattern-match what a user "should" say and deliver it with total confidence, while telling you nothing a real person's messy, surprising, contradictory experience would. Plausible is not the same as true.
The appeal is obvious - instant, cheap, infinitely scalable "participants" with no recruiting. But a synthetic user has never used your product, felt your pricing or hit your bug. It generates the statistically likely answer given your prompt and its training, which means it reflects existing assumptions (including your own) and the internet's averages, not a specific human's reality.
The specific risks are sharp: it confabulates plausible detail that no one experienced; it amplifies the false-consensus and confirmation biases by agreeably echoing your framing; it flattens the variety, contradiction and surprise that make real research valuable; and it speaks with a confidence that invites teams to skip the messy, expensive step of talking to actual people.
There may be narrow, honest uses - pressure-testing or drafting interview questions, rehearsing a study, generating hypotheses to then check with humans. But the line is firm: synthetic users can supplement preparation, never substitute for evidence. Any decision that affects real users needs real users. If a finding only exists because an AI said so, you haven't found anything.