UX by Example Buy me a coffee

research methods

Leading questions

A question worded so it nudges you toward a particular answer - through a loaded word, a built-in assumption, or a scale that only points one way. It measures the wording, not the truth.

The demo

Four quick questions about a to-do app called Tasklet. Answer honestly - then we will look back at how each one was worded.

1. How much easier did Tasklet make your week?

Loaded scale + presupposition

You answered:

Every option already agreed it was easier - there was no "no change" and no "harder". The question settled the verdict; you only got to choose the size.

Asked neutrally: "Did Tasklet change how easy your week felt? Easier / No change / Harder."

2. Don't you agree that Tasklet's clean design makes it a joy to use?

Acquiescence bias + double-barrelled

You answered:

"Don't you agree" fishes for a yes, and the only options were shades of yes. It also bundles two claims - the design and the joy - so you can't praise one while faulting the other.

Asked neutrally: "How would you describe Tasklet's design?" - and ask about how it feels to use as a separate question.

3. How frustrating was the old, clunky tool you used before Tasklet?

Loaded words

You answered:

"Old" and "clunky" put the answer in your mouth, and the scale only measured how frustrated, never whether. Maybe the previous tool was perfectly fine.

Asked neutrally: "How did you find the tool you used before Tasklet?"

4. How fast and reliable is Tasklet's sync?

Double-barrelled

You answered:

Two questions wearing one coat. If sync is quick but quietly drops tasks, there is no honest answer here - you are forced to average two things that each deserve their own.

Asked neutrally: "How fast is Tasklet's sync?" and, separately, "how reliable is it?"

What this demo shows (text version)

Four survey questions about a fictional to-do app, each rigged to push a positive answer. Question 1 presupposes the app made your week easier and offers a scale that only runs from "a little easier" to "much easier". Question 2 opens with "don't you agree" (an invitation to say yes) and bundles two claims into one. Question 3 calls the previous tool "old" and "clunky" and only measures how frustrating it was. Question 4 asks about "fast and reliable" together, so a quick-but-flaky answer has nowhere to go.

After you answer, each question reveals its technique and a neutral rewrite. The point: whatever you picked, the wording had already leaned your answer - which is why a leading question tells you how you asked, not what people actually think.

Whichever options you picked, every one of them leaned the same way - so the questions had half-answered themselves before you arrived. That is a leading question, and you just answered four.

My rule for writing a survey question: if you can already guess the answer you are hoping for, scrap it and start again. A leading question doesn't gather data, it manufactures it - and the more flattering the result, the less it is worth.

The tell is usually the scale. "How delighted were you?" with options running from "delighted" to "very delighted" can only ever flatter you. Offer the whole range, including the answer you would rather not hear, or don't bother asking.