Usability testing questions that get honest, useful answers

Usability testing questions are the tasks, prompts and follow-ups a moderator uses while watching real people use a product, to find where it gets in their way. Good ones ask people to do something and describe what they see. Weak ones ask people to agree. This guide lists the usability testing questions we use before, during and after each session, and the usability testing methods they suit. It also covers the mistakes that lead users to tell you what you want to hear.

The questions come from the same bank we use for website usability testing with startup teams. They work just as well on mobile apps, clickable prototypes and paper sketches.

What makes a good usability testing question

The international standard on usability, ISO 9241-11, defines it as the extent to which a product can be used by specified users to achieve specified goals. It judges that by effectiveness, efficiency and satisfaction in a specified context of use. That gives your questions three jobs: find out whether people can do the task, how much effort it takes them, and how they feel about it afterwards.

Four rules keep the answers honest:

• Tasks over opinions. "Find last month's invoice and download it" teaches you more than "What do you think of this page?".
• Past over future. Ask what people did last time, not what they might do one day.
• Open over closed. Questions that start with what, how or tell me get stories. Questions that can be answered with yes or no get a yes.
• Neutral over hopeful. Never name the button you want clicked or hint at the answer. The Digital.gov usability test guide gives an example. Ask how someone would view the status of their application, rather than which link in the header they would click to log in.

Usability testing questions to ask before the session

Before the session you need two sets of questions: screener questions to recruit the right people, and warm-up questions that put them at ease and give you context.

Screener questions

Recruit people who really do the task your product supports. Digital.gov's guidance warns against testing with experts, friends, family or close colleagues of the product team. They know too much about the product, or they like you too much to be blunt.

• When did you last [do the task]? What did you use?
• How often do you do it in a typical month?
• Which tools do you use for it today?
• What is your role, and who else is involved in the task?
• Do you use any assistive technology, such as a screen reader or screen magnification?
• Have you taken part in research about this product before?

Warm-up questions

Open by saying that you are testing the product, not the person, and that there are no wrong answers. Then ask a few easy questions you can use to make the tasks realistic.

• Tell me a little about your work and what a normal week looks like.
• Where does [the task] fit into that week?
• Talk me through the last time you did it.
• What was frustrating about it, if anything?
• How do you usually learn a new tool: on your own, from a colleague or from help pages?

User testing questions during the tasks

During the session most of your words are task prompts, and everything else is a short, neutral probe. Ask people to think aloud, describing what they look at and what they are trying to do, then stay quiet and watch.

First look

• Without clicking anything, what do you think this page is for?
• What would you expect to be able to do here?
• Where would you start?

Task prompts

Write each task as a goal with a reason, never as a set of instructions. The examples below are for a hypothetical invoicing app.

• A client says they never received last month's invoice. Send it to them again.
• You have just hired a bookkeeper. Give them access to your account.
• Your business has moved. Update the address that appears on your invoices.
• You want to stop paying monthly. Find out what the annual option would cost.

Neutral probes

• What are you looking for right now?
• What did you expect to happen when you clicked that?
• What is going through your mind?
• You paused there. What were you thinking?
• Can you say more about that?
• How would you know you were finished?
• What would you do next if I were not here?

When someone gets stuck

Let people struggle for a while before you step in, because the struggle is the finding. If they are completely stuck, ask "What would you try next?" before you help. If you do help, note it and count that task as a failure. The GOV.UK Service Manual advises interrupting sparingly, for example to explore something interesting or to help someone who is completely stuck.

Questions to ask after the tasks

The debrief is where you learn how people felt and what they compared you with. Keep it short, and keep it neutral.

• How did that compare with the way you do it today?
• Which part was hardest? What made it hard?
• Was anything missing that you expected to find?
• If you could change one thing, what would it be, and why?
• Who else at your company would need to use this, and what would they think?
• What should we have asked that we did not?

If you need a number to compare rounds, finish with a standard questionnaire such as the System Usability Scale, and use the same one every round. A single score will not tell you what to fix, but it shows whether a round of fixes moved things.

Questions that lead users to agree

Most bad usability testing questions are not rude. They are hopeful. They tell the participant what the team wants to hear, and polite people oblige. Swap them like this:

Two habits do as much damage as bad questions: explaining the interface while someone is using it, and nodding when they get it right. Both teach the participant how to pass your test instead of showing you where the product fails.

Usability testing methods, and when to use each

The questions above work across most usability testing methods, but the balance between tasks, probes and debrief shifts with the method. These are the main types of usability testing and what each one is good for.

• Moderated testing. A facilitator runs the session live, in person or remotely. Use it when you need to ask why, such as with early prototypes and complex flows.
• Unmoderated testing. Participants complete tasks alone while a testing tool records the screen. Use it for quick checks of simple flows, and write every task and question in advance, because nobody is there to clarify them.
• Remote or in person. Remote sessions reach more people and let them use their own setup. In person shows you the room, the device and the interruptions. The GOV.UK Service Manual notes that people who use assistive technology are usually best off using their own devices.
• Qualitative or quantitative. Qualitative tests with a few people find problems. Quantitative tests with many people measure success rates and time on task, which helps you compare versions or track progress over time.
• Explorative, assessment and comparative tests. Explorative tests probe early concepts, assessment tests check how well a working product performs, and comparative tests put two designs side by side.
• Focused methods. Tree testing checks whether people can find things using only your navigation labels, with no visual design. First-click tests check whether the first click on a screen goes to the right place.

How many participants, tasks and minutes

Small rounds find most problems. In Robert Virzi's 1992 experiments, four or five participants found 80% of the usability problems, and the most severe problems tended to appear in the first few sessions. A year later, Jakob Nielsen and Thomas Landauer modelled problem discovery across 11 studies and, for a medium-sized example, found that the benefit-to-cost ratio peaked at four evaluations. For a startup, that means testing with about five people from each distinct user group, fixing what you find, then testing again.

Keep each session between 30 and 60 minutes, the usual range in the GOV.UK Service Manual, with three to six tasks. The same guidance suggests no more than six one-hour sessions in a day, with at least 15 minutes between them.

Before you record anything, tell people what you will record, who will see it and why, and get their consent. The Office of the Privacy Commissioner of Canada asks organizations to describe their purposes in meaningful language and to make the key points easy to find. It warns against vague phrases such as "service improvement".

Turning answers into fixes

Debrief with the observers straight after each session while it is fresh, then look for patterns across participants. A problem that several people hit outranks one that a single person mentioned. Rank findings by how badly they block the task and how many people they affect, much as our UX audit template scores severity and reach. Then turn the top problems into backlog items with the clip or quote attached.

Testing tells you whether people can use what you built. It does not tell you what to build next; for that, use product discovery techniques such as problem interviews and demand tests. If you have no users to test with yet, an expert review is a cheaper first pass, and our UI/UX audit services cover onboarding, key flows, accessibility and conversion.

Frequently asked questions