Recruiting Representative Users & Screener Design

The most brilliant interview script or [usability test](/glossary/usability-testing) protocol is worthless if you test with the wrong people. Convenience sampling—such as interviewing teammates, friends, or random passersby—introduces catastrophic bias because internal colleagues understand the underlying system architecture and forgive usability flaws that real users cannot navigate.

When recruiting participants, you must screen out candidates who do not fit your target criteria. However, if your screener asks:

• 'Do you manage a cloud infrastructure team of 10+ engineers?' [Yes / No]

Participants incentivized by honorariums or gift cards will instantly deduce that 'Yes' qualifies them for the paid session.

The Non-Leaking Formula: Use multi-select or multi-choice lists where the qualifying criteria are embedded alongside plausible foils:

• 'Which of the following describes your primary daily responsibility?'
  • Sales pipeline management
  • Cloud infrastructure engineering (Qualifying)
  • Digital marketing campaigns
  • Legal compliance & contracts
  • None of the above

One of the most frequently cited principles in UX research is Jakob Nielsen's discovery that 5 users uncover approximately 85% of usability defects in a qualitative test.

Why does this hold?

• The first participant uncovers ~31% of usability problems.
• The second participant repeats much of what the first said, adding novel insights.
• By the fifth participant, you enter diminishing returns where observers see the same core friction points repeated.