Survey Architecture & Bias-Free Question Design

Surveys are a quantitative research method. They are exceptional at answering 'How many?' and 'How frequently?', but poor at answering 'Why?'. If you do not yet understand the problem space, launching a survey will only measure prevalence across flawed assumptions.

When measuring user attitudes, rating scales must be balanced, symmetrical, and fully anchored:

• Balanced Scale: An equal number of positive and negative choices with a neutral midpoint (e.g., Very Dissatisfied, Somewhat Dissatisfied, Neutral, Somewhat Satisfied, Very Satisfied).
• Unbalanced Bias (Fatal Defect): Offering Okay, Good, Great, Amazing, Incredible. This artificially forces positive responses.
• Clear Anchors: Label every scale point with words. Number-only scales (1 to 7) force participants to guess what a '4' or '5' means.

A double-barreled question bundles two separate concepts into a single question with one answer choice:

• 'How satisfied are you with the speed and accuracy of the search results?'

If the search was blazing fast but returned completely irrelevant results, how should the participant respond? Their response is mathematically uninterpretable.

The Solution: Split every compound thought into separate, single-focus questions:

1. 'How satisfied are you with the speed of search results?'
2. 'How satisfied are you with the relevance of search results?'