Clarity and Simplicity: Questions must be phrased in plain language to avoid ambiguity. Researchers should avoid 'double-barreled' questions (asking two things at once) and technical jargon that might confuse the respondent.
Operationalization of Aims: Every question should directly relate to the research aim. If the goal is to measure 'stress,' the questions must be designed to capture specific indicators of stress rather than general unhappiness.
Sequencing and Length: The order of questions should be logical, often starting with simple, non-threatening items to build rapport. The overall length must be balanced; too long, and participants suffer from 'respondent fatigue,' leading to careless answers.
| Feature | Open Questions | Closed Questions |
|---|---|---|
| Data Type | Qualitative (Words/Themes) | Quantitative (Numbers/Stats) |
| Reliability | Lower (Harder to replicate exactly) | Higher (Standardized and consistent) |
| Validity | Higher (Captures true depth/meaning) | Lower (Forces choices that may not fit) |
| Analysis | Time-consuming (Thematic analysis) | Fast (Statistical software/Graphs) |
Reliability in questionnaires is often achieved through standardization. Because closed questions use the same fixed options for every participant, the procedure is highly replicable, which is a hallmark of scientific rigor.
Validity refers to whether the questionnaire actually measures what it claims to measure. Open questions often have higher ecological validity because they allow participants to express their unique reality rather than picking from a list that might not represent their views.
Identify the Bias: When evaluating a study using questionnaires, always check for Social Desirability Bias. This occurs when participants answer in a way they think makes them look 'good' or 'normal' rather than being honest.
Justify Question Choice: In exam questions asking you to design a study, explain why you chose a specific question type. For example, 'I used a Likert scale to allow for statistical comparison of attitude intensity across the sample.'
Check for Ambiguity: Always review sample questions for 'leading' language. A question like 'Don't you agree that...' is biased and will lower the internal validity of the research.
A common mistake is assuming that more data is always better. Including too many open questions can lead to a massive amount of data that is nearly impossible to analyze objectively, often leading to researcher bias during the interpretation phase.
Another pitfall is the Acquiescence Bias, where participants have a tendency to agree with all statements regardless of their content. This is often countered by 'reverse-scoring' some items (phrasing some questions positively and others negatively).
Students often confuse 'anonymous' with 'confidential.' Anonymity means the researcher doesn't know who the participant is; confidentiality means the researcher knows but promises not to reveal the identity. Questionnaires are often preferred because they can be truly anonymous, increasing honesty.
