How many survey questionnaires have you completed during your career? Probably quite a few. Some of the survey items you completed likely used the bipolar Likert Scale, with agreement options on one side, disagreement options on the other side, and neutral in the middle. You may also have designed your own survey questionnaires using the Likert Scale or similar styles—for example, satisfaction options on one side and dissatisfaction option on the other.
When using these scales, there are two ways of listing response options—in ascending or descending order—depending upon whether you list the most positive response option on the left or right side of the response scale. In the following survey item, for example, which order of the Likert Scale options would you use? Does it matter?
I feel valued at work.
○ Strongly agree ○ Agree ○ Neutral ○ Disagree ○ Strongly disagree (horizontal descending order)
I feel valued at work.
○ Strongly disagree ○ Disagree ○ Neutral ○ Agree ○ Strongly agree (horizontal ascending order)
Research has shown that whether you use a descending or ascending scale, the order of response options can make a difference in the survey results. Before we talk about the effects of these, we need to understand some aspects of human psychology and behavior.
Think about this—you’re ordering food at a restaurant, and the server is telling you several available salad dressing or wine options. You listen and, despite all the choices, find yourself selecting one of the last ones listed. This is called a recency effect—people tend to select the options that they see or hear at the end of the response option list. The recency effect tends to happen when options are presented orally.
When the response options are written in self-administered survey questionnaires, a few outcomes can occur. The primacy effect happens when survey respondents select the options that are presented at the beginning of the response option list listed horizontally (as shown above). Similarly, when the text is written from left to right, survey respondents tend to select what’s written on the left side of the response scale—what’s known as left-side selection bias. Survey respondents also tend to agree rather than disagree with the statement provided—an outcome known as acquiescence bias or yea-saying bias. This choice is also related to respondents’ desires to select options they think are more socially acceptable (often the positive ones) instead of expressing their true thoughts. Social-desirability bias plays into this scenario. Survey respondents also tend to select options that are simply satisfactory (good enough)— satisficing behavior—to minimize their cognitive effort in completing the survey.
When given written self-administered survey items with response options presented horizontally, survey respondents usually select among the ones that:
- they read first on the left side (primacy effect and left-side selection bias)
- indicate agreement, which is more socially acceptable and does not require too much thinking to decide (acquiescence bias, social-desirability bias, and satisficing).
So, what would happen when you use the Likert or a Likert-type scale ascending or descending response options? Research has shown mixed results:
1. Descending-ordered response options result in statistically significantly more positive ratings.
2. Different response orders do not make a statistically significant difference in survey ratings.
Between the two, however, the first type of results seem to be more frequent. Why could that be?
As shown in Table 1, the left-side selection bias may apply to ascending- and descending-ordered response scales. When acquiescence bias, social desirability bias, and satisficing behavior are applied to descending-ordered responses scales, you likely see more positive average survey ratings.
1. Clear 10 9 8 7 6 5 4 3 2 1 0 Unclear
2. Clear 0 1 2 3 4 5 6 7 8 9 10 Unclear
3. Unclear 10 9 8 7 6 5 4 3 2 1 0 Clear
4. Unclear 0 1 2 3 4 5 6 7 8 9 10 Clear
Research found that the first one that puts positive wording (“Clear”) and the highest numerical value (“10”) on the left side resulted in the highest average score.
Some research, though, showed no statistically significant differences between the ascending- and descending-ordered response scales on the average survey ratings.
Also, what if you present Likert Scale options (as shown below) or other Likert-type scale response options vertically? Will you see top selection bias (similar to left-side selection bias)?
I feel valued at work. ○ Strongly agree (vertical descending order) |
I feel valued at work. ○ Strongly disagree (vertical ascending order) |
Again, research shows mixed results. Some studies have not clearly shown top selection bias from vertically presented response options, but some did.
What does this all mean to you as a practitioner when designing your survey items with Likert-type response scales? You intend to use your survey data as part of your evidence-based practice; therefore, it is important that you collect data that is as accurate as possible. Now that you know descending-ordered response scales can give you inflated, inaccurate data, here are some suggestions:
- If you suspect that your survey items may be prone to showing inflated data when using descending-ordered response scales, use ascending-ordered response scales.
- Provide clear directions about the order of response options because some survey respondents may use the “first-is-best” heuristic method and assume the far-left option is the most positive option without carefully reading it.
- If you choose to use descending-ordered response scales, you may want to make your survey questionnaire short because fatigued survey respondents (due to lengthy survey items) are more prone to response-order effects.
