Personality and Individual Differences · Simplified
Reverse-Item Disaster
Two items were worded the other way round and nobody recoded them. Look at what that does to the scale.
As scored
A six-item scale, scored exactly as answered
A fictional persistence scale, six items answered from 1 to 5 by six respondents. Two items are worded so that agreeing means less persistence, and every answer has been added up at face value.
Key terms
- Reverse-keyed item
- An item worded so that agreement indicates less of the trait, not more. It has to be turned round before it is added to the others.
- Recoding
- Turning a reverse-keyed answer round by subtracting it from one more than the sum of the scale endpoints. On a 1 to 5 scale that is 6 minus the answer.
- Corrected item-total correlation
- How strongly one item agrees with the total of the other five. A positive value means the item is pulling the same way as the rest. A negative value means it is pulling against them.
- Cronbach's alpha
- A summary of how consistently the items of a scale agree with one another. It runs up to 1, and a value at or below zero means the items are not measuring one thing at all.
One bar per item, showing its corrected item-total correlation. Bars to the right of the line agree with the rest of the scale. Bars to the left pull against it.
What changed
One scale, three reliabilities
Nothing about the six people changed. Nobody answered anything differently. The only thing that changed was the arithmetic applied to two columns.
Key idea: An unrecoded reverse item does not sit quietly at the bottom of a reliability table. It correlates negatively with everything else, drags alpha to nothing, and makes the scale look as though it is not measuring one thing. It does not even stay in its own column: with both reverse items unrecoded, four of the six items correlated negatively with the rest, because the total they were being measured against was itself broken. Repairing the first item barely moved alpha, because the second was still pulling the other way, and it took both before the scale behaved. The item-total correlations are what pointed at which column to fix; alpha only said that something was wrong.
Two cautions. Six respondents is far too few for a real reliability estimate, and every figure here is arithmetic on an invented dataset chosen to make the effect visible. And reverse wording is a trade rather than a free fix: it does reduce agreement bias, and it also introduces comprehension difficulty, is answered inconsistently by people reading quickly, and routinely forms a method factor that can be mistaken for a second substantive dimension.
The longer version adds two further ways of getting the keying wrong, including one that leaves every total looking entirely plausible, and shows the effect on the respondents' totals and rank order alongside the scale statistics. It is at Reverse-Item Disaster in the main collection.