Personality & Individual Differences · Simplified
Chasing Alpha
There is a reliable way to push Cronbach's alpha above 0.9. It is also a reliable way to make the scale worse.
Step 1 of 2
First, a prediction
You have a ten-item scale of academic conscientiousness covering five areas: planning, orderliness, persistence, punctuality and resisting distraction. Its alpha is 0.82. You are going to swap items out for near-duplicates of items already in it, until alpha is above 0.94. Meanwhile the scale is being correlated with something it should predict, a tutor's rating of dependability.
What happens to that correlation as alpha climbs?
Step 2 of 2
Ten items, however you spend them
The scale stays ten items long. The only thing you change is how many of those ten are near-duplicates of one another, all asking about meeting deadlines in slightly different words.
Key terms
- Cronbach's alpha
- An index of how consistently a set of items covary. It is a function of two things only: how many items there are, and how strongly they correlate on average.
- Near-duplicate items
- Items whose wording is almost the same. They correlate highly with each other for a reason that has nothing to do with the construct.
- Unidimensionality
- Whether the items measure one thing. It is a different question from internal consistency, and alpha does not answer it.
- Construct validity
- Whether the scale measures what it claims to. Answering that needs evidence from outside the scale itself.
Two lines across every version of this scale. The upper line is Cronbach's alpha. The lower line is how well the scale predicts the tutor's rating. The upright marker is the scale you have built.
What this shows
Alpha went up because the scale got narrower
Key idea: Alpha is a function of how many items there are and how strongly they agree with each other. Near-duplicates agree with each other almost perfectly, because they are the same question twice, so they raise alpha very efficiently. None of that agreement is news about academic conscientiousness. Meanwhile every duplicate you added cost you a question about something else, so the scale that ends up with the best alpha is the one that asks about the least. That is why the two lines separate. A high alpha is consistent with an excellent scale and equally consistent with a scale that asks the same question nine times, and nothing inside the scale can tell those apart. Deciding between them takes evidence from outside it, which is what the lower line is.
Every number here is what a fictional correlation model implies for a population, not an estimate from a sample, so nothing carries the sampling error a real alpha or a real validity coefficient would. Three things this must not be taken to say. Low alpha is not thereby good: a scale whose items do not covary at all has its own problems, and the argument here is against treating alpha as a target rather than against alpha. Alpha is also not a test of unidimensionality, and a set of items measuring two distinct things can return a perfectly respectable alpha. And a single validity coefficient against one criterion is itself thin evidence; the lower line is here to show that it moves independently of alpha, not to be the last word on whether a scale is any good.
The longer version lets you choose which item to paraphrase, shows the full item pool and the inter-item correlation matrix, and adds two closing challenges. It is at The Alpha Trap in the main collection.