Research Methods

Operationalisation Laboratory

Choose a construct, choose how you would measure it, and watch what your measure reaches, what it misses, and what else it quietly picks up instead.

Simulated — illustrative measures and facet models, no real instrument

Learning objective

By the end you should be able to say why an operational definition is a claim about a construct rather than a neutral translation of it.

About 20 minutes. Nothing you do here is saved or sent anywhere.

The facet lists are a model, not a fact

Each construct here is broken into five named facets so that coverage can be shown. Those lists are teaching simplifications. Which facets a construct has — and whether it has facets at all — is itself contested, and the coverage figure inherits every assumption in the list it is measured against.

  1. Answer the opening question to unlock the laboratory.
  2. Choose a construct, then tick the measures you would use.
  3. Watch the coverage map at the top of the stage as you tick.
  4. Read what each measure also picks up, then try the challenge.

First, a judgement

A study needs a measure of academic engagement. Which single measure would come closest to capturing it?

Challenge — the intervention that measures itself

A department introduces weekly online quizzes to raise academic engagement. Engagement is measured before and after by hours logged on the virtual learning environment. Hours rise by 40%.

What has been shown? Select every statement you would defend.

What this demonstrates

An operational definition is a claim, not a translation

Writing "engagement was operationalised as hours logged on the virtual learning environment" looks like a technical step. It is an argument: that the hours stand in for the construct well enough for the claim being made. Every measure in this laboratory makes such an argument, and every one of them can be attacked.

Two ways a measure goes wrong, and they pull in opposite directions

Construct under-representation is missing parts of the construct: attendance records say nothing about whether anyone thought about the lecture. Construct-irrelevant variance is picking up things that are not the construct: attendance records also measure commuting distance, timetable clashes and whether the lecture was recorded. Fixing one usually worsens the other, which is why measurement is a set of trade-offs rather than a problem with a solution.

Reactivity is measurement changing the thing measured

An observer in a seminar changes the seminar. A prompt asking how engaged you feel interrupts what it is asking about. These are not flaws to be designed away; they are what it costs to get close to a construct that lives inside an activity.

Combining measures buys coverage, and does not buy the construct

Three measures reach more facets than one and give you somewhere to stand when a critic says your measure was the wrong one. They also cost more, burden participants more, and can disagree — which is information, not a nuisance. What they never do is turn into the construct, because the construct is a theoretical idea and the measures are records of things people did.