Research Methodology · Module 05 of 10

Instrument design and validation

This module covers how to operationalise constructs into items, write unambiguous questions and response scales, and establish validity and reliability through expert review, cognitive interviewing, piloting and factor analysis. A poorly validated instrument undermines the credibility of every result built on it.

Watch the first lesson of every module free. The remaining lessons, exercises and live sessions open when you enrol.

What you will be able to do

  • Operationalise a construct into measurable items with an appropriate response scale
  • Assess content validity using expert panels and a content validity index
  • Plan exploratory and confirmatory factor analysis to examine construct validity
  • Assess internal consistency with Cronbach's alpha and McDonald's omega and explain their limits

Key terms

  • Operationalisation
  • Likert scale
  • Content validity index
  • Construct validity
  • Exploratory factor analysis
  • Confirmatory factor analysis
  • Cronbach's alpha
  • Back-translation

Hands-on practice

Revise a provided draft questionnaire, then compute Cronbach's alpha and omega for one subscale on pilot data in SPSS, JASP or R.

Common pitfalls this module fixes

  • Treating a high Cronbach's alpha as proof of unidimensionality; alpha rises with the number of items and assumes tau-equivalence, so check structure with factor analysis and report omega as well.
  • Running EFA and CFA on the same sample; confirm the structure on an independent sample or a split of the data.
  • Adopting a published scale in a new language or population without adaptation; translate, back-translate and re-test validity in your context.