Academic Statistics · Module 10 of 11

Data quality and reliability testing

This module covers data screening and cleaning, missing data mechanisms and handling, outliers, and reliability evidence: Cronbach's alpha, McDonald's omega, test-retest reliability with the intraclass correlation coefficient and inter-rater agreement with Cohen's kappa. Clean data and credible reliability evidence are preconditions for trustworthy results.

Watch the first lesson of every module free. The remaining lessons, exercises and live sessions open when you enrol.

What you will be able to do

  • Screen a dataset for entry errors, impossible values and patterns of missingness
  • Distinguish MCAR, MAR and MNAR and choose between complete-case analysis and multiple imputation
  • Compute and interpret Cronbach's alpha and McDonald's omega and explain the limitations of alpha
  • Assess test-retest and inter-rater reliability with the ICC and Cohen's kappa

Key terms

  • Data screening
  • Missing completely at random
  • Missing at random
  • Multiple imputation
  • Cronbach's alpha
  • McDonald's omega
  • Intraclass correlation
  • Cohen's kappa

Hands-on practice

Clean a provided survey dataset, document every change in a log and report alpha and omega for each scale in SPSS, JASP or R.

Common pitfalls this module fixes

  • Using listwise deletion by default; examine the missingness pattern and use multiple imputation or maximum likelihood when data are not MCAR.
  • Reporting a single alpha for a multidimensional scale; report reliability for each subscale and check dimensionality first.
  • Forgetting to reverse-score negatively worded items before computing reliability; a negative item-total correlation is the usual warning sign.