Academic Statistics · Module 09 of 11

Experimental and quasi-experimental analysis

This module covers the analysis of randomised and non-randomised intervention designs: pre-test and post-test comparisons, mixed ANOVA, ANCOVA, difference-in-differences, interrupted time series and propensity score methods. You will learn which analysis matches which design and how far each supports causal claims.

Watch the first lesson of every module free. The remaining lessons, exercises and live sessions open when you enrol.

What you will be able to do

  • Analyse a pre-test and post-test control group design with ANCOVA or a mixed ANOVA
  • Explain the assumptions of ANCOVA, including homogeneity of regression slopes
  • Describe difference-in-differences, interrupted time series and regression discontinuity designs and their key assumptions
  • Explain how propensity score matching or weighting addresses observed confounding

Key terms

  • Randomisation
  • ANCOVA
  • Mixed ANOVA
  • Homogeneity of regression slopes
  • Difference-in-differences
  • Parallel trends
  • Interrupted time series
  • Propensity score

Hands-on practice

Analyse a provided pre-test and post-test intervention dataset with ANCOVA in SPSS or R, check its assumptions and report adjusted means with an effect size.

Common pitfalls this module fixes

  • Using ANCOVA to 'correct' for pre-existing group differences in a non-randomised study; it cannot remove all confounding, so state the limitation clearly.
  • Testing baseline differences between randomised groups with significance tests; any differences are due to chance by design, so adjust for prognostic covariates you specified in advance instead.
  • Making causal claims from a quasi-experiment without testing its key assumption, such as parallel pre-treatment trends.