How-to articleMethodology & analyticsEconomics & finance

Econometrics or machine learning? Prediction, causal inference and combining both in a PhD

Neither econometrics nor machine learning is becoming obsolete. They answer different questions: one is built to estimate parameters, often causal ones, and the other to predict well out of sample. This guide explains the difference, shows where each fits in a doctoral project, and introduces methods that use both.

What you will learn

  • Explain the difference between estimating a parameter and predicting an outcome
  • Choose between econometric and machine learning methods for a given research question
  • Describe how double/debiased machine learning uses prediction to support causal estimates
  • Evaluate prediction models honestly, avoiding leakage and time-series pitfalls
  • Report a combined analysis so that examiners can judge both prediction and inference

What you will unlearn

  • Machine learning will replace econometrics.Machine learning predicts well but does not by itself identify causal effects, which remains the core task of econometrics.
  • Econometrics ignores prediction and machine learning ignores theory.Econometricians forecast routinely, and good machine learning work uses domain knowledge to choose features and evaluation.
  • A model that predicts well has the right coefficients.Many different models predict equally well with very different coefficients, so predictive accuracy says little about parameters.
  • Feature importance tells you what causes the outcome.Importance scores describe what the model relies on, not causal effects in the world.

Skills you will practise

  • Framing prediction versus inference
  • Identification strategy
  • Cross-validation
  • Double machine learning
  • Leakage checks
  • Model interpretation

The real difference: what question are you asking?

A common question is whether econometrics or machine learning is becoming obsolete. Eight years on, the answer is clear: neither is. The useful distinction is not between old and new tools but between two kinds of question. A prediction question asks what the value of an outcome will be for a new case. An estimation question asks how an outcome would change if one variable changed, holding others fixed. Mullainathan and Spiess (2017) put this as the difference between predicting y and estimating beta.

Econometrics developed to estimate beta credibly from observational data. Its central concern is identification: the conditions under which a coefficient can be read as a causal effect rather than a correlation (Angrist & Pischke, 2009; Wooldridge, 2010). Machine learning developed to predict y accurately for cases the model has not seen. Its central concern is generalisation: avoiding a model that fits the training data closely but fails on new data. Methods such as random forests trade interpretability of individual coefficients for predictive accuracy (Breiman, 2001).

Econometrics versus machine learning
Figure 1. Econometrics and machine learning.

How the two traditions compare

Econometrics and machine learning compared
FeatureEconometricsMachine learning
Main targetA parameter, often a causal effectOut-of-sample prediction of an outcome
Role of theoryGuides specification and identificationGuides features and evaluation; model form is learned from data
Typical modelsLinear and nonlinear regression, instrumental variables, panel and time-series modelsRegularised regression, tree ensembles, boosting, neural networks
How success is judgedUnbiased or consistent estimates with valid standard errorsError on held-out data, such as mean squared error or AUC
Main riskOmitted variables, reverse causality, weak instrumentsOverfitting, leakage between training and test data
InterpretationCoefficients have a stated meaningParts of the model may be hard to interpret

The table simplifies. Econometricians forecast, and time-series econometrics is a forecasting discipline with its own evaluation methods (Hyndman & Athanasopoulos, 2021). Machine learning researchers increasingly care about interpretation. But the difference in targets explains most disagreements between the two communities.

Why prediction alone cannot answer causal questions

A model that predicts house prices well may use the number of bathrooms, the floor area and the postcode. It does not follow that adding a bathroom raises the price by the amount the model implies. Predictors are often correlated, so many different models predict equally well while assigning very different weights to each variable (Mullainathan & Spiess, 2017). That is harmless for prediction and fatal for estimation.

The same caution applies to the explanation tools now common in machine learning. Shapley-value methods describe how much each feature contributed to a model's prediction (Lundberg & Lee, 2017), but those contributions describe the model, not the economy. For high-stakes decisions, some researchers argue for models that are interpretable by design rather than black boxes explained after the fact (Rudin, 2019). Causal claims still need a research design that rules out alternative explanations (Antonakis et al., 2010).

Econometrics has its own history of fragile results. Leamer (1983) showed how conclusions could change with small, undisclosed choices of specification, which helped push the field towards credible research designs and robustness checks.

Combining both: machine learning inside econometrics

Since 2017 the most productive work has used machine learning for the parts of a model where prediction is what is needed. The best-known example is double/debiased machine learning (Chernozhukov et al., 2018). Suppose you want the effect of a treatment D on an outcome Y, controlling for many variables X. The method uses machine learning to predict Y from X and D from X, takes the residuals of each, and regresses one set of residuals on the other. Two features make it work: the estimating equation is constructed so that small errors in the machine learning predictions have little effect on the treatment estimate (Neyman orthogonality), and the predictions for each observation are made on data that did not include it (cross-fitting). The result is an estimate of the treatment effect with valid confidence intervals, provided the usual identifying assumption holds: that X captures all confounders.

Machine learning can also help econometric work in other ways: selecting among many candidate controls, building a first stage for instrumental variables, estimating how effects vary across groups, and creating new variables from text, images or satellite data (Mullainathan & Spiess, 2017). In the other direction, econometric thinking about identification and inference makes machine learning research more credible. Packages implementing double machine learning are available in R and Python.

Worked example (illustrative)

A doctoral researcher in development economics has administrative data on 20,000 small firms, including whether each received a government training grant, its revenue growth and 150 firm characteristics. She has two questions.

Two questions, two approaches (illustrative)
QuestionTypeSuitable approachWhat to report
Which firms are most likely to fail within two years?PredictionGradient boosting or regularised logistic regression with cross-validationHeld-out performance, calibration, comparison with a simple baseline
Did the grant raise revenue growth?Causal estimationDouble machine learning with the 150 characteristics as controls; or a regression discontinuity if eligibility used a score cut-offEffect estimate and confidence interval, identifying assumption, sensitivity checks

For the first question she reports which model predicts best and how much better it is than a simple rule. She does not interpret its feature importances as causes. For the second she states the assumption that the controls capture everything that drove both grant receipt and growth, and she discusses what could violate it. If the grant was allocated by a score with a cut-off, a regression discontinuity design gives a more credible estimate for firms near the cut-off, and she can use the double machine learning estimate as a comparison.

Evaluating prediction models honestly

Prediction claims fail in a predictable way. Kapoor and Narayanan (2023) documented data leakage, in which information from the test data reaches the model during training, across many fields that use machine learning, producing inflated performance claims. Economic and financial data are especially exposed because they are ordered in time. Random cross-validation can leak future information into the past, so time-series work needs evaluation designs that respect order (Bergmeir & Benítez, 2012; Hyndman & Athanasopoulos, 2021).

Always compare a machine learning model with a sensible econometric benchmark. In forecasting South African inflation, for example, Botha et al. (2023) compared machine learning methods applied to a large set of predictors with conventional benchmark models, which is the comparison an examiner will expect.

Choosing for your thesis, and what to report

Start from the sentence that states your research question, not from the method. If the question contains words such as "effect of", "impact of" or "does X raise Y", you are estimating a parameter, and the first chapter of your methodology should be the identification strategy: the source of variation in X that you treat as good as random, and the assumptions that make it so. Machine learning may then help with controls, heterogeneity or measurement, but it does not replace that argument. If the question asks "which cases", "how many" or "what will happen next", you are predicting, and the methodology should centre on the data split, the benchmark and the loss function that reflects the real cost of errors.

Many economics and finance theses contain both kinds of question, as in the worked example. Keep them in separate sections with separate evaluation criteria, so that an examiner never has to wonder whether a coefficient is being read as a cause or a forecast is being judged by its standard errors. Whatever you choose, report the following.

  • For estimation: the estimand, the identifying assumption, the estimator, standard errors and how they were clustered, robustness checks and the main threats that remain.
  • For prediction: the outcome and prediction horizon, how data were split, the benchmark, the tuning procedure, held-out performance and any checks for leakage.
  • For combined methods: which parts of the model use machine learning, which learners and settings were used, how cross-fitting was done, and how sensitive the estimate is to the choice of learner.
  • For both: software, versions, random seeds and code, so that the results can be reproduced.

Related pages: the steps of an empirical econometric study, how to choose a machine learning algorithm, what big data and machine learning can and cannot show in empirical finance and reporting regression analyses.

Common mistakes

  • Reading coefficients or feature importances from a prediction model as causal effects.
  • Using random cross-validation on time-ordered data, which leaks future information.
  • Reporting only the best of many models without saying how many were tried.
  • Comparing a tuned machine learning model with an untuned or weak benchmark.
  • Assuming that using machine learning for controls removes the need for an identification strategy.
  • Omitting the code and settings needed for another researcher to reproduce the results.

Key takeaways

Your next step

Want to apply this to your own study?

  1. 1Tell us your topic, data and deadline
  2. 2A statistician reviews your work with you
  3. 3You get a plan and a quote
Quantitative analysis

References

  1. Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press. https://doi.org/10.1515/9781400829828
  2. Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(6), 1086–1120. https://doi.org/10.1016/j.leaqua.2010.10.010
  3. Bergmeir, C., & Benítez, J. M. (2012). On the use of cross-validation for time series predictor evaluation. Information Sciences, 191, 192-213. https://doi.org/10.1016/j.ins.2011.12.028
  4. Botha, B., Burger, R., Kotzé, K., Rankin, N., & Steenkamp, D. (2023). Big data forecasting of South African inflation. Empirical Economics, 65(1), 149-188. https://doi.org/10.1007/s00181-022-02329-y
  5. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
  6. Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1–C68. https://doi.org/10.1111/ectj.12097
  7. Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles and practice (3rd ed.). OTexts. https://otexts.com/fpp3/
  8. Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), 100804. https://doi.org/10.1016/j.patter.2023.100804
  9. Leamer, E. E. (1983). Let's take the con out of econometrics. American Economic Review, 73(1), 31–43. https://ideas.repec.org/a/aea/aecrev/v73y1983i1p31-43.html
  10. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (pp. 4765–4774). Curran Associates. https://neurips.cc/virtual/2017/poster/9253
  11. Mullainathan, S., & Spiess, J. (2017). Machine learning: An applied econometric approach. Journal of Economic Perspectives, 31(2), 87–106. https://doi.org/10.1257/jep.31.2.87
  12. Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x
  13. Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data (2nd ed.). MIT Press. https://mitpress.mit.edu/9780262232586/econometric-analysis-of-cross-section-and-panel-data/

How to cite this article

Choose your referencing style, then copy. Check your university's or journal's guide for local variations.

PhD Research Lab Academy. (2018, June 28). Econometrics or machine learning? Prediction, causal inference and combining both in a PhD. PhD Research Lab. https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete/

PhD Research Lab Academy. “Econometrics or Machine Learning? Prediction, Causal Inference and Combining Both in a PhD.” PhD Research Lab, 28 June 2018, www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete. Accessed [access date].

PhD Research Lab Academy (2018) Econometrics or machine learning? Prediction, causal inference and combining both in a PhD. PhD Research Lab. Available at: https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete/ (Accessed: [access date]).

PhD Research Lab Academy. “Econometrics or Machine Learning? Prediction, Causal Inference and Combining Both in a PhD.” PhD Research Lab. June 28, 2018. https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete.

PhD Research Lab Academy, ‘Econometrics or Machine Learning? Prediction, Causal Inference and Combining Both in a PhD’ (PhD Research Lab, 28 June 2018) <https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete> accessed [access date].

PhD Research Lab Academy. Econometrics or machine learning? Prediction, causal inference and combining both in a PhD [Internet]. PhD Research Lab; 2018 Jun 28 [cited [access date]]. Available from: https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete/

PhD Research Lab Academy, “Econometrics or machine learning? Prediction, causal inference and combining both in a PhD,” PhD Research Lab, June 28, 2018. [Online]. Available: https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete. [Accessed: [access date]].

@online{data_analysis_econometric_v_machine_learning_is_one_becoming_obsolete_2018,
  author = {{PhD Research Lab Academy}},
  title = {{Econometrics or machine learning? Prediction, causal inference and combining both in a PhD}},
  organization = {PhD Research Lab},
  date = {2018-06-28},
  url = {https://www.phdassistance.com/blog/data-analysis-econometric-v-machine-learning-is-one-becoming-obsolete},
  urldate = {[access date]}
}

This article is for learning. Use it to understand the method and cite the original sources above in your own work; do not copy or submit it as your own.

Econometrics or machine learning? Prediction, causal inference and combining botNeed help applying it to your study?
Quantitative analysis