­Presentation at the Causal Data Science Meeting 2023

At this year’s Causal Data Science Meeting 2023, Tom Dudda presented preliminary results of a new meta-study co-authored with Lars Hornuf called The Perks and Perils of Machine Learning in Business Research. Lars Hornuf and Tom Dudda examine the use of predictive machine learning models in top business and economic journals (FT50 research rank) and the predictive performance of these models relative to models that have traditionally been used in the literature (such as OLS and Logit regressions). They show in which disciplines ML models are applied most frequently and identify some potential to catch up, for example, in HR, entrepreneurship, and accounting. On average, the best-performing ML model, relative to a traditional benchmark model, considerably improves the metric reported to evaluate predictions. However, if the performance of all ML models for which results are reported in a study is considered, and not only the best-performing model, there is only relatively little improvement, on average, against traditional benchmark models. Also, many studies do not even report results for these benchmark models. The first findings indicate that much time and energy often seem to be required to beat traditional models. Whether the economic gain of machine learning models is always significant enough to justify increased time and energy consumption remains an open question for future research.

Link to the conference keynote by Dominik Janzing (Amazon Research): YouTube