Bühler Symposium: Building and testing Theories in New Ways

Prof. Dr. Sebastian Musslick (Osnabrück University)
Automated Scientific Discovery of Mind and Behavior

Artificial intelligence is transforming scientific discovery across disciplines, from predicting protein structures in biology to identifying novel quantum states in physics. In the natural sciences, automated discovery systems now enable researchers to navigate vast experimental and hypothesis spaces, moving beyond the limits of human intuition. In this talk, I explore how these methods can be adapted to the study of mind and behavior, introducing automated scientific discovery as a paradigm for cognitive science. I present AutoRA, an open-source framework for automating key stages of empirical research -- including experimental design, data collection, and model inference -- and demonstrate how it can be used to uncover new computational models of cognition and novel behavioral phenomena. Through case studies in psychophysics, learning, decision-making, and cognitive control, I illustrate how closed-loop discovery systems can support more integrative, scalable approaches to understanding human cognition. Finally, I discuss the unique challenges that automated scientific discovery faces in cognitive science, such as experimental fragmentation and the need for multi-level explanations, and outline future directions for building systems that not only accelerate scientific progress, but reshape how we generate and evaluate theories of the mind.

Prof. Dr. Andreas Glöckner (Köln University)
How to Analyze and Improve Theory Specification in Psychology

Theories are the backbone of cumulative science. For empirical findings to inform theory development, not only empirical results but also the content and predictions of the theories being tested need to be reproducibly interpretable. Yet theoretical reproducibility—the extent to which independent researchers recover the same substantive content from a theory  description—remains largely unknown. In a large international  collaboration involving 139 theorists, 44 established social-psychological theories were independently analyzed and specified using a standardized theory-specification procedure. Theoretical reproducibility was operationalized as content agreement between independent specifications concerningimplications, definitions, and operationalizations. Mean Overall Theory-Specification Agreement under the primary scoring rule was substantially below a desirable level. A substantial source of disagreement was that independent researchers did not always identify the same theoretical elements as belonging to a theory, rather than merely interpreting matched elements differently. Agreement was highest for definitions and lowest for operationalizations. Analyses of the consensus versions of the specified theories further showed that 5% of implications, 34% of definitions, and 59% of operationalizations required information that was missing or had to be derived. Consensus-based analyses revealed frequent underspecification risks, tautology risks, and, less frequently, contradiction risks. These findings provide an  initial large-scale estimate of theoretical reproducibility and a systematic quantitative assessment of the specification of established theories in social psychology. The collaborative  project also shows how structured theory analysis and specification can contribute to psychological science by establishing common ground concerning theory content and by identifying ambiguities, missing content, and specification problems before empirical testing.