A9

Computational modelling of cognitive control over multiple trials

A key question of cognitive control research is: How does the brain select its actions, especially when there is conflict about what to do? This task can be taxing in our often uncertain and dynamic environment, especially when we have to cling to a goal over extended periods of time. In recent years, computational theories for goal-directed behaviour have been proposed, in particular reinforcement learning (RL) and hierarchical RL (HRL). Currently, in practice, different versions of RL models are used to explain different aspects of goal-directed behaviour. For example, it has been proposed that the brain uses two different controllers to reach incentivized goals: the so-called model-free and model-based controllers where an additional arbitration balances the two. It is a possibility that the brain runs two different controllers with two distinct computational mechanisms to reach a single goal; however a viable alternative is that the brain just uses a single controlling mechanism, which adapts its parameters to different environmental situations. Such a single-model approach would have several advantages, among them the possibility to predict new findings, as opposed to explaining away new findings with yet another RL model instantiation.
Therefore, the overarching goal of this project is to develop such a single computational model that explains how humans exert cognitive control, over extended time periods, to reach goals in uncertain and dynamic environments. To do this, we will build on a recently developed Bayesian approach, the so-called active inference (ActInf) framework. The ActInf framework has two advantages over RL and HRL models. First, it was specifically designed to account for the variability and uncertainty in our dynamic environment and is therefore better suited for addressing questions about resolving cognitive control dilemmas in ecologically valid experiments of goal-reaching under choice conflicts. Second, in comparison to HRL modelling, it is straightforward to build unifying hierarchical models with the ActInf framework. Such a hierarchical unification was hypothesized and identified before as an important research question for the model-free vs. model-based division but it was unclear with what model this may be achieved.
We expect four key outcomes of this project. First, the ActInf model will be able to explain human behaviour and physiological parameters as well or even better than conventional models and in a more efficient way. We will show this with two representative goal-reaching tasks, which require cognitive control. Second, the model can be used to test specific predictions about how goal-directed behaviour computations are implemented by the brain. Third, the work will be of fundamental importance for the CRC, because it will provide an explanation how specific control dilemmas are adaptively resolved without resorting to an additional (homunculus) arbitration mechanism. Fourth, the project will provide a novel way of analysing behavioural and neuroimaging data that has a powerful translational potential for many other types of cognitive control experiments, particularly those with ecologically valid stimulus conditions.

Project Members

Principal Investigators

Prof. Dr. Stefan Kiebel
Professor of Neuroimaging
Phone: +49 (0)351 - 431 45
E-Mail:

Prof. Dr. phil. Thomas Goschke
CRC 940 Spokesperson; Professor for General Psychology (W3)
Phone: +49 (0)351 463-34695
E-Mail: 

Staff

Dr. phil. nat. Dimitrije Marković
Research Associate (Postdoc)
Phone: +49 (0)351 - 463 43145

Dr. Ben Jonathan Wagner                                                                                                                   Research Associate (Postdoc)                                                                                                   Phone: +49(0)351 - 463 42695                                                                   

M.Sc. Alexandra Felea                                                                                                               Research Associate (Predoc)                                                                                                     Phone: +49(0)351 - 463 43142                                                                                  

Dr. Sarah Schwöbel                                                                                                                           Research Associate (Postdoc)                                                                                                   Phone: +49(0)351 - 463 42697                                                                              

  MSc. Eric Legler                                                                                                                    Research Associate (Predoc)
Phone: +49 351 463-43140
E-Mail:

Publications

• Markovic, D., Goschke, T., & Kiebel, S. J. (2021). Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales. Cognitive, Affective & Behavioral Neuroscience, 21(3), 509-533. doi:10.3758/s13415-020-00837-x

• Schwöbel, S., Markovic, D., Smolka, M. N., & Kiebel, S. J. (2021). Balancing control: A Bayesian interpretation of habitual and goal-directed behavior. Journal of Mathematical Psychology, 100. doi:10.1016/j.jmp.2020.102472

• Ott, F., Markovic, D., Strobel, A., & Kiebel, S. J. (2020). Dynamic integration of forward planning and heuristic preferences during multiple goal pursuit. PLoS Comput Biol, 16(2), e1007685. doi:10.1371/journal.pcbi.1007685

• Marković, D., Reiter, A. M. F., & Kiebel, S. J. (2019). Predicting change: Approximate inference under explicit representation of temporal structure in changing environments. PLoS Computational Biology, 15(1), e1006707. doi:10.1371/journal.pcbi.1006707
• Parr, T., Marković, D., Kiebel, S. J., & Friston, K. L. (2019). Neuronal message passing using Mean-field, Bethe, and Marginal approximations. Scientific Reports, 9(1), 1889. doi:10.1038/s41598-018-38246-3

• Cuevas Rivera, D., Ott, F., Marković, D., Strobel, A., Kiebel, S. J. (2018). Context-dependent risk aversion: a model-based approach. Frontiers in Psychology, 9, 2053. doi:10.3389/fpsyg.2018.02053

• Schwöbel, S., Kiebel, S., & Marković, D. (2018). Active inference, belief propagation, and the Bethe approximation. Neural Computation, 30(9), 2530-2567. doi:10.1162/neco_a_01108.

• Marković, D, & Kiebel, S. J. (2016). Comparative analysis of behavioral models for adaptive learning in changing environments. Frontiers in Computational Neuroscience,10, 33. doi:10.3389/fncom.2016.00033

• Marković , D., Gläscher, J., Bossaerts, P., O’Doherty, J., & Kiebel, S. J. (2015). Modeling the evolution of beliefs using an attentional focus mechanism. PLoS Computational Biology, 11(10), e1004558.doi:10.1371/journal.pcbi.1004558