Insights into reward-based decisions using computational models and ultra-high field MRI

The ability to integrate past and current feedback associated with di↵erent environmental stimuli is crucial for adaptive and goal-directed behavior. The field of reinforcement learning (RL) focuses on understanding such ability: Computational models propose algorithms for the updating of beliefs based on the received feedback, while neural models describe how these algorithms are implemented in the brain. In this dissertation, I investigate learning- by-feedback both at the algorithmic level (in the first part) and at the implementation level (in the second part).
In the first part, in particular, I focus on human behavior during learning-by- feedback in the presence of both monetary gains and losses (first manuscript) or of di↵erent magnitudes of monetary gains (second manuscript). To date, most computational RL mod- els focused on trial-by-trial dynamics and have not explained how response times (RTs) change during learning or are a↵ected by di↵erent learning contexts (e.g., in the presence of gains vs. losses). In both manuscripts, I argue that RTs are crucial for the understand- ing of reward-based decisions: In both studies, participants’ RTs were a↵ected by di↵erent learning contexts, while their choice preferences not always were. To jointly explain the e↵ects on both preferences and RTs, I used a sequential sampling model, the di↵usion decision model (DDM). In the first manuscript, I propose a meta-analytical approach to simultaneously analyze the e↵ects of di↵erent learning contexts on choice preferences and RTs from four independent experiments. In the second manuscript, I propose a new model that incorporates an RL algorithm into the DDM.
In the second part of my thesis, I investigated the coding of losses and gains by the dopaminergic nuclei in the human brain. Since these nuclei are situated deep in the brain, their signal is hard to study using non-invasive imaging techniques such as mag- netic resonance imaging (MRI): To date, human studies have provided incomplete and partially contradicting findings about the reward signals in dopaminergic nuclei. In the third manuscript, I provide evidence that clarifies the role of the dopaminergic nuclei when receiving more or less surprising gains and losses, as well as when expecting higher or lower outcome risk. To do so, I capitalize on ultra-high field MRI and on the use of multimodal images to delineate the dopaminergic nuclei on a participant level.