How the brain switches between rules

In this study, the authors trained mice on a rule switching task for the first time. They found that, similar to primates (including humans), an area of the brain called the prefrontal cortex is involved, but that for it to function properly requires configuration signals from another structure called the mediodorsal thalamus.

HFSP Young Investigator Grant holder Michael Halassa and colleagues
authored on Fri, 01 February 2019

The perfrontal cortex (PFC) is a brain structure that is known to represent abstract concepts, task rules and what we generally think of as 'cognitive control signals'. In order for us as humans to be flexible, our PFC representations have to rapidly switch depending on the context at hand. We call the ability to switch between rules and action plans 'cognitive flexibility’.

How does cognitive flexibility apply to real life situations? Imagine that you're driving in a city, where traffic lights are the cues used to regulate your stops. Now you've transitioned to a suburb, where traffic signs have replaced traffic lights at major intersections. Failure to switch from a city to a suburb context, and realize that the rules have changed may result in running the first few STOP signs that you encounter. The prefrontal cortex (PFC) is the brain region thought to be critical for the generation of rule signals in the brain based on studies in both humans and nonhuman primates. However, how the signals for changes in context are generated and what the mechanisms underlying the switching of PFC activity from one set of rules (drive based on light to drive based on signs) are, have been unclear.

To address this question, we trained mice on a rule switching task where they used either a set of sound cues or light cues to solve an attention task. We could reliably isolate the periods of time in the task where they had to switch from one set of rules to another. We also found that, similar to primates, mouse PFC neurons represented the individual rules associated with each cue set, and that PFC representations had to switch when the mouse's behavior switched. Importantly, we found that the context neural signal was enriched not in the PFC, but in an area that is strongly connected to it called the mediodorsal thalamus. Specifically, we found that the mediodorsal thalamus uses these context representations to regulate the switching between PFC representations primarily by suppressing representations that are currently no longer relevant for the task at hand. Taking advantage of the optical/genetic tools in mice, we suppressed the mediodorsal thalamus and found that such manipulation made PFC representations interfere with each other and animals less flexible.

Our results are reminiscent of the lack of cognitive flexibility seen in schizophrenia, which had traditionally been linked to problems of the PFC, but more recently to problems of the MD as well. Based on our results, we think we now have a better understanding of what each area is doing during cognitive switching which will hopefully be useful for designing new therapies. Similarly, a component of the paper addressed the issue of memory interference in AI algorithms and showed that including a module that is thalamus-like along with traditional AI architectures can be useful for lessening interference.


Thalamic regulation of switching between cortical representations enables cognitive flexibility. Rajeev V. Rikhye, Aditya Gilra & Michael M. Halassa. Nature Neuroscience volume 21, pages1753–1763 (2018).

Link to article

PubMed link