What the mouse sees is what the mouse expects to see – or what it doesn’t expect to see

Neural activity in visual cortex of a behaving mouse can be better explained by a combination of expectations of what the mouse is about to see and deviations from these expectations, than by visual input alone.

HFSP Long-Term Fellow Georg Keller and colleagues
authored on Thu, 12 July 2012

Ever since the pioneering work of Hubel and Wiesel, sensory processing in visual cortex is typically described as a processing hierarchy of ever more complex sensory filters. However, almost all of our knowledge of visual processing comes from experiments performed on anesthetized or awake fixating animals. Under natural circumstances most of the visual input is sensory feedback driven by motor output. We move our eyes, head, or body and this has a direct and predictable effect on the visual input we are getting. The critical question then of course is: is processing of visual feedback different from processing of passive visual input?

To address this question, we record neural activity in visual cortex of mice navigating a simple virtual environment. In this virtual environment the mouse's locomotion on a treadmill is coupled to movement along a virtual corridor, such that running forward on the treadmill results in visual flow backwards. In this virtual environment we can now quantify the contribution of visual input and motor related input to the activity in visual cortex separately. This is done by either playing back visual flow to a passively watching mouse, or by having the mouse run in complete darkness. We can also probe responses of visual cortex to mismatch in the coupling between locomotion and visual flow. We do this by briefly halting visual flow feedback at random times.

We show that visual cortex receives strong motor related input. Locomotion is a better predictor of activity in visual cortex than visual input. We also show that the strongest average population response to any stimulus is elicited by a mismatch between locomotion and visual flow feedback. This suggests that motor related input to visual cortex is the correlate of an expectation of visual input, and that differences between actual and expected visual input then trigger mismatch signals. Such mismatch signals are crucial not only to the rapid detection of movement in the visual field that is not the result of self-motion (i.e. that of an approaching predator), but also of fundamental importance to sensorimotor learning.


Sensorimotor Mismatch Signals in Primary Visual Cortex of the Behaving Mouse, Georg B. Keller, Tobias Bonhoeffer, Mark Hübener, Neuron 74:809-815, DOI 10.1016/j.neuron.2012.03.040.

Pubmed link

Neuron link