Stan B. Floresco is an Associate Professor of Psychology at the University of British Columbia, a member of the UBC Brain Research Center Institute, and a member of the American College of Neuropsychopharmacology.   He received his BSc in Psychology in 1994, his MA in Psychology in 1996, and his PhD 2000, all at the University of British Columbia under the mentorship of Anthony Phillips.  He was a Human Frontiers Science Organization postdoctoral fellow in the laboratory of Anthony Grace at the University of Pittsburgh until 2003, after which he returned to UBC to take up his current position.  His research employs behavioral and neurophysiological approaches to study neural circuits within the mesocorticolimbic dopamine system that facilitate cognitive processes such as behavioural flexibility, cost/benefit decision making and reward-related learning, using rodents as a model system. His work is currently funded by the Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research.

---

We routinely encounter situations requiring us to deal with unexpected changes in our environments.  Many times, certain actions that led to rewards in the past are no longer effective for obtaining our goals, and we must enact novel modes of responding to achieve our objectives. For example, when trying to make your morning coffee, you may find that your spouse has reorganized the kitchen so that the coffee container which always used to be in the freezer is now located on one of the cupboard shelves. You may find that for the first few days (or weeks) you automatically (and now, erroneously) reach for the freezer rather than the cupboard for your coffee. Over time, your behavior eventually adapts (a process referred to as “reversal learning”), but every once in a while, you regress back to your old habits and reach for the freezer before correcting yourself.  Imagine a more complex situation where a local take-out eatery that you frequent has closed down. Obtaining a new source of your favorite dish requires you to look at your neighborhood a bit differently (maybe checking out side streets rather than main roads) and take a number of steps to change your foraging strategy.  First, stop going to the old location (i.e., stop perseverating), next search around for a reasonable substitute restaurant, and then in the future, make sure you don’t subconsciously drift over to the old place when you’re having a craving for pad thai (or what have you).

Our ability to behave in a flexible manner and adjust actions to adapt to these types of changes is an essential survival skill that taps multiple cognitive operations, requiring inhibition of outmoded responding, searching for novel effective strategies and then maintaining these new strategies.  It is well established that the frontal lobes play a fundamental role in enabling these different forms of behavioral flexibility.  Patients with damage to prefrontal cortical regions can acquire novel skills or rules with relative ease. However, they have great difficulty when they must change their behavior in response to changes in reinforcement contingencies, both in real-life settings and in laboratory tests of behavioral flexibility such as the Wisconsin Card Sorting task.  Damage to the frontal lobes typically leads to perseverative deficits, suggesting that these regions play a key role in suppressing the use of old, ineffective behaviors. Moreover, a number of neuropsychiatric disorders have also been associated with impairment in behavioral flexibility, including schizophrenia, obsessive-compulsive disorder and attention deficit and hyperactivity disorder.  As such, understanding the neural circuitry that facilitates this form of executive functioning can provide insight into the brain pathology that may underlie impairments in the functions associated with these diseases.

Of course, no one brain region can solve these types of problems single-handedly, and the prefrontal cortex is interconnected with numerous subcortical structures that it may interact with to enable flexible responding.  One of the main research goals of our laboratory has been to elucidate the broader neural circuitry that facilitates these forms of behavioral flexibility, and the specific contribution that these circuits make to solve these problems, using rodents as a model system. Rats (like all mammals) are quite adept at altering their behavior in response to changes in their environment. One way we can assess these abilities in rats is by training them to learn simple discrimination tasks performed either on a maze or in an operant chamber, and then changing the manner they must discriminate between different stimuli to obtain food reward. For example, we may first teach a hungry rat to always enter the right arm of a T-maze in order to obtain a tasty food reward (Figure 1).  On any given trial, we also place a local visual cue pseudorandomly in the left or right arm of the maze, but this cue does not reliably predict the location of food. Rats can learn this type of rule very quickly, requiring about 60 trials within a single day of training to make 10 consecutive correct responses.  On the next day, we change the rule; now to obtain food, it must always enter the arm with the visual cue, which sometimes is on the right, and other times is on the left. So now, the rat has to view the maze a bit differently than before, and change its discrimination strategy. 

Using similar procedures, we can also assess reversal learning (a simpler form of flexibility), where rats are required to first learn to enter the left arm, and then the right arm, so that the basic strategy remains the same (always turn in one direction), but the specific stimulus to be approached has changed.  A key advantage of using these procedures is that we can conduct a detailed analysis of the types of errors made during the shift, to assess if impairments in shifting induced by a particular brain manipulation are due to deficits in suppressing of old modes of responding (perseveration), acquisition of novel strategies, or maintenance of novel strategies once perseveration has ceased. Previous studies have shown that, across species, reversal learning is mediated by the orbital regions of the prefrontal cortex (Dias, Robbins, & Roberts, 1996; McAlonan & Brown, 2003; Ghods-Sharifi, Haluk, & Floresco, 2008).  More complex shifts between different strategies (which have been a primary focus of our work) is mediated by the medial prefrontal cortex in rats, whereas in primates and humans, lateral prefrontal regions play a key role (Dias et al., 1996; Birrell & Brown, 2000; Floresco, Block, & Tse, 2008).  In all instances, lesions or inactivation of frontal lesions typically lead to increases perseverative tendencies during a reversal or strategy shift, but these manipulations do not impair the initial learning of one of these simple rules (as observed in human frontal lobe patients).

 

Figure 1. Example of the task used to assess strategy shifting in rodents. The arrows in the maze represent the correct navigation pattern to receive reinforcement.  During initial response discrimination training (upper panel), a rat always had to make a 90° turn to the left to receive food.  A visual-cue is randomly placed in one of the choice arms on each trial but does not reliably predict the location of food.  During the shift on Day 2 (middle panel), the rat is required to use a visual-cue discrimination strategy.  Here, the rat was started from the same arms but had to always enter the arm with the visual-cue, which could require either a right or left turn.  Thus, the rat must shift from the old strategy and approach the previously-irrelevant cue in order to obtain reinforcement. The bottom panel shows examples of the three types of errors that rats could make during the shift.  Early in the choice sequence, animals persist in using the old, now incorrect strategy (e.g., continuing to turn left, even on trials where the visual cue is in the right arm; perseverative errors).  Over trials, rats eventually start to use the new rule on more than 50% of trials where the two rules are in conflict, but every once in a while, will make the same type of error.  These are referred to as “regressive” errors, and index how well animals maintain a strategy after perseveration has ceased. A third type of error occurs when rats make a response that is not consistent with either the old rule or the new rule (e.g., turning right when the cue is in the left arm; never-reinforced errors). A few of these types of errors are required through normal trial and error learning to ascertain the new strategy. However, a disproportionate increase in this type of response is indicative of a problem in being able to identify and acquire a novel strategy.

 

Much of our research has focused on circuits related to the medial prefrontal cortex of the rat, which appears to selectively mediate shifts between strategies.  Two key brain regions that are anatomically connected with this region of the frontal lobes are the nucleus accumbens region of the ventral striatum, and the mediodorsal nuclei of the thalamus.  The nucleus accumbens is a key output of the prefrontal cortex, and is thought to facilitate the transformation of action plans mediated by the frontal lobes into overt behavior through its connection with motor systems (Mogenson, Brudzynski, Wu, Yang, & Yim, 1993; Floresco, 2007).  The mediodorsal region of the thalamus is reciprocally connected with the prefrontal cortex, and certain nuclei within this region also send projections to the nucleus accumbens.  To assess whether these input and output regions of the prefrontal cortex also play a role in switching between strategies, we would induce a reversible inactivation of these structures via microinfusion of either local anesthetics or GABA agonists that suppress neural activity for about an hour or two (in essence, inducing a temporary lesion of that structure).  We observed that inactivation of either of these regions did not disrupt the ability to learn a simple discrimination rule (e.g., always turn left), but severely impaired shifting from one strategy to another. What was of particular interest was that each of these manipulations resulted in qualitatively different types of impairments.  Unlike prefrontal inactivation, suppression of neural activity in the nucleus accumbens did not lead to increased perseveration, but rather, impaired the acquisition and maintenance of a new strategy (Floresco,  Ghods-Sharifi, Vexelman, & Magyar, 2006).  On the other hand, thalamic inactivation induced a pattern of deficits that somewhat resembled prefrontal and accumbens inactivation, increasing perseverative tendencies, but also impairing the acquisition of a novel strategy (Block, Dhanji, Thompson-Tardif, & Floresco, 2007). 

 

Figure 2.  Analysis of the type of strategy shifting errors induced by disconnection of different nodes in prefrontal-thalamic-ventral striatal circuitry. Disconnection of the mediodorsal thalamus (MD) and prefrontal cortex (PFC) increased perseverative/regressive errors (top panel) but did not affect never-reinforced errors (middle panel), relative to vehicle and unilateral/ipsilateral inactivation controls.  A similar profile was observed following disconnection of PFC and the nucleus accumbens (NAc). In contrast, only MD-NAc disconnections resulted in an increase in never-reinforced type errors. Bottom panel diagrams the specific asymmetrical inactivations that were employed to selectively disconnect these different circuits.

 

Our findings suggested that intact neural activity in each of these three interconnected structures is essential for facilitating shifts between different strategies, with each region playing distinct yet complementary roles in these processes.  It was of particular interest to ascertain the specific functional neural circuitries that facilitate these different components of set-shifting.  To do so, we conducted a series of asymmetrical disconnection inactivations, which permit the relatively selective disruption of communication between two brain regions in a particular circuit, while leaving communication between other parallel circuits relatively intact. We observed that disconnection of thalamo-cortical circuitry induced a selective increase in perseverative errors, indicating that this circuit is necessary for successful disengagement from a previously relevant strategy (Figure 2, top).  Furthermore, disconnections between the prefrontal cortex and nucleus accumbens increased perseverative/regressive errors. Given inactivation of the accumbens alone increased regressive errors (Floresco, Ghods-Sharifi et al., 2006), we interpreted these data to suggest that prefrontal inputs to interacting with a particular ensemble of ventral striatal neurons facilitate the maintenance of a novel behavioral strategy once perseveration has ceased.  Interestingly, inactivation of thalamo-accumbens circuitry did not induce a perseverative deficit, nor did it interfere with strategy maintenance.  Rather, disconnection of this circuit selectively disrupted learning of a novel strategy during the shift, facilitating the elimination of ineffective response options when reward contingencies have changed (Figure 2, middle)  Parallel pharmacological studies have further shown that the specific contributions that the prefrontal cortex and nucleus accumbens make to strategy shifting are critically dependent on the neurotransmitter dopamine, which has been heavily implicated in mediating cognitive and reward functions of these regions.  Within the prefrontal cortex, dopamine D1 and D2 receptors interact in a cooperative manner to reduce perseveration, whereas activity at D1 receptors in the accumbens facilitates strategy maintenance (Floresco, Magyar, Ghods-Sharifi, Vexelman & Tse, 2006; Haluk & Floresco, 2009).  Viewed in a broader context, these results highlight the fact that different components of behavioral flexibility (suppression of irrelevant strategies, acquisition and generation of novel strategies, and maintenance of effective strategies) are subserved by parallel brain circuits incorporating the prefrontal cortex, different regions of the striatum, the medial thalamus and the dopamine system, with each circuit making a specialized contribution to behavior (Figure 3).

 

Figure 3. Schematic of some of the neural circuitry that mediate different components of strategy shifting.

 

Even though it is well-established that prefrontal cortical neural activity plays a critical role mediating complex shifts between different strategies, there has been relatively little research on the how specific patterns of activity of prefrontal neuronal ensembles may encode information to facilitate these shifts.  We addressed this question in a recent study, by recording activity from multiple prefrontal neurons in rats performing a similar strategy shifting task conducted in an operant chamber (Durstewitz, Vittoz, Floresco & Seamans, 2010).  We observed that on each trial that rats responded accurately in accordance to the first rule they were taught (e.g., always press one of two levers that has a light illuminated above it), groups of prefrontal neurons tended to increase or decrease their relative rates of firing in a consistent manner prior to each choice.  For the sake of example, in this setting, recordings from a network of 4 neurons may have shown that cells 1 and 2 increased firing relative to baseline, cell 3 showed no change and cell 4 decreased activity, and this pattern of firing changes were observed for every correct choice (what we termed the network’s “steady state”).  When rats now had to change their responding following a rule shift, we observed that when rats eventually learned the new rule and displayed good performance, the specific patterns of activity within this network reorganized to a new steady state (e.g., now neurons 1 and 3 decreased firing, and cells 1 and 4 increased activity).  Thus, performance of different rules is represented by distinct patterns of firing in populations of prefrontal neurons.  Note however, that this activity does not appear to be necessary for implementation of a well-established rule, because suppressing activity in this region does not interfere with the acquisition or expression of these relatively simple types of behavior (Floresco et al., 2008).  Thus, it seems that under conditions where everything is working properly, prefrontal neural activity seems to keep track of rules, actions and outcomes, but this activity does not directly contribute to behavior.

What was particularly striking was how network activity changed when the rules were switched unexpectedly.  During the initial part of the rule shift, prefrontal network activity went into a state of disarray, displaying various patterns of activity, as opposed to the harmonious patterns that were observed during performance of the old rule.  However, at some point during the shift, the network activity stabilized relatively quickly, and behaved in a consistent manner (reaching its new “steady state”), as if it had suddenly figured out the new rule. What’s more, the re-stabilization of the network occurred a few trials after the rat started to string together a few correct choices. What this suggested to us is upon changes in the status quo, other brain regions may attempt to use different strategies to figure out the best way to adapt. In these situations, the prefrontal cortex keeps track of different actions and outcomes and monitoring which actions are (or are not) paying off. Once a new strategy meets with some success, prefrontal networks detect this and may coordinate activity amongst different subcortical memory systems to ensure that the best strategy wins out.

What our work shows is that adapting our behavior in response to changes in our environment requires communication within a complex and distributed neural circuit, with separate nodes in these circuits subserving distinct roles to enable smooth transitions from one mode of responding to another.  Furthermore, in these situations, the frontal lobes may play a key role in alerting other brain systems that something is not quite right (things have changed), and subsequently, supervising progress towards a new goal, coordinating information flow between other learning systems to facilitate a smooth transition from one pattern of behavior to another.

References

Birrell, J.M. & Brown, V.J. (2000). Medial frontal cortex mediates perceptual attentional set shifting in the rat. Journal of Neuroscience, 20, 4320-4324.

Block, A.E., Dhanji, H., Thompson-Tardif, S.F. & Floresco, S.B. (2007). Thalamic-prefrontal cortical-ventral striatal circuitry mediates dissociable components of strategy set shifting. Cerebral Cortex, 17,1625-1636.

Durstewitz, D., Vittoz, N.M., Floresco, S.B. & Seamans, J.K. (2010). Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron,66, 438-448.

Dias, R., Robbins, T.W. & Roberts, A.C. (1996). Dissociation in prefrontal cortex of affective and attentional shifts. Nature, 380, 69-72.

Floresco, S.B., Block, A.E. & Tse, M.T. (2008). Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure. Behavioural Brain Research, 190, 85-96.

Floresco, S.B. (2007). Dopaminergic regulation of limbic-striatal interplay. Journal of Psychiatry Neuroscience, 32, 400-411

Floresco, S.B., Magyar, O., Ghods-Sharifi, S., Vexelman, C. & Tse, M.T. (2006). Multiple dopamine receptor subtypes in the medial prefrontal cortex of the rat regulate set-shifting. Neuropsychopharmacology, 31, 297-309.

Floresco, S.B., Ghods-Sharifi, S. Vexelman, C., & Magyar, O. (2006). Dissociable roles for the nucleus accumbens core and shell in regulating set shifting. Journal of Neuroscience, 26, 2449-2457.

Ghods-Sharifi, S., Haluk, D.M. & Floresco, S.B. (2008). Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning. Neurobiology of Learning and Memory, 89, 567-573.

Haluk, D.M. & Floresco, S.B. (2009). Ventral striatal dopamine modulation of different forms of behavioral flexibility. Neuropsychopharmacology, 34, 2041-2052.

McAlonan, K. & Brown, V.J. (2003). Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behavioural Brain Research 146, 97–103.

Mogenson, G.L., Brudzynski, S.M., Wu, M., Yang, C.R., & Yim, C.Y. (1993). From motivation to action: a review of dopaminergic regulation of limbic→nucleus accumbens→ventral pallidum→pedunculopontine nucleus circuitries involved with limbic-motor integration. In P.W. Kalivas and C.D. Barnes (Eds.), Limbic-Motor Circuits and Neuropsychiatry (pp. 193-263) Boca Raton: CRC Press.

The views expressed in Science Briefs are those of the authors and do not reflect the opinions or policies of APA.