Cerebral Cortex, Vol. 10, No. 3, 284-294,
March 2000
© 2000 Oxford University Press
The Orbitofrontal Cortex and Reward
University of Oxford, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, UK
 |
Abstract |
|---|
 Top Abstract IntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
The primate orbitofrontal cortex contains the secondary taste cortex, in which the reward value of taste is represented. It also contains the secondary and tertiary olfactory cortical areas, in which information about the identity and also about the reward value of odors is represented. The orbitofrontal cortex also receives information about the sight of objects and faces from the temporal lobe cortical visual areas, and neurons in it learn and reverse the visual stimulus to which they respond when the association of the visual stimulus with a primary reinforcing stimulus (such as a taste reward) is reversed. However, the orbitofrontal cortex is involved in representing negative reinforcers (punishers) too, such as aversive taste, and in rapid stimulus–reinforcement association learning for both positive and negative primary reinforcers. In complementary neuroimaging studies in humans it is being found that areas of the orbitofrontal cortex (and connected subgenual cingulate cortex) are activated by pleasant touch, by painful touch, by rewarding and aversive taste, and by odor. Damage to the orbitofrontal cortex in humans can impair the learning and reversal of stimulus– reinforcement associations, and thus the correction of behavioral responses when these are no longer appropriate because previous reinforcement contingencies change. This evidence thus shows that the orbitofrontal cortex is involved in decoding and representing some primary reinforcers such as taste and touch; in learning and reversing associations of visual and other stimuli to these primary reinforcers; and in controlling and correcting reward-related and punishment-related behavior, and thus in emotion.
|
Introduction |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
The prefrontal cortex is the cortex that receives projections from the mediodorsal nucleus of the thalamus and is situated in front of the motor and premotor cortices (areas 4 and 6) in the frontal lobe. Based on the divisions of the mediodorsal nucleus, the prefrontal cortex may be divided into three main regions (Fuster, 1997
). First, the magnocellular, medial, part of the mediodorsal nucleus projects to the orbital (ventral) surface of the prefrontal cortex (which includes areas 13 and 12). This part of the prefrontal cortex is called the orbitofrontal cortex, and receives information from the ventral or object-processing visual stream, and taste, olfactory and somatosensory inputs. Second, the parvocellular, lateral, part of the mediodorsal nucleus projects to the dorsolateral prefrontal cortex. This part of the prefrontal cortex receives inputs from the parietal cortex and is involved in tasks such as spatial short-term memory tasks (Fuster, 1997; Rolls and Treves, 1998). Third, the pars paralamellaris (most lateral) part of the mediodorsal nucleus projects to the frontal eye fields (area 8) in the anterior bank of the arcuate sulcus.
The functions of the orbitofrontal cortex are considered in this paper. The cortex on the orbital surface of the frontal lobe includes area 13 caudally and area 14 medially, and the cortex on the inferior convexity includes area 12 caudally and area 11 anteriorly (Fig. 1) (Carmichael and Price, 1994; Petrides and Pandya, 1994; Price et al., 1996). This brain region is well developed in primates, including humans, but poorly developed in rodents, with homologies to areas found in primates uncertain, so that care must be used in interpretation of the term ‘orbitofrontal' when applied to rodents (Uylings and van Eden, 1990). To understand the function of this brain region in humans, the majority of the studies described were therefore performed with macaques or with humans.
|
|
Connections |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
Rolls et al. (1990) discovered an area with taste-responsive neurons in the lateral part of the orbitofrontal cortex, and showed that this was the secondary taste cortex in that it receives a major projection from the primary taste cortex and not from the thalamic taste relay nucleus (VPMpc) (Baylis et al., 1994
. More medially, there is an olfactory area (Rolls and Baylis, 1994). Anatomically, there are direct connections from the primary olfactory cortex (pyriform cortex) to area 13a of the posterior orbitofrontal cortex, which in turn has onward projections to a middle part of the orbitofrontal cortex (area 11) (Morecraft et al., 1992; Barbas, 1993; Carmichael et al., 1994) (Figs 1 and 2
). Visual inputs reach the orbitofrontal cortex directly from the inferior temporal cortex, in which representations of objects are found (Booth and Rolls, 1998), the cortex in the anterior part of the superior temporal sulcus, in which face-responsive neurons are found (Hasselmo et al., 1989a,b; Wallis and Rolls, 1997), and the temporal pole (Barbas, 1988, 1993, 1995; Barbas and Pandya, 1989; Seltzer and Pandya, 1989; Morecraft et al., 1992; Carmichael and Price, 1995). There are corresponding auditory inputs from the superior temporal cortex (Barbas, 1988, 1993), and somatosensory inputs from somatosensory cortical areas 1, 2 and SII in the frontal and pericentral operculum, and from the insula (Barbas, 1988; Carmichael and Price, 1995). The caudal orbitofrontal cortex receives strong inputs from the amygdala (Price et al., 1991). The orbitofrontal cortex also receives inputs via the mediodorsal nucleus of the thalamus, the pars magnocellularis, which itself receives afferents from temporal lobe structures such as the prepyriform (olfactory) cortex, amygdala and inferior temporal cortex (Price, 1999). The orbitofrontal cortex projects back to temporal lobe areas such as the inferior temporal cortex and, in addition, to the entorhinal cortex (or ‘gateway to the hippocampus') and cingulate cortex (Insausti et al., 1987). The orbitofrontal cortex also projects to the preoptic region and lateral hypothalamus, the ventral tegmental area (Nauta, 1964; Johnson et al., 1968), and the head of the caudate nucleus (Kemp and Powell, 1970). Reviews of the cytoarchitecture and connections of the orbitofrontal cortex are provided elsewhere (Carmichael and Price, 1994, 1995; Petrides and Pandya, 1994; Barbas, 1995; Pandya and Yeterian, 1996; Price, 1999).
|
|
Effects of Lesions of the Orbitofrontal Cortex |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
Macaques with lesions of the orbitofrontal cortex are impaired at tasks which involve learning about which stimuli are rewarding and which are not, and especially in altering behavior when reinforcement contingencies change. The monkeys may respond when responses are inappropriate, e.g. no longer rewarded, or may respond to a non-rewarded stimulus. For example, monkeys with orbitofrontal damage are impaired on go/no-go task performance, in that they go on the no-go trials (Iversen and Mishkin, 1970
); in an object reversal task, in that they respond to the object which was formerly rewarded with food; and in extinction, in that they continue to respond to an object which is no longer rewarded (Butter, 1969; Jones and Mishkin, 1972). There is some evidence for dissociation of function within the orbitofrontal cortex, in that lesions to the inferior convexity produce the go/no-go and object reversal deficits, whereas damage to the caudal orbitofrontal cortex, area 13, produces the extinction deficit (Rosenkilde, 1979).
Lesions produced more laterally, e.g. in the inferior convexity, can influence working memory tasks in which objects must be remembered for short periods, e.g. delayed matching to sample and delayed matching to non-sample tasks (Passingham, 1975; Mishkin and Manning, 1978; Kowalska et al., 1991), and neurons in this region may help to implement this visual object short-term memory by holding the representation active during the delay period (Rosenkilde et al., 1981; Wilson et al., 1993) by using the attractor properties of autoassociation networks (Rolls and Treves, 1998; Renart et al., 1999). Whether this inferior convexity area is specifically involved in a short-term object memory is not yet clear, and a medial part of the frontal cortex may also contribute to this function (Kowalska et al., 1991). It should be noted that this short-term memory system for objects (which receives inputs from the temporal lobe visual cortical areas in which objects are represented) is different to the shortterm memory system in the dorsolateral part of the prefrontal cortex, which is concerned with spatial short-term memories, consistent with the inputs to the dorsolateral prefrontal cortex from the parietal cortex, but also probably operates using the attractor properties of autoassociation networks [in ways described elsewhere (Rolls and Treves, 1998)].
Damage to the caudal orbitofrontal cortex in the monkey also produces emotional changes (e.g. decreased aggression to humans and to stimuli such as a snake and a doll), and a reduced tendency to reject foods such as meat (Butter et al., 1969, 1970; Butter and Snyder, 1972) or to display the normal preference ranking for different foods (Baylis and Gaffan, 1991). In humans, euphoria, irresponsibility and lack of affect can follow frontal lobe damage (Damasio, 1994; Kolb and Whishaw, 1996; Rolls, 1999a), particularly orbitofrontal damage (Rolls et al., 1994; Hornak et al., 1996).
|
Neurophysiology of the Orbitofrontal Cortex |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
Taste
One of the recent discoveries that has helped us to understand the functions of the orbitofrontal cortex in behavior is that it contains a major cortical representation of taste (Rolls, 1989, 1995a, 1997a) (cf. Fig. 2). Given that taste can act as a primary reinforcer, i.e. as a reward or punishment innately without the need for learning, we now have the start for a fundamental understanding of the function of the orbitofrontal cortex in stimulus–reinforcement association learning. We know how one class of primary reinforcers reaches and is represented in the orbitofrontal cortex. A representation of primary reinforcers is essential for a system that is involved in learning associations between previously neutral stimuli and primary reinforcers, e.g. between the sight of an object and its taste. [In this paper, the terms ‘reward' and ‘positive reinforcer' are used equivalently, and are stimuli which an animal will work to obtain. Similarly, the terms ‘punisher' and ‘negative reinforcer' are used to describe stimuli that an animal will work to escape from or avoid. A more detailed description is provided elsewhere (Rolls, 1999a, 2000b).]
The representation (shown by analyzing the responses of single neurons in macaques) of taste in the orbitofrontal cortex includes robust representations of the prototypical tastes sweet, salty, bitter and sour (Rolls et al., 1990), but also separate representations of the taste of water (Rolls et al., 1990), of protein or umami as exemplified by monosodium glutamate (Baylis and Rolls, 1991) and inosine monophosphate (Rolls et al., 1996a, 1998), and of astringency as exemplified by tannic acid (Critchley and Rolls, 1996c). All of these tastes are rewards or punishers, i.e. reinforcers (Rolls, 1999a), and it is important to realize that it is not just some general ‘reward' that is represented in the orbitofrontal cortex, but instead a very detailed and information-rich representation of which particular reward or punisher is present (as shown by the tuning curves for neurons responding preferentially to each of the tastes described in the papers just cited). This is essential given that a choice must continually be made of which reinforcer to work for (or avoid), depending on current need (e.g. homeostatic) states and also on what rewards have been received recently (Rolls, 1999a, 2000c).
There is direct evidence that the reward value of taste is represented in the orbitofrontal cortex. Part of the evidence is that the responses of orbitofrontal taste neurons are modulated by hunger (as is the reward value or palatability of a taste). In particular, it has been shown that orbitofrontal cortex taste neurons stop responding to the taste of a food with which the monkey is fed to satiety (Rolls et al., 1989). In contrast, the representation of taste in the primary taste cortex (Scott et al., 1986; Yaxley et al., 1990) is not modulated by hunger (Rolls et al., 1988; Yaxley et al., 1988). This finding shows that in the primary taste cortex the reward value of taste is not represented, but instead the identity of the taste is represented. Additional evidence that the reward value of food is represented in the orbitofrontal cortex is that monkeys work for electrical stimulation of this brain region if they are hungry, but not if they are satiated (Mora et al., 1979; Rolls, 1999a). Further, neurons in the orbitofrontal cortex are activated from many brain-stimulation reward sites (Mora et al., 1980; Rolls et al., 1980). Thus there is clear evidence that it is the reward value of taste that is represented in the orbitofrontal cortex (Rolls, 1999a).
The secondary taste cortex is in the caudolateral part of the orbitofrontal cortex, as defined anatomically (Baylis et al., 1994). This region projects onto other regions in the orbitofrontal cortex (Baylis et al., 1994), and neurons with taste responses (in what can be considered as a tertiary gustatory cortical area) can be found in many regions of the orbitofrontal cortex (Rolls et al., 1990, 1996b; Rolls and Baylis, 1994). Neurons from these regions project to the hypothalamus and the basal forebrain, and it is probably by this route that hypothalamic neurons receive the inputs which make them respond to the taste and/or sight of food if the monkey is hungry (Fig. 2) (Rolls et al., 1986; Rolls, 1999a).
Convergence of Taste and Olfactory Inputs in the Orbitofrontal Cortex: the Representation of Flavor
In these regions of the orbitofrontal cortex, not only unimodal taste neurons but also unimodal olfactory neurons are found. In addition, some single neurons respond to both gustatory and olfactory stimuli, often with correspondence of tuning between the two modalities (Rolls and Baylis, 1994) (cf. Fig. 2). For example, some neurons respond to the taste of glucose and to the odor of banana. Other neurons respond to salty taste or to savory odours. It is probably here in the orbitofrontal cortex of primates that these two modalities converge to produce the representation of flavor (Rolls and Baylis, 1994). Evidence will soon be described that indicates that these representations are built by olfactory–gustatory association learning, an example of stimulus–reinforcement (e.g. stimulus–reward) association learning.
An Olfactory Representation in the Orbitofrontal Cortex
Takagi, Tanabe and colleagues (Takagi, 1991) described single neurons in the macaque orbitofrontal cortex that were activated by odors. A ventral frontal region has been implicated in olfactory processing in humans (Jones-Gotman and Zatorre, 1988; Zatorre and Jones-Gotman, 1991; Zatorre et al., 1992). Rolls and colleagues have analyzed the rules by which orbitofrontal olfactory representations are formed and operate in primates. For 35% of neurons in the orbitofrontal olfactory areas, Critchley and Rolls (Critchley and Rolls, 1996a) showed that the representation of the olfactory stimulus depended on its association with taste reward (analyzed in an olfactory discrimination task with taste reward). For the remaining 65% of the neurons, the odors to which a neuron responded were not influenced by the taste (glucose or saline) with which the odor was associated. Thus the odor representation for 35% of orbitofrontal neurons appeared to be built by olfactory to taste association learning, where the taste is a primary rewarding or punishing stimulus. This possibility was confirmed by reversing the taste with which an odor was associated in the reversal of an olfactory discrimination task. It was found that 68% of the sample of neurons analyzed altered the way in which they responded to odor when the taste reinforcement association of the odor was reversed (Rolls et al., 1996b). (Of the 68%, 25% showed reversal and 43% no longer discriminated after the reversal. The olfactory to taste reversal was quite slow, both neurophysiologically and behaviorally, often requiring 20–80 trials, consistent with the need for some stability of flavor representations. The relatively high proportion of neurons with modification of responsiveness by taste association was probably related to the fact that the neurons were preselected to show differential responses to the odors associated with different tastes in the olfactory discrimination task.) Thus the rule according to which the orbitofrontal olfactory representation is formed is for some neurons by association learning with taste reward or punishment.
To analyze the nature of the olfactory representation in the orbitofrontal cortex, Critchley and Rolls measured the responses of olfactory neurons that responded to food while they fed the monkey to satiety (Critchley and Rolls, 1996b). They found that the majority of orbitofrontal olfactory neurons decreased their responses to the odor of the food (e.g. fruit juice) with which the monkey was fed to satiety. Thus for these neurons the reward value of the odor is what is represented in the orbitofrontal cortex. These sensory-specific decreases in neuronal responses to odors produced by feeding to satiety with a particular food follow closely the sensory-specific decrease in the pleasantness of the odor of a food produced by feeding to satiety in humans (Rolls and Rolls, 1997). We do not yet know whether this is the first stage of processing at which reward value is represented in the olfactory system [although in rodents the influence of reward association learning appears to be present in some neurons in the pyriform cortex (Schoenbaum and Eichenbaum, 1995)].
An important principle in the representation of reward in the orbitofrontal cortex is that not only is there a detailed representation of different rewarding stimuli (including olfactory, taste, visual and texture stimuli), but also the reward value of each stimulus is updated continually. This is shown by experiments on sensory-specific satiety, in which individual neurons stop responding to the food on which a monkey is satiated in parallel with the decrease in its reward value, but continue to respond to the other stimuli to which they are tuned which remain rewarding (Rolls et al., 1989, 1999a; Critchley and Rolls, 1996b; Rolls, 1999a). Sensory-specific satiety is computed by these neurons in the primate orbitofrontal cortex (Rolls and Treves, 1998; Rolls, 1999a), in that effects of satiety on neuronal responses to sensory stimuli are not found in the primary taste cortex (Rolls et al., 1988; Yaxley et al., 1988) or in the inferior temporal visual cortex (Rolls et al., 1977). This rich representation provided by ensembles of orbitofrontal cortex neurons, each tuned to different sets of olfactory, taste, visual and tactile stimuli and providing detailed information about stimuli for which an animal might work, is very different from that provided by dopamine neurons (Schultz et al., 1995), the activity of which reflects what the animal does rather than the information about the reward value of a wide range of stimuli on the basis of which a decision might be taken (Rolls, 1999a, 2000b). Indeed, in that dopamine release occurs to aversive as well as to rewarding stimuli to which an animal performs actions (Gray et al., 1997), it has been suggested that the dopamine system may be related to processing which reflects whether a behavioral response should be, or is being, performed rather than to reward per se (Rolls, 1999a, 2000b). A way to investigate this further would be to record from dopamine neurons, as well as to measure dopamine release from the terminal areas, when monkeys initiate active responses in order to avoid aversive stimuli (Rolls, 2000b).
Although individual neurons do not encode large amounts of information about which of 7–9 odors has been presented, we have shown that the information does increase linearly with the number of neurons in the sample (Rolls et al., 1996c). This ensemble encoding results in useful amounts of information about which odor has been presented being provided by orbitofrontal olfactory neurons.
Visual Inputs to the Orbitofrontal Cortex, and Visual Stimulus–Reinforcement Association Learning and Reversal
We have been able to show that there is a major visual input to many neurons in the orbitofrontal cortex, and that what is represented by these neurons is in many cases the reinforcement association of visual stimuli. The visual input is from the ventral, temporal lobe, a visual stream concerned with ‘what' object is being seen in that orbitofrontal visual neurons frequently respond differentially to objects or images depending on their reward association (Thorpe et al., 1983; Rolls et al., 1996b). The primary reinforcer that has been used is taste. Many of these neurons show visual–taste reversal in one or a very few trials (e.g. Fig. 3). (In a visual discrimination task they will reverse the stimulus to which they respond, e.g. from a triangle to a square, in one trial when the taste delivered for a behavioral response to that stimulus is reversed.) This reversal learning probably occurs in the orbitofrontal cortex, for it does not occur one synapse earlier in the visual inferior temporal cortex (Rolls et al., 1977), and it is in the orbitofrontal cortex that there is convergence of visual and taste pathways onto the same neurons (Thorpe et al., 1983; Rolls and Baylis, 1994; Rolls et al., 1996a). The probable mechanism for this learning is Hebbian modification of synapses conveying visual input onto taste-responsive neurons, implementing a pattern association network (Rolls and Treves, 1998; Rolls, 1999a). Further evidence that the visual responses of these neurons reflect the reward value of the visual stimuli is that these neurons respond to the sight of a particular food when the monkey is hungry but not when satiated (Critchley and Rolls, 1996b). In doing this, the neurons show sensory-specific satiety, continuing to respond to the sight of other foods which are still rewarding because they have not been fed to satiety (Critchley and Rolls, 1996b). The fact that the responses of these visual neurons in the orbitofrontal cortex reflect the reward value and not the physical properties of the visual stimuli (Thorpe and Rolls, 1983; Rolls et al., 1996b; Critchley and Rolls, 1996b) has been confirmed by Tremblay and Schultz (1999).
|
In addition to these neurons that encode the reward association of visual stimuli, other neurons in the orbitofrontal cortex detect non-reward, in that they respond for example when an expected reward is not obtained when a visual discrimination task is reversed (Thorpe et al., 1983
). Different populations of such neurons respond to other types of nonreward, including the removal of a formerly approaching taste reward and the termination of a taste reward (Thorpe et al., 1983). The presence of these neurons is fully consistent with the hypothesis that they are part of the mechanism by which the orbitofrontal cortex enables very rapid reversal of behavior by stimulus–reinforcement association relearning when the association of stimuli with reinforcers is altered or reversed (Rolls, 1986a, 1990). Different orbitofrontal cortex neurons respond to different types of non-reward (Thorpe et al., 1983), potentially enabling taskor context-specific reversal to occur.
Another type of information represented in the orbitofrontal cortex is information about faces. There is a population of orbitofrontal neurons which respond in many ways similar to those in the temporal cortical visual areas, the properties of which are described elsewhere (Rolls, 1984, 1992a, 1994a, 1995b, 1996a, 1997b; Wallis and Rolls, 1997). The orbitofrontal face responsive neurons, first observed by Thorpe et al. (Thorpe et al., 1983) then by Rolls et al. (Booth et al., 1998; Rolls, 1999a; Rolls et al., 2000), tend to respond with longer latencies than temporal lobe neurons (130–220 ms typically, compared with 80–100 ms); they also convey information about which face is being seen, by having different responses to different faces, and are typically rather harder to activate strongly than temporal cortical faceselective neurons, in that many of them respond much better to real faces than to two-dimensional images of faces on a video monitor (Rolls and Baylis, 1986). Some of the orbitofrontal cortex face-selective neurons are responsive to face gesture or movement. The findings are consistent with the likelihood that these neurons are activated via the inputs from the temporal cortical visual areas in which face-selective neurons are found (see Fig. 2). The significance of the neurons is likely to be related to the fact that faces convey information that is important in social reinforcement. One way in which these neurons carry useful information in such situations is that by encoding face expression (Hasselmo et al., 1989) (e.g. a smile or angry expression), their activation can act as a reinforcer. This may be partly innate and partly by association with a primary reinforcer such as a pleasant touch or pain. Another way in which faceselective neurons may carry useful information in such situations is that they encode information about which individual is present (Hasselmo et al., 1989), which is also important in social situations as learned associations of particular individuals with reinforcers such as touch or pain can again guide behavior. Indeed, in primate social interactions, individuals are constantly updating their evaluation of other individuals in terms of the reinforcers received, and rapid learning of associations between representations of face identity and reinforcers in the primate orbitofrontal cortex is likely to be part of this process.
Somatosensory Inputs to the Orbitofrontal Cortex
Some neurons in the macaque orbitofrontal cortex respond to the texture of food in the mouth. Some neurons alter their responses when the texture of a food is modified by adding gelatine or methyl cellulose, or by partially liquefying a solid food such as apple (Critchley et al., 1993). Another population of orbitofrontal neurons responds when a fatty food such as cream is in the mouth. These neurons can also be activated by pure fat, such as glyceryl trioleate, and by non-fat substances with a fat-like texture, such as paraffin oil (hydrocarbon) and silicone oil (Si(CH3)2O)n). These neurons thus provide information by somatosensory pathways that a fatty food is in the mouth (Rolls et al., 1999a). These inputs are perceived as pleasant when hungry, because of the utility of ingestion of foods which are likely to contain essential fatty acids and to have a high calorific value (Rolls, 1999a). In addition to these oral somatosensory inputs to the orbitofrontal cortex, there are also somatosensory inputs from other parts of the body, and indeed an investigation we have performed with functional magnetic resonance imaging (fMRI) in humans indicates that pleasant and painful touch stimuli to the hand produce greater activation of the orbitofrontal cortex relative to the somatosensory cortex than do affectively neutral stimuli (Rolls et al., 1997a; Francis et al., 1999) (see below).
|
A Neurophysiological Basis for Stimulus–Reinforcement Learning and Reversal in the Orbitofrontal Cortex |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
The neurophysiological evidence and the effects of lesions described suggests that one function implemented by the orbitofrontal cortex is rapid stimulus–reinforcement association learning, and the correction of these associations when reinforcement contingencies in the environment change. To implement this, the orbitofrontal cortex has the necessary representation of primary reinforcers, including taste and somatosensory stimuli. It also receives information about objects, e.g. visual view-invariant information (Booth and Rolls, 1998
), and can associate this at the neuronal level with primary reinforcers such as taste and reverse these associations very rapidly. Another type of stimulus which can be conditioned in this way in the orbitofrontal cortex is olfactory, although here the learning is slower. It is likely that auditory stimuli can be associated with primary reinforcers in the orbitofrontal cortex, though there is less direct evidence of this yet. The orbitofrontal cortex also has neurons which detect non-reward, which are likely to be used in behavioral extinction and reversal. They may do this not only by helping to reset the reinforcement association of neurons in the orbitofrontal cortex, but also by sending a signal to the striatum which could be routed by the striatum to produce appropriate behaviors for non-reward (Rolls and Johnstone, 1992; Williams et al., 1993; Rolls, 1994b). Indeed, the striatal route may be an important one through which the orbitofrontal cortex influences behavior when the orbitofrontal cortex is decoding reinforcement contingencies and their changes (Rolls, 1999a). Some of the evidence for this is that neurons with responses that reflect the output of orbitofrontal neurons are found in the ventral part of the head of the caudate nucleus (Rolls et al., 1983a) and the ventral striatum (Rolls and Williams, 1987; Schultz et al., 1992; Williams et al., 1993) — parts of the striatum that receive connections from the orbitofrontal cortex — and that lesions of the ventral part of the head of the caudate nucleus impair visual discrimination reversal (Divac et al., 1967), which is also impaired by orbitofrontal cortex lesions. The relation between orbitofrontal cortex and striatal processing is considered further elsewhere (Rolls and Johnstone, 1992; Rolls, 1994b, 1999a; Rolls and Treves, 1998).
Decoding the reinforcement value of stimuli, which involves for previously neutral (e.g. visual) stimuli learning their association with a primary reinforcer, often rapidly, and which may involve not only rapid learning but also rapid relearning and alteration of responses when reinforcement contingencies change, is then a function proposed for the orbitofrontal cortex. Using this decoding to specify the goals for action would be important in, for example, motivational and emotional behavior. It would be important in, for example, feeding and drinking by enabling primates to learn rapidly about the food reinforcement to be expected from visual stimuli (Rolls, 1994c, 1999a). This is important, for primates frequently eat more than 100 varieties of food; vision by visual–taste association learning can be used to identify when foods are ripe; and during the course of a meal, the pleasantness of the sight of a food eaten in the meal decreases in a sensory-specific way (Rolls et al., 1983b), a function that is probably implemented by the sensoryspecific satiety-related responses of orbitofrontal visual neurons (Critchley and Rolls, 1996b).
With respect to emotional behavior, decoding and rapidly readjusting the reinforcement value of visual signals is likely to be crucial, for emotions can be described as states elicited by reinforcing signals (Rolls, 1986a,b, 1990, 1995b, 1999a, 2000b). For example, fear is a state produced by a stimulus or event associated with a punisher such as pain. The ability to perform this learning very rapidly is probably very important in social situations in primates, in which reinforcing stimuli are continually being exchanged, and the reinforcement value of stimuli must be continually updated (relearned), based on the actual reinforcers received and given. Although the functions of the orbitofrontal cortex in implementing the operation of reinforcers such as taste, smell, tactile and visual stimuli including faces are most understood, in humans the rewards processed in the orbitofrontal cortex include quite general learned rewards (i.e. secondary reinforcers) such as working for ‘points', as will be described shortly.
Although the amygdala is concerned with some of the same functions as the orbitofrontal cortex, and receives similar inputs (see Fig. 2), there is evidence that it may function less effectively in the very rapid learning and reversal of stimulus reinforcement associations, as indicated by the greater difficulty in obtaining reversal from amygdala neurons (Sanghera et al., 1979; Rolls, 1992b, 2000a; Wilson and Rolls, 2000), and by the greater effect of orbitofrontal lesions in leading to continuing choice of no longer rewarded stimuli (Jones and Mishkin, 1972). In primates, the necessity for very rapid stimulus–reinforcement re-evaluation and the development of powerful cortical learning systems may result in the orbitofrontal cortex effectively taking over this aspect of amygdala functions (Rolls, 1992b, 1999a, 2000a).
|
The Human Orbitofrontal Cortex |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
It is of interest that a number of the symptoms of damage to some parts of the frontal lobes in humans appear to be related to the type of function just described, namely altering behavior when stimulus–reinforcement associations alter, as described next. Thus, some humans with frontal lobe damage can show impairments in a number of tasks in which an alteration of behavioral strategy is required in response to a change in environmental reinforcement contingencies (Goodglass and Kaplan, 1979
; Jouandet and Gazzaniga, 1979; Eslinger and Grattan, 1993; Kolb and Whishaw, 1996). For example, Milner showed that in the Wisconsin Card Sorting Task (in which cards are to be sorted according to the color, shape or number of items on each card depending on whether the examiner says ‘right' or ‘wrong' to each placement), some frontal patients had difficulty either in determining the first sorting principle or in shifting to a second principle when required to (Milner, 1963). Also, in stylus mazes, frontal patients have difficulty in changing direction when a sound indicates that the correct path has been left (Milner, 1982). It is of interest that, in both types of test, frontal patients may be able to verbalize the correct rules yet be unable to correct their behavioral sets or strategies appropriately. Some of the personality changes that can follow frontal lobe damage may also be related to a dysfunction in the alteration of stimulus– reinforcer associations. For example, the euphoria, irresponsibility, lack of affect and lack of concern for the present or future which can follow frontal lobe damage (Hecaen and Albert, 1978; Damasio, 1994) may also be related to a dysfunction in altering behavior appropriately in response to a change in reinforcement contingencies. Indeed, insofar as the orbitofrontal cortex is involved in the disconnection of stimulus–reinforcer associations, and such associations are important in learned emotional responses (above), then it follows that the orbitofrontal cortex is involved in emotional responses by correcting stimulus– reinforcer associations when they become inappropriate.
These hypotheses, and the role in particular of the orbitofrontal cortex in human behavior, have been investigated in recent studies in humans with damage to the ventral parts of the frontal lobe. (The description ‘ventral' is given to indicate that there was pathology in the orbitofrontal or related parts of the frontal lobe but not in the more dorsolateral parts of it.) A task which was designed to directly assess the rapid alteration of stimulus–reinforcement associations was used, because the findings above indicate that the orbitofrontal cortex is involved in this type of learning. This task was used instead of the Wisconsin Card Sorting Task, which requires patients to shift from category (or dimension) to category, e.g. from color to shape, and clearly requires cognitive processing that is different from or additional to the stimulus–reinforcement association learning in which the orbitofrontal cortex is implicated. The task used was visual discrimination reversal, in which patients could learn to obtain points by touching one stimulus when it appeared on a video monitor but had to withhold a response when a different visual stimulus appeared, otherwise a point was lost. After the subjects had acquired the visual discrimination, the reinforcement contingencies unexpectedly reversed. The patients with ventral frontal lesions made more errors in the reversal task (or in a similar extinction task in which the reward was no longer given), and completed fewer reversals, than control patients with damage elsewhere in the frontal lobes or in other brain regions (Rolls et al., 1994). The impairment correlated statistically significantly with the socially inappropriate or disinhibited behavior of the patients (assessed in a Behavior Questionnaire) (Spearman 
= 0.76, P = 0.002) (Rolls et al., 1994). The patients were not impaired at other types of memory task, such as paired associate learning. The continued choice of the no-longer-rewarded stimulus in the reversal of the visual discrimination task is interpreted as a failure to reverse stimulus–reinforcer, that is, sensory–sensory, associations, and not as the motor response perseveration which may follow much more dorsal damage to the frontal lobes, and this is being investigated further in this type of patient. However, I note that one of the types of evidence which bears very directly on this comes from the responses of orbitofrontal cortex neurons in macaques. The evidence comes from the orbitofrontal cortex neurons that respond in relation to a sensory stimulus such as a visual stimulus when it is paired with another sensory stimulus to which the neuron responds such as a taste stimulus (Thorpe et al., 1983; Rolls et al., 1996b). The taste stimulus is a primary reinforcer. These neurons do not respond in relation to motor responses, and could not be involved in stimulus-to-motor response association learning. Bechara and colleagues also have findings that are consistent with these in patients with frontal lobe damage when they perform a gambling task (Bechara et al., 1994, 1996, 1997; Damasio, 1994). The patients could choose cards from different decks. The patients with frontal damage were more likely to choose cards from a deck which gave rewards with a reasonable probability but also had occasional very heavy penalties, resulting in lower net gains than choices from the other deck. In this sense, the patients were not affected by the negative consequences of their actions: they did not switch from the deck of cards which was providing significant rewards even when large punishments were incurred.
It is of interest that in the reversal and extinction tasks the patients can often verbalize the correct response yet commit the incorrect action (Rolls et al., 1994). This is consistent with the hypothesis that the orbitofrontal cortex is normally involved in executing behavior when the behavior is performed by evaluating the reinforcement associations of environmental stimuli (below). The orbitofrontal cortex appears to be involved in this in both humans and non-human primates, when the learning must be performed rapidly, e.g. in acquisition and during reversal.
An idea of how such stimulus–reinforcer learning may play an important role in normal human behavior, and may be related to the behavioral changes seen clinically in these patients with ventral frontal lobe damage, can be provided by summarizing the behavioral ratings given by the carers of these patients. The patients were rated high in the Behavior Questionnaire on at least some of the following: disinhibited or socially inappropriate behavior; misinterpretation of other people's moods; impulsiveness; unconcern or underestimation of the seriousness of their condition; and lack of initiative (Rolls et al., 1994). Such behavioral changes correlated statistically with the stimulus– reinforcer reversal and extinction learning impairment (see above) (Rolls et al., 1994). The suggestion thus is that the insensitivity to reinforcement changes in the learning task may be at least part of what produces the changes in behavior found in these patients with ventral frontal lobe damage. The more general impact on the behavior of these patients is that their irresponsibility tended to affect their everyday lives. For example, if such patients had received their brain damage in a road traffic accident and compensation had been awarded, they often tended to spend their money without appropriate concern for the future, sometimes, for example, buying a very expensive car. Such patients often find it difficult to invest in relationships too, and are sometimes described by their family as having changed personalities, in that they care less about a wide range of factors than before the brain damage. The suggestion that follows from this is that the orbitofrontal cortex may normally be involved in much social behavior, and the ability to respond rapidly and appropriately to social reinforcers is of course an important aspect of primate (including human) social behavior.
To investigate the possible significance of face-related inputs to orbitofrontal visual neurons described above, we also tested the responses of these patients to faces. We included tests of face (and also voice) expression decoding, because these are ways in which the reinforcing quality of individuals is often indicated. Impairments in the identification of facial and vocal emotional expression were demonstrated in a group of patients with ventral frontal lobe damage who had socially inappropriate behavior (Hornak et al., 1996; Rolls, 1999b). The expression identification impairments could occur independently of perceptual impairments in facial recognition, voice discrimination or environmental sound recognition. The face and voice expression problems did not necessarily occur together in the same patients, providing an indication of separate processing. The impairment was found on most expressions apart from happy (which as the only positive face expression was relatively easily discriminable from the others), with sad, angry, frightened and disgusted showing lower identification than surprised and neutral (Rolls, 1999b). Poor performance on both expression tests was correlated with the degree of alteration of emotional experience reported by the patients (Spearman = 0.88, P << 0.05). There was also a statistically strong positive correlation between the degree of altered emotional experience and the severity of the behavioral problems (e.g. disinhibition) found in these patients (Hornak et al., 1996; Rolls, 1999b) (Spearman = 0.66, P < 0.01). A comparison group of patients with brain damage outside the ventral frontal lobe region, without these behavioral problems, was unimpaired on the face expression identification test, was significantly less impaired at vocal expression identification and reported little subjective emotional change (Hornak et al., 1996). These investigations are being extended in current studies, and it is being found that patients with face expression decoding problems do not necessarily have impairments at visual discrimination reversal, and vice versa. This is consistent with some topography in the orbitofrontal cortex (Rolls and Baylis, 1994).
To elucidate the role of the human orbitofrontal cortex in emotion further, Rolls, Francis et al. (Rolls et al., 1997a; Francis et al., 1999) performed an investigation to determine where the pleasant affective component of touch is represented in the brain. Touch is a primary reinforcer that can produce pleasure. They found with fMRI that a weak but very pleasant touch of the hand with velvet produced much stronger activation of the orbitofrontal cortex than a more intense but affectively neutral touch of the hand with wood. In contrast, the affectively neutral but more intense touch produced more activation of the primary somatosensory cortex than the pleasant stimuli (Fig. 4). These findings indicate that part of the orbitofrontal cortex is concerned with representing the positively affective aspects of somatosensory stimuli. The significance of this finding is that a primary reinforcer that can produce affectively positive emotional responses is represented in the human orbitofrontal cortex. This provides one of the bases for the human orbitofrontal cortex to be involved in the stimulus–reinforcement association learning that provides the basis for emotional learning. In more recent studies, we (Rolls, McGlone, Francis, Bowtell and O'Doherty) are finding that there is also a representation of the affectively negative aspects of touch, including pain, in the human orbitofrontal cortex. This is consistent with findings that humans with damage to the ventral part of the frontal lobe may report that they know that a stimulus is pain-producing, but that the pain does not feel very bad to them (Freeman and Watts, 1950; Valenstein, 1974; Melzack and Wall, 1996). It will be of interest to determine whether the regions of the human orbitofrontal cortex that represent pleasant touch and pain are close topologically or overlap. Even if fMRI studies show that the areas overlap, it would nevertheless be the case that different populations of neurons would be being activated, for this is what recordings from single cells in monkeys indicate about positively versus negatively affective taste, olfactory and visual stimuli (above).
|
It is also of interest that nearby, but not overlapping, parts of the human orbitofrontal cortex are activated by taste stimuli (such as glucose) and by olfactory stimuli (such as vanilla) (Rolls et al., 1997b
; Francis et al., 1999). It is not yet known from human fMRI studies whether it is the reinforcing aspects of taste and olfactory stimuli that are represented here, but this is likely in view of the findings in non-human primates (Rolls, 1999a) and a recent paper showing orbitofrontal cortex activation in humans that is related to olfactory sensory-specific satiety (O'Doherty et al., 2000). |
Conclusions |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
The investigations described here show that the primate orbitofrontal cortex is involved in representing primary (unlearned) reinforcers such as taste and touch, and in learning associations of other stimuli, such as visual and olfactory stimuli, with these primary reinforcers. For these reasons, the orbitofrontal cortex has important functions in motivational behavior such as feeding and drinking, and in emotion and social behavior (Rolls, 1999a
). The type of learning in which the orbitofrontal cortex is involved is stimulus–reinforcer association learning, which is a particular case of stimulus–stimulus association learning. The model for the implementation is a pattern association between the conditioned (to-be-learned, e.g. visual) stimulus, which activates the output neurons through associatively modifiable synapses, and the primary reinforcer, which activates the neurons through nonmodifiable synapses (Rolls and Treves, 1998). Once learned, the same conditioned stimulus will activate the output neurons, with no need for ongoing activity of neurons in order to implement the memory. If the contingency reverses, the synapses from the neurons representing the previous conditioned stimulus are no longer active when the output neuron is active, and the synapses become weaker by a process of heterosynaptic long-term depression (Rolls and Treves, 1998).
Stimulus–reinforcer association memory is distinct from the type of working memory implemented in the dorsolateral and inferior convexity prefrontal areas. The model for the implementation of such working memories is an autoassociation neural network in which the memory state is kept active by continuously recirculating neuronal activity implemented by recurrent collateral associatively modifiable synapses between the pyramidal cells (Rolls and Treves, 1998). The dorsolateral part of the prefrontal cortex receives inputs particularly from area 7 of the parietal cortex, and may be especially involved in spatial response working memory; while the inferior convexity prefrontal cortex receives activity particularly from the inferior temporal visual cortex, and may be more involved in object working memory (Fuster, 1997; Goldman-Rakic, 1996; Rolls and Treves, 1998). The orbitofrontal cortex stimulus–reinforcer pattern association memory is also very distinct from the episodic declarative memory in which the hippocampal system is implicated. This system may store memories by forming arbitrary associations between conjunctive events which need not be reinforcers and which typically include a spatial component. The storage may occur using an autoassociation network which does not operate in a continuous attractor mode in order to store a memory (Rolls, 1996b; Rolls and Treves, 1998).
One set of output pathways by which the orbitofrontal cortex implements these functions for behavior is via the striatum (Rolls, 1996a, 1999a). Another output of the orbitofrontal cortex is to the hypothalamus, and it is probably by this route that hypothalamic neurons in primates come to respond to the sight and taste of food when hunger is present [Fig. 2; further evidence is presented elsewhere (Rolls, 1999a)]. The functions of this output system, and the orbitofrontal cortex connections which are directed further caudally in the brainstem, include autonomic and endocrine responses learned and updated to changing environmental stimuli (Rolls, 1999a).
|
Notes |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for...The Human Orbitofrontal CortexConclusionsNotesReferences |
|---|
The author has worked on some of the experiments described here with L.L. Baylis, G.C. Baylis, R. Bowtell, A.D. Browning, H.D. Critchley, S. Francis, M.E. Hasselmo, J. Hornak, C.M. Leonard, F. McGlone, F. Mora, D.I. Perrett, T.R. Scott, S.J. Thorpe, E.A. Wakeman and F.A.W. Wilson, and their collaboration is sincerely acknowledged. Some of the research described was supported by the Medical Research Council, PG8513790.
Address correspondence to Edmund T. Rolls, University of Oxford, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, UK. Email: edmund.rolls{at}psy.ox.ac.uk.
|
References |
|---|
|
TopAbstractIntroductionConnectionsEffects of Lesions of...Neurophysiology of the...A Neurophysiological Basis for... |
|---|