Cerebral Cortex Advance Access originally published online on August 28, 2007
Cerebral Cortex 2008 18(5):1160-1168; doi:10.1093/cercor/bhm150
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Action Outcomes Are Represented in Human Inferior Frontoparietal Cortex
1 Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA, 2 Department of Psychology University of California, Santa Barbara Santa Barbara, CA 93106, USA
Address correspondence to Dr Antonia F. de C. Hamilton, School of Psychology, University of Nottingham, University Park, Nottingham NG7 2RD, UK. Email: antonia.hamilton{at}nottingham.ac.uk.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
The simple action of pressing a switch has many possible interpretations—the actor could be turning on a light, deleting critical files from a computer, or even turning off a life-support system. In each of these cases, the motor parameters of the action are the same but the physical outcome differs. We report evidence of suppressed responses in right inferior parietal and right inferior frontal cortex when participants saw repeated movies showing the same action outcome, but these regions did not distinguish the kinematic parameters by which the action was accomplished. Thus, these brain areas encode the physical outcomes of human actions in the world. These results are compatible with a hierarchical model of human action understanding in which a cascade of specialized processes from occipital to parietal and frontal regions allow humans to understand the physical consequences of actions in the world and the intentions underlying those actions.
Key Words: Action understanding • Intention • Mirror neuron • Outcome • Parietal
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
Hierarchical theories of human motor control have an august history (Sherrington 1906
; Hebb 1949; Bernstein 1996). Such models distinguish between different levels of motor representation, for example, a muscle level, a kinematic level (reach and grasp), an object-goal level (grasp a hammer), and an outcome level (hammer a nut). The distinction between these 4 levels is not only descriptive but also likely to reflect the fundamental organization of action in the central nervous system.
Much is known about the organization of the lowest levels of the motor hierarchy, including the representation of muscle force (Evarts 1968; Porter and Lemon 1993) and reach and grasp actions (Jeannerod et al. 1995) in the primary and premotor cortex. In contrast, few studies have attempted to examine the higher levels of the motor hierarchy, in particular the levels of goals and outcomes. Increasing evidence suggests that this more abstract level of representation is important for both motor and social cognition. Several psychophysical studies have shown that human actions are encoded in terms of their outcomes (Hommel et al. 2001; Mechsner et al. 2001), and observed outcomes provide an important cue to an actor's mental state (Frith CD and Frith U 2006). The aim of the current paper was to examine the neural representation of action outcomes in the human brain based on action observation. By doing so, we can advance our knowledge of both hierarchical motor control and social action understanding.
The parallel study of social and motor cognition is made possible by discovery of mirror neurons in the macaque brain, which encode actions of the self and others (di Pellegrino et al. 1992; Gallese et al. 1996). Several theorists have suggested that mirror neurons may be important for representing action goals, intentions, and outcomes for the self and for others (Gallese and Goldman 1998; Rizzolatti and Fadiga 1998). However, empirical demonstrations of neurons encoding a goal or outcome other than object grasping in the macaque brain are rare. Mirror neurons in the macaque inferior frontal gyrus (IFG) respond when an action is inferred even if it is not seen (Umilta et al. 2001), whereas neurons in macaque inferior parietal lobule (IPL) encode specific action sequences for the self and others (Fogassi et al. 2005). These results suggest that inferior frontal or inferior parietal cortex might contain goal or outcome representations. Unfortunately, the macaque experiments did not record from both regions under the same conditions and did not systematically separate goals from the underlying kinematics which achieve the goal, so the relative contribution of these different cortical regions to understanding goals and intentions remains undefined.
In the human brain, a mirror neuron system (MNS) has been localized to the IFG and IPL (Rizzolatti and Craighero 2004). Many studies have shown that IFG and IPL respond when hand actions are performed (Grafton et al. 1992), imagined (Grafton et al. 1996), observed (Buccino et al. 2001), planned (Johnson et al. 2002), and imitated (Iacoboni et al. 1999; Buccino et al. 2004; Aziz-Zadeh et al. 2006). However, these studies are limited in 2 critical ways. The majority of studies activated all 4 putative mirror neuron regions (left and right IPL and IFG) and did not help us understand the differential processing of information within and between these regions. Similarly, none of these studies were able to distinguish between different levels of the action hierarchy, in particular between the kinematic level and the goal or outcome level. This is because all actions are made up of a wide variety of features including low-level kinematic parameters, such as hand speed and finger configuration, as well as more abstract goals, intentions, and outcomes. It is almost impossible to constrain natural and rich stimuli in order to separate different types of action representation with simple subtraction designs because an "action without a goal" or an "outcome without an action" is seldom plausible.
Recognizing this confusion between kinematic, goal, and outcome representations, Jacob and Jeannerod (2005) recently argued that the MNS provides only kinematic representations (motor intentions), which are insufficient to determine a "prior" intention. For example, pressing a switch might in one situation turn on a light but in another might turn off a life-support machine, and presumably the intention of the actor is very different in each case. Under Jacob's theory, when you see a person press a switch, the mirror neuron regions should encode the kinematic parameters of the pressing action, such as the shape of the hand as it grasps, but not the outcome of that action—turning on the light—nor the prior intention to turn on the light which guided the outcome. If this interpretation is correct, bold theories which attempt to link the MNS to intentionality and theory of mind (Gallese and Goldman 1998; Gallese 2003) would have to be reassessed.
A new way to discriminate between kinematic, goal, and outcome levels of representation is to measure repetition suppression (RS) of the blood oxygen level–dependent (BOLD) signal in response to observed videos showing novel or repeated kinematics, goals, and outcomes. In 2 previous studies using an RS method, we showed that left anterior intraparietal sulcus (aIPS) represents object goals. Object goal was defined by the identity of the object grasped by a person, for example, a "take-cookie" goal compared with a "take-diskette" goal (Hamilton and Grafton 2006, forthcoming). The present study generalizes and extends this result by examining the neural representation of outcomes in relation to kinematic parameters. We define an outcome as the (desired) physical consequence of an action in the world, for example, pushing a light switch results in the outcome of a lit lamp in one case or the outcome of a dim lamp in another. In the current study, we did not attempt to isolate a representation at the object-goal level (grasping the light switch), which precedes the action outcome. Instead, we examined the representation linked to the outcome of lighting the lamp, which completes the action. Thus, outcomes involve more complex actions with differing effects in the world and may be represented at a higher level of the action hierarchy than object goals.
The current paper distinguishes the representation of outcomes from those related to the kinematic components of actions used to generate those outcomes (e.g., pushing vs. pulling, etc.). In order to localize brain activity at these 2 levels, we carried out an functional magnetic resonance imaging (fMRI) study measuring RS to sets of action movies with novel and repeated outcomes and novel and repeated kinematic parameters. This achieves 2 objectives. First, we could address Jacob's challenge and determine if the putative human MNS represents intentions and outcomes rather than just kinematics. Second, we would be able to extend our knowledge of the motor hierarchy for action observation, building on our previous work examining goal representations (Hamilton and Grafton 2006, forthcoming). We tested if the representation of observed outcomes was found in a different part of the putative MNS from the representation of observed kinematic features within each movement. Given the variability of recruitment of local cortical areas within the MNS in prior studies of action observation, we compared the relative involvement of inferior frontal and parietal cortex and also tested for differences in hemispheric involvement.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
We used RS in an event-related fMRI experiment to localize the neural representation of action outcomes. First, 12 distinct sets of action movies were generated (see Supplementary Fig. 1). Within every set, there were 2 possible outcomes and 2 possible kinematic means of achieving that outcome. For example, a box with a sliding lid can be opened or closed by either pushing the lid with a finger or pulling with both finger and thumb (Fig. 1). The sets differed in low-level stimulus characteristics such as lighting and the precise objects used. The wide variety of stimuli meant that we could be confident that the results revealed a general neural representation of outcomes, rather than being a quirk of one particular stimulus set. All movie stimuli created by filming intentional natural, right-handed actions on an object to cause a particular outcome. In 50% of movies, the left hand stabilized an object in the scene but remained still for the duration of the clip. Care was taken to ensure that only the outcome or action varied between clips, all other cues were constant. Clip durations ranged from 2.5 to 9 s, according to the natural duration of the event depicted but were constant within a set.
|
Twenty right-handed participants (7 males, mean age 20.8 years) gave their informed consent to take part in the study in accordance with the requirements of the local ethics board. Participants watched the 12 sets of movies presented in a pseudorandom order during fMRI scanning. They were instructed to watch carefully and press a key if the movie froze in the middle of the action (stop trials, 12% of total). Each stop trial ended a set and was followed by a rest of 4–8 s before the next set of movies began. Intertrial interval was 500 ms. All stimuli were presented with Cogent running under Matlab 6.5, which provides accurate trial timing and synchronization with the scanner.
Within each set of movies, the movies were presented in a pseudorandom order according to a one-back RS design (Fig. 1). A different pseudorandom trial order was used for every set of movies and every participant to ensure that the results reflect only the desired one-back RS effects and are not contaminated with any second- or higher order effects. For the purposes of analysis, each movie was classified relative to the previous movie as either novel Outcome–novel Kinematics (nOnK), novel Outcome–repeated Kinematics (nOrK), repeated Outcome—novel Kinematics (rOnK), or repeated Outcome–repeated Kinematics (rOrK). Participants saw 11 RS trials for each set of movies, so conditions were not precisely balanced within set but were balanced over sets. That is, over the 12 sets of movies, each participant saw 132 RS trials falling evenly into the 2 x 2 factorial design for Outcomes and Kinematics. A post hoc analysis of the actual trial order was conducted to check for any possible artifacts due to the randomization algorithm. This analysis showed that the probability that an nO trial was preceded by an nK trial was equal to the probability that it was preceded by an rK trial and similarly for all other combinations of trial types. Thus, we can be confident that the reported results are not due to artifacts of the trial sequence.
The experiment was carried out in a 3T Philips Achieva Quasar Dual 8 channel scanner using an 8 channel–phased array coil and 30 slices per time repetition (TR) (4-mm thickness, 0.5-mm gap), TR: 1975 ms, time-echo: 35 ms, flip angle: 90°, field of view: 24 cm, matrix: 80 x 80. For each of 4 functional runs, the first 2 brain images were discarded, then 240 images were collected and stored. Data were realigned, unwarped, and normalized to the Montreal Neurological Institute template with a resolution of 2 x 2 x 2 mm in the Statistic Parametric Mapping software package (SPM2, www.fil.ion.ucl.ac.uk/spm). A design matrix was fitted for each subject with one regressor for each movie type (nOnK, nOrK, rOnK, and rOrK) in each set of movies plus a regressor each for New and Stop movies in each run, giving a total of 56 regressors. Each movie was modeled as a boxcar with the duration of that movie convolved with the standard hemodynamic response function. Every design matrix was tested for the efficiency of the outcome and kinematic contrasts and for the orthogonality of the regressors (Price et al. 2004) in order to be sure that the conditions were fully independent. The design matrix weighted each raw image according to its overall variability to reduce the impact of movement artifacts (Diedrichsen and Shadmehr 2005). After estimation, 9-mm smoothing was applied to the beta images.
To identify brain regions showing RS for outcomes, we calculated a contrast for the main effect of Outcome (novel > repeated) over all movies. Contrast images for all 20 participants were taken to the second level for a random effects analysis. We report regions that survive a threshold of P < 0.001 uncorrected and 10 voxels over the whole brain in Table 1. Our discussion of the results focuses on the regions within this set, which passed correction for multiple comparisons at this threshold (t > 3.58 and 10 voxels) and the cluster-corrected level, searching over the whole brain. In addition, we tested for RS for outcome in a region of interest (ROI) in the left aIPS (10-mm radius, centered on –52, –32, 44), defined by our previous work on object goals (Hamilton and Grafton 2006).
|
An ROI analysis was used to test for differences in the magnitude of the RS effect between the left and right IPL and IFG. We used a voxel counting approach with large ROIs because we do not have localizer scans or precise "a priori" predictions. Four ROIs were generated, covering the left and right IFG and left and right IPL as defined in previous studies of the MNS (Iacoboni et al. 1999
; Johnson-Frey et al. 2003, 2005; Iacoboni et al. 2005). Each ROI contained over 3000 voxels, providing an inclusive definition of the frontal and parietal components associated with the putative human MNS. For each participant, we localized the strongest RS effect by examining the magnitude of the main effect of Outcome in every voxel over all 4 ROIs and defining the voxels which were in the top 5% for effect size as "strong RS voxels." If the outcome effect was equally distributed between all 4 ROIs, we would expect 5% of the voxels in each ROI to be "strong RS voxels" by chance. For each participant and each ROI, the total number of strong RS voxels was calculated. This gives a measure of the extent of the RS effect in each ROI, regardless of individual differences in the global level of the BOLD signal. A repeated-measures analysis of variance (ANOVA) with factors lateralization (left–right) and site (IFG–IPL) was then used to test if strong RS voxels were preferentially located in one of the 4 ROIs. The same analysis was repeated with thresholds of 20% and 1% to ensure generality. To identify brain regions showing RS for the kinematic features of an action, we calculated a contrast for the main effect of kinematics (novel > repeated), regardless of the outcome of the observed action. We report regions that survive a threshold of P < 0.001 uncorrected and 10 voxels over the whole brain in the lower part of Table 1. An analysis of the distribution of the top 5% of voxels for the kinematics contrast (strong RS voxels) was also carried out to determine if the left or right inferior frontal or parietal regions preferentially encoded kinematic features and to check that this method of analysis was not systematically biased by physiological factors.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
A stronger response to novel outcomes than to repeated outcomes was found in regions throughout the expected frontoparietal action circuits (Table 1). Two of these regions, the right IPL and the right IFG extending to the inferior frontal sulcus, survived the whole-brain cluster-corrected threshold and are illustrated in Figure 2. In both these clusters, the robust response to novel outcomes was suppressed when the same outcome was repeated on a second trial, regardless of the hand action. This pattern of data signifies that outcomes rather than motor parameters are encoded in neuronal populations within these regions (Grill-Spector and Malach 2001
; Hamilton and Grafton 2006). Because of previous evidence for an RS effect for the goal of a simple grasping action in left aIPS (Hamilton and Grafton 2006), an ROI analysis was performed in this area to test for RS for the more complex outcomes. Significant suppression for repeated outcomes was seen in this region (Fig. 2) as part of a larger cluster extending into the postcentral sulcus.
|
The plots of parameter estimates shown in Figure 2 also hint at an interaction between RS for outcome and for kinematics, but an analysis of interactions over the whole brain did not reveal any effect in this region even at the weaker P < 0.001 threshold. This further supports the conclusion that these areas are primarily sensitive to novelty for outcome irrespective of the kinematic parameters. Analysis of the responses of right IPL to each individual set of movies indicates that these effects are not driven by a single stimulus but generalize across a wide variety of actions (Supplementary Fig. 1). A plot of a poststimulus time histogram in this region confirms that activity started near zero in all conditions (Supplementary Fig. 3).
A further analysis was used to determine if the RS effect for outcomes differentially recruited inferior parietal or inferior frontal cortex and if there was an interaction with hemisphere. Descriptively, the peak RS effect in the right IPL region was 50% greater than the next largest peak RS effect in the right IFG region, and the right IPL cluster contains over 4 times the number of voxels found in the right IFG cluster (Table 1). A t-test directly comparing the magnitude of the RS effect in the peak voxel of right IPL to the peak voxel of right IFG revealed a significantly stronger effect size in the parietal region (t = 1.77, degrees of freedom (df) = 19, P = 0.046). A 2 x 2 factorial repeated-measures ANOVA comparing the number of strongly activated voxels in each participant and each ROI revealed a main effect of side (F = 9.01, df = 1,19, P = 0.007) with a stronger RS effect on the right compared with the left. There were no effects of lobe (F = 0.19), but there was an interaction between lobe and hemisphere (F = 4.44, df = 1,19, P = 0.049), with the more voxels activated in the right IPL compared with other regions. This effect is illustrated in Figure 3 (dark bars). Two secondary analyses using harsher (top 1%) and weaker (top 20%) thresholds to define the number of strongly activated voxels gave a similar pattern of results for the main effect of side (P < 0.025 for both thresholds) but a mixed result for the interaction (P < 0.014 for the 20% threshold but not significant for the 1% threshold). This is likely due to floor effects on the number of voxels in the 1% case.
|
These results demonstrate a clear lateralization of the RS effect for outcomes to the right hemisphere. Though both the right IFG and right IPL showed robust RS effects for outcomes, the ANOVA analysis also revealed a stronger effect in the right IPL ROI, leading to the conclusion that this is the dominant region for outcome representations in the brain.
An analysis of novel kinematic features compared with repeated kinematic features did not yield any region with significance exceeding the corrected statistical thresholds accounting for a whole-brain search volume. In an exploratory analysis, RS for kinematics passed an uncorrected threshold (P < 0.001 and 10 voxels) in left middle intraparietal sulcus, left lateral occipital cortex, and left superior temporal sulcus (Supplementary Fig. 2). These results are consistent with previous data showing RS in left lateral occipital cortex for hand trajectory (Hamilton and Grafton 2006) and grasp (Hamilton and Grafton, forthcoming).
The analysis of the distribution of the 5% of voxels with the strongest RS effect (strong RS voxels) was also conducted using data from the kinematics contrast in order to check the generality of the method, and results are illustrated in Figure 3 (light bars). An ANOVA on the number of voxels strongly activated for the kinematics contrast in each ROI of each participant revealed a significant lobe by hemisphere interaction (F = 5.97, df = 1,19, P = 0.024) driven by the large number of voxels in left IFG. There were no significant main effects (all P > 0.3). The finding of excess "kinematic" voxels in left IFG is in line with other results from our laboratory showing RS for grasps in the left IFG (Hamilton and Grafton, forthcoming). The apparent discrepancy between this result and the lack of a main effect of kinematics in left IFG could be due to intersubject variability in IFG, but further studies will be required to examine this issue fully. Importantly, the distribution of highly active voxels in the kinematics contrast demonstrates that this method of examining voxel distribution is not systematically biased toward any one brain region by physiological factors but rather provides a valuable assessment of effect sizes in the different lobes of the brain.
In addition to these analyses, the data were examined at the single-subject level as described in the Supplementary Material. The results broadly support our contention that the right IPL shows RS for the outcomes of human actions but also emphasize the need for further studies of individual differences in the localization of high-level action representations in the brain.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
Our data demonstrate a robust RS effect for outcomes in right IPL and right IFG, regardless of the kinematic features of the observed action generating the outcome. RS occurs when a population of neurons encoding a particular stimulus characteristic gives a weaker response when that characteristic is repeated (Grill-Spector and Malach 2001
). Thus, our data demonstrate that human IPL and IFG contain populations of neurons, which encode the outcome of an observed action. These results are concordant with previous data implicating IPL and IFG in goals (Hamilton and Grafton 2006) and intentions (Fogassi et al. 2005; Iacoboni et al. 2005).
The present results have important repercussions for models of human action understanding in relation to the brain. In particular, our data provide some preliminary evidence against the claim that the inferior frontal and inferior parietal brain regions commonly termed the MNS represent only the kinematic parameters, such as the shape of the hand during observed grasping, and cannot reflect higher order intentions (Jacob and Jeannerod 2005). Before addressing the issue of intention representations in the brain, we first consider possible confounds in our data and summarize how our results fit into an emerging hierarchical model of human action understanding.
Limitations
There are 3 possible confounds which might affect our interpretation of these data. First, there is evidence that the kinematics of performed hand actions are altered by the goal of the action (Ansuini et al. 2006), so it could be argued that our outcome RS reflects a subtle sensitivity to kinematic parameters. However, if this were the case, we would expect to see similar RS for both the subtle kinematic effects when repeating particular outcomes and the more dramatic kinematic differences in the kinematics contrast. This was not the case, and similar results were obtained in other RS studies of high-level action representations (Hamilton and Grafton 2006, forthcoming). Thus, it is implausible to suggest that kinematic representations rather than outcome representations underlie the observed effects.
The second question which arises from this result is the possibility of an attentional or task switching confound. The right parietal cortex is associated with spatial attention (Corbetta and Shulman 2002) and the right inferior frontal sulcus with task switching (Brass et al. 2005)—could the sequence of repeated and novel stimuli have engaged these more general processes? We suggest that the data do not support this interpretation. fMRI (Corbetta and Shulman 2002), transcranial magnetic stimulation (Mevorach et al. 2006), and lesion (Husain and Rorden 2003) studies of attention and salience all demonstrate a role for posterior inferior parietal cortex in this function. The locus of the RS for outcome we report is in the anterior IPL, which does not have a specific role in attention. Furthermore, our experiment used a one-back RS design, where every movie functions as both "prime" and "target." Participants have no awareness of the sequence and performed the same cognitive task (detect the stop movies) throughout the experiment, which rules out a switching account. There were no systematic differences in the use of space between the video clips, and all clips within a set showed the same objects in contact with the actor's hands. This rules out spatial or object-based attentional explanations of our data. Any brain mechanism that detects the sequence of repeated or changing action outcomes and motor parameters is the subject of our study, and we suggest that it would be a tautology to label such a mechanism "attention" or "switching" without specifying how and why it detects these changes.
A third limitation arises from our incomplete knowledge of the neural mechanisms underlying RS effects and their link to behavior. Several different mechanisms have been proposed (Grill-Spector et al. 2006; James and Gauthier 2006; Krekelberg et al. 2006). These differ in their attitude to general neural suppression versus more rapid responses versus the sharpening of neuronal tuning curves but agree that a population-code explanation for suppression of the BOLD signal and the release from suppression is plausible. Neurophysiological recordings from visual processing regions support this conclusion (Desimone 1996; Van Wezel and Britten 2002). Though single cell recordings have not yet been made in parietal or premotor regions to test for RS, fMRI evidence suggests that RS occurs throughout the brain in domains as varied as number, object, and syntax (Buckner et al. 1998; Noppeney and Price 2004; Piazza et al. 2004).
Some RS studies go further than ours and make links between RS in the brain and behavioral priming (Wig et al. 2005), but this was not possible in the present experiment. Normal participants perform perfectly in tasks requiring psychophysical judgments about the intentions and actions present in the simple video clips, and reaction times cannot be measured for video stimuli with a duration of several seconds. Thus, behavioral measures were not available in the present task. This means that we are able to avoid the complex debate over the precise links between RS and behavioral priming and focus on the most general and parsimonious interpretation of our RS effects. This is the proposal that suppression of the BOLD signal reflects the repeated engagement of the same neuronal population and release from suppression reflects the engagement of a different neuronal population within the same brain region (Grill-Spector and Malach 2001; Hamilton and Grafton, forthcoming). Therefore, we suggest that the observed RS effects for outcome mean that right IPL and right IFG contains populations of neurons that encode the physical outcomes of other people's actions.
A Hierarchy for Observed Actions
The results presented here demonstrate that the outcomes of observed actions are encoded in a right lateralized, frontoparietal circuit. Specifically, evidence for outcome representations was found in the right IFG, the right IPL, and a cluster extending from the left postcentral sulcus to the left aIPS. Previous work has associated an inferior parietal–inferior frontal network with the performance and observation of complex, goal-directed hand actions (Grafton et al. 1996; Ehrsson et al. 2000). In particular, some studies implicate the left intraparietal sulcus in performing grasps (Culham et al. 2003; Frey et al. 2005) and in the observation of grasps (Shmuelof and Zohary 2006), whereas others implicate the IFG in the observation (Iacoboni et al. 2005) and imitation of goals (Koski et al. 2002). As these studies indicate, the question of the representation of goals, grasps, and other kinematic parameters in the frontoparietal motor circuit is a complex one which is unlikely to have a straightforward answer. Furthermore, these studies did not use RS methods and did not systematically aim to distinguish between different levels of the motor hierarchy.
We suggest that our data can best be interpreted in the context of previous studies which used RS to examine object-goal, trajectory, and grasp representations (Shmuelof and Zohary 2005; Hamilton and Grafton 2006, forthcoming). The study by Shmuelof et al. revealed encoding of different types of grasp in aIPS, postcentral sulcus, and precentral sulcus, as well as encoding object identity (goal) in the left aIPS and in visual regions. Congruent with these results, the studies from our own laboratory demonstrate that the left aIPS encodes the object goal of an action, whereas inferior frontal and lateral occipital regions encode kinematic features of an observed action, such as trajectory and hand shape. The data from the previous studies were used to develop a hierarchical model of action understanding, in which goal representations in the inferior parietal regions sit at the top of the hierarchy, whereas visual kinematic representations in lateral occipital regions and motor kinematic representations in inferior frontal regions provide a lower level analysis of action (Hamilton and Grafton, forthcoming). The new data demonstrating a particular role for the right IPL and, to a lesser extent, right IFG in encoding the outcomes of observed actions complement and extend the previous model. It also reinforces our argument that, despite the widespread assumption that abstract and complex entities must be linked to the frontal cortex, in fact the parietal cortex is also important for higher level action representations of goals and outcomes.
There are also some key differences between the present results and the prior studies of goal representation, which can help us understand the brain systems for action understanding in more detail. Our 2 previous studies examined object goals, defined by the identity of a grasped object. In contrast, the current study examined the more complex case of outcomes, defined by the physical consequences of the action of a person. In parallel with this distinction in the precise nature of the action understanding, we found a striking difference in brain lateralization. Although both studies of object goals reported robust left parietal effects, the outcomes studied here induced strong right parietal effects. This means that for the brain, object goals are not the same as outcomes. Instead, these 2 different representations may fall at different levels in the motor hierarchy or have different functions in understanding the intentions of others. To gain some insight into the reasons for this pattern of lateralization, we must turn to studies characterizing lateralization of motor control in the human brain.
A Hierarchy for Performed Actions
Few studies distinguish between different hierarchical levels of action representation, in particular for complex, goal-oriented behavior. Beginning at the lower levels, primary motor cortex clearly provides the most primitive cortical motor representation. Slightly more complex representations of different grasp configurations and kinematic parameters can be found in the inferior frontal and premotor cortex (Rizzolatti et al. 1988; Ehrsson et al. 2000). These regions are intimately connected with the primary motor cortex and even the spinal cord (Dum and Strick 2002) and are likely to provide a kinematic representation of action. For example, IFG is robustly activated during imitation of simple finger movements (Iacoboni et al. 1999).
Moving to higher levels of the hierarchy, many studies report the activation of both inferior frontal and inferior parietal cortex in a variety of planning tasks (Johnson-Frey et al. 2005) and action tasks (Rizzolatti et al. 1996), and it is clear that these regions function as a closely integrated network. Inferior parietal cortex and the cerebellum also seem to have a role in using and switching between internal models for the control of tools (Imamizu et al. 2004). However, there is some evidence for more abstract, goal representations in the inferior parietal cortex, in particular from studies of patients with apraxia (Haaland et al. 2000; Buxbaum et al. 2005) and in studies which use transcranial magnetic stimulation to disrupt goal processing in typical participants (Tunik et al. 2005). Similarly, neurophysiological recordings from the monkey IPL report mirror neurons for complex action sequences in this region (Fogassi et al. 2005).
All these data suggest a hierarchy for performed actions, from kinematic representations in IFG to more complex goal representations in IPL, which matches the hierarchy for observed actions described above. However, there is also a distinct lateralization of observed action representations in the parietal cortex, with observed object goals represented in the left IPL (Hamilton and Grafton 2006) and observed outcomes represented in the right IPL. Can this distinction also be made for performed actions?
Many studies of high-level action planning and execution report a left lateralization. For example, planning of grasping actions activates the left IFG and IPL regardless of the hand used (Johnson-Frey et al. 2005), and the left IPL is activated for hand–object interactions of either hand (Naito and Ehrsson 2006). Similarly, patients with damage to left IPL have difficulty pantomiming or performing tool-use actions (Haaland et al. 2000). These data point to a role for the left IPL in planning goal-directed reaching and grasping actions. However, there is also increasing evidence for a role for right IPL in complex action planning. Patients with right IPL damage have particular difficulties with multistep actions (Hartmann et al. 2005) and with imagined action (Sirigu et al. 1996). Right IPL is robustly activated in tasks requiring multistep planning such as the Tower of London task (Baker et al. 1996; Newman et al. 2003) and when participants must plan and remember future actions (den Ouden et al. 2005). This region may also be particularly important for assessing feedback on performed actions (Blakemore et al. 1998; Farrer et al. 2003; Ogawa et al. 2006), which is essential in multistep and goal-oriented behaviors.
Together, all these data on action planning and execution support the idea that there is a hierarchy for performed action representations, with kinematic parameters found in the IFG, planning of goals, such as taking a tool in the left IPL, and monitoring of action sequences and action outcomes in right IPL. Taken together, the joint hierarchical models of action execution and action observation have important implications for our understanding of mirroring and intention representation in the brain.
Mirroring in the Action Hierarchy
The 2 hierarchical systems described above, for action observation and for action performance, can be mapped together. That is, it seems that the right inferior parietal cortex contributes more to the representation of the outcomes of complex actions for the self and for others, whereas the left inferior parietal cortex is related more to motor plans and goals for the self and for others. This result implies that a strong principle of mirroring may exist within frontoparietal cortex and that there is a tight locking between the neural activations for self and other. However, this tight locking need not imply a unitary "direct-matching" mechanism acting in a single step over the whole MNS (Gallese 2003). Rather, the application of the principle of mirroring at each level of a complex action hierarchy supports more sophisticated, multistage models of goal and intention understanding (Csibra, forthcoming).
The suggestion of mirroring for outcomes and intentions also has important implications for Jacob's challenge to the mirror neuron theory of intention understanding (Jacob and Jeannerod 2005). Jacob argued that the MNS represents only kinematic properties of actions (motor intentions) and not prior intentions. The data presented here demonstrate that the right IPL and right IFG encode outcomes, which are independent of the kinematic parameters of the action. The outcomes in our study, such as turning on a light or opening a box, meet Jacob's definition of a prior intention (Jacob and Jeannerod 2005). Thus, our data demonstrate that other people's prior intentions are represented in parts of the human MNS, contradicting part of Jacob's hypothesis. In addition, a series of papers demonstrate a role for the right IPL in controlling and planning one's own intentional actions (Baker et al. 1996; Blakemore et al. 1998; Farrer et al. 2003; Newman et al. 2003; den Ouden et al. 2005; Hartmann et al. 2005; Ogawa et al. 2006). Thus, it seems possible that the principle of mirroring applies at the level of intentions and outcomes. We note that this does not mean that the MNS must also represent mental states, such as beliefs (Gallese and Goldman 1998), which are likely encoded elsewhere (Saxe 2005).
As a neuroimaging study, the current paper cannot demonstrate that individual neurons that encode the outcomes of other people's actions also encode the outcomes of one's own actions, but our data is congruent with single-unit recordings in the macaque IPL (Fogassi et al. 2005). Further experiments, in both humans and nonhuman primates, will be required to fully explore the overlap of outcome and intention representations for self and other and to provide a definitive test of whether mirroring applies to prior intentions. Our data provide some initial evidence that Jacob's challenge may fail and that mirror neuron regions of the human brain are not restricted to kinematic parameters but represent complex action outcomes.
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
This paper demonstrates that RS for the outcomes of complex actions is found in the right IPL and right IFG. Therefore, these brain regions contain populations of neurons that encode observed action outcomes, regardless of the kinematic parameters of the action. These results support a hierarchical model of action understanding (Hamilton and Grafton, forthcoming
) in which a cascade of visuomotor processing in parietal and frontal regions allows us to understand the goals and outcomes of other people's actions. The principle of mirroring may apply at each level of this hierarchy, providing a common code for both planning our own intentional actions and understanding the intentions of other people. |
Supplementary Material |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
Supplementary figures can be found at: http//www.cercor.oxford.journals.org/.
|
Funding |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
National Institutes of Health (NS33504); James S. McDonnell Foundation.
|
Acknowledgments |
|---|
We thank Chris Frith, Daniel Wolpert, Emily Cross, and Eugene Tunik for helpful comments on the manuscript. Conflict of Interest: None declared.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionSupplementary MaterialFundingReferences |
|---|
Ansuini C, Santello M, Massaccesi S, Castiello U. Effects of end-goal on hand shaping. J Neurophysiol (2006) 95:2456–2465.
Aziz-Zadeh L, Koski L, Zaidel E, Mazziotta J, Iacoboni M. Lateralization of the human mirror neuron system. J Neurosci (2006) 26:2964–2970.
Baker SC, Rogers RD, Owen AM, Frith CD, Dolan RJ, Frackowiak RS, Robbins TW. Neural systems engaged by planning: a PET study of the Tower of London task. Neuropsychologia (1996) 34:515–526.[CrossRef][Web of Science][Medline]
Bernstein NA. On dexerity and its development. In: Dexerity and its development—Latash ML, Turvey MT, eds. (1996) Lawerence Earlbaum Associates: Mahwah (NJ). 3–207.
Blakemore SJ, Rees G, Frith CD. How do we predict the consequences of our actions? A functional imaging study. Neuropsychologia (1998) 36:521–529.[CrossRef][Web of Science][Medline]
Brass M, Derrfuss J, Forstmann B, von Cramon DY. The role of the inferior frontal junction area in cognitive control. Trends Cogn Sci (2005) 9:314–316.[CrossRef][Web of Science][Medline]
Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci (2001) 13:400–404.[CrossRef][Web of Science][Medline]
Buccino G, Vogt S, Ritzl A, Fink GR, Zilles K, Freund HJ, Rizzolatti G. Neural circuits underlying imitation learning of hand actions: an event-related fMRI study. Neuron (2004) 42:323–334.[CrossRef][Web of Science][Medline]
Buckner RL, Goodman J, Burock M, Rotte M, Koutstaal W, Schacter D, Rosen B, Dale AM. Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron (1998) 20:285–296.[CrossRef][Web of Science][Medline]
Buxbaum LJ, Kyle KM, Menon R. On beyond mirror neurons: internal representations subserving imitation and recognition of skilled object-related actions in humans. Brain Res Cogn Brain Res (2005) 25:226–239.[CrossRef][Medline]
Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci (2002) 3:201–215.[Web of Science][Medline]
Csibra G. Forthcoming. Action mirroring and action understanding: an alternative account. In: Attention and performance, XXII—Rossetti Y, Kawato M, Haggard P.
Culham JC, Danckert SL, DeSouza JF, Gati JS, Menon RS, Goodale MA. Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas. Exp Brain Res (2003) 153:180–189.[CrossRef][Web of Science][Medline]
den Ouden HE, Frith U, Frith C, Blakemore SJ. Thinking about intentions. Neuroimage (2005) 28:787–796.[CrossRef][Web of Science][Medline]
Desimone R. Neural mechanisms for visual memory and their role in attention. Proc Natl Acad Sci USA (1996) 93:13494–13499.
di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G. Understanding motor events: a neurophysiological study. Exp Brain Res (1992) 91:176–180.[Web of Science][Medline]
Diedrichsen J, Shadmehr R. Detecting and adjusting for artifacts in fMRI time series data. Neuroimage (2005) 27:624–634.[CrossRef][Web of Science][Medline]
Dum RP, Strick PL. Motor areas in the frontal lobe of the primate. Physiol Behav (2002) 77:677–682.[CrossRef][Medline]
Ehrsson HH, Fagergren A, Jonsson T, Westling G, Johansson RS, Forssberg H. Cortical activity in precision- versus power-grip tasks: an fMRI study. J Neurophysiol (2000) 83:528–536.
Evarts EV. Relation of pyramidal tract activity to force exerted during voluntary movement. J Neurophysiol (1968) 31:14–27.
Farrer C, Franck N, Georgieff N, Frith CD, Decety J, Jeannerod M. Modulating the experience of agency: a positron emission tomography study. Neuroimage (2003) 18:324–333.[CrossRef][Web of Science][Medline]
Fogassi L, Ferrari PF, Gesierich B, Rozzi S, Chersi F, Rizzolatti G. Parietal lobe: from action organization to intention understanding. Science (2005) 308:662–667.
Frey SH, Vinton D, Norlund R, Grafton ST. Cortical topography of human anterior intraparietal cortex active during visually guided grasping. Brain Res Cogn Brain Res (2005) 23:397–405.[CrossRef][Medline]
Frith CD, Frith U. How we predict what other people are going to do. Brain Res (2006) 1079:36–46.[CrossRef][Web of Science][Medline]
Gallese V. The manifold nature of interpersonal relations: the quest for a common mechanism. Philos Trans R Soc Lond B Biol Sci (2003) 358:517–528.
Gallese V, Fadiga L, Fogassi L, Rizzolatti G. Action recognition in the premotor cortex. Brain (1996) 119:593–609.
Gallese V, Goldman A. Mirror neurons and the simulation theory of mind-reading. Trends Cogn Sci (1998) 2:493–501.[CrossRef][Web of Science]
Grafton ST, Arbib MA, Fadiga L, Rizzolatti G. Localization of grasp representations in humans by positron emission tomography. 2. Observation compared with imagination. Exp Brain Res (1996) 112:103–111.[Web of Science][Medline]
Grafton ST, Mazziotta JC, Woods RP, Phelps ME. Human functional anatomy of visually guided finger movements. Brain (1992) 115(Pt 2):565–587.
Grill-Spector K, Henson R, Martin A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn Sci (2006) 10:14–23.[CrossRef][Web of Science][Medline]
Grill-Spector K, Malach R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) (2001) 107:293–321.[CrossRef][Medline]
Haaland KY, Harrington DL, Knight RT. Neural representations of skilled movement. Brain (2000) 123(Pt 11):2306–2313.
Hamilton AF, Grafton ST. Goal representation in human anterior intraparietal sulcus. J Neurosci (2006) 26:1133–1137.
Hamilton AF, Grafton ST. Forthcoming. The motor hierarchy: from kinematics to goals and intentions. In: Attention and performance XXII—Rosetti Y, Kawato M, Haggard P.
Hartmann K, Goldenberg G, Daumuller M, Hermsdorfer J. It takes the whole brain to make a cup of coffee: the neuropsychology of naturalistic actions involving technical devices. Neuropsychologia (2005) 43:625–637.[CrossRef][Web of Science][Medline]
Hebb DO. The organization of behavior: a neuropsychological theory (1949) New York: Wiley.
Hommel B, Musseler J, Aschersleben G, Prinz W. The Theory of Event Coding (TEC): a framework for perception and action planning. Behav Brain Sci (2001) 24:849–878.[Web of Science][Medline]
Husain M, Rorden C. Non-spatially lateralized mechanisms in hemispatial neglect. Nat Rev Neurosci (2003) 4:26–36.[CrossRef][Web of Science][Medline]
Iacoboni M, Molnar-Szakacs I, Gallese V, Buccino G, Mazziotta JC, Rizzolatti G. Grasping the intentions of others with one's own mirror neuron system. PLoS Biol (2005) 3:e79.[CrossRef][Medline]
Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzolatti G. Cortical mechanisms of human imitation. Science (1999) 286:2526–2528.
Imamizu H, Kuroda T, Yoshioka T, Kawato M. Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models. J Neurosci (2004) 24:1173–1181.
Jacob P, Jeannerod M. The motor theory of social cognition: a critique. Trends Cogn Sci (2005) 9:21–25.[CrossRef][Web of Science][Medline]
James TW, Gauthier I. Repetition-induced changes in BOLD response reflect accumulation of neural activity. Hum Brain Mapp (2006) 27:37–46.[CrossRef][Web of Science][Medline]
Jeannerod M, Arbib MA, Rizzolatti G, Sakata H. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci (1995) 18:314–320.[CrossRef][Web of Science][Medline]
Johnson-Frey SH, Maloof FR, Newman-Norlund R, Farrer C, Inati S, Grafton ST. Actions or hand-object interactions? Human inferior frontal cortex and action observation. Neuron (2003) 39:1053–1058.[CrossRef][Web of Science][Medline]
Johnson-Frey SH, Newman-Norlund R, Grafton ST. A distributed left hemisphere network active during planning of everyday tool use skills. Cereb Cortex (2005) 15:681–695.
Johnson SH, Rotte M, Grafton ST, Hinrichs H, Gazzaniga MS, Heinze HJ. Selective activation of a parietofrontal circuit during implicitly imagined prehension. Neuroimage (2002) 17:1693–1704.[CrossRef][Web of Science][Medline]
Koski L, Wohlschlager A, Bekkering H, Woods RP, Dubeau MC, Mazziotta JC, Iacoboni M. Modulation of motor and premotor activity during imitation of target-directed actions. Cereb Cortex (2002) 12:847–855.
Krekelberg B, Boynton GM, van Wezel RJ. Adaptation: from single cells to BOLD signals. Trends Neurosci (2006) 29:250–256.[CrossRef][Web of Science][Medline]
Mechsner F, Kerzel D, Knoblich G, Prinz W. Perceptual basis of bimanual coordination. Nature (2001) 414:69–73.[CrossRef][Medline]
Mevorach C, Humphreys GW, Shalev L. Opposite biases in salience-based selection for the left and right posterior parietal cortex. Nat Neurosci (2006) 9:740–742.[CrossRef][Web of Science][Medline]
Naito E, Ehrsson HH. Somatic sensation of hand-object interactive movement is associated with activity in the left inferior parietal cortex. J Neurosci (2006) 26:3783–3790.
Newman SD, Carpenter PA, Varma S, Just MA. Frontal and parietal participation in problem solving in the Tower of London: fMRI and computational modeling of planning and high-level perception. Neuropsychologia (2003) 41:1668–1682.[CrossRef][Web of Science][Medline]
Noppeney U, Price CJ. An fMRI study of syntactic adaptation. J Cogn Neurosci (2004) 16:702–713.[CrossRef][Web of Science][Medline]
Ogawa K, Inui T, Sugio T. Separating brain regions involved in internally guided and visual feedback control of moving effectors: an event-related fMRI study. Neuroimage (2006) 32:1760–1770.[CrossRef][Web of Science][Medline]
Piazza M, Izard V, Pinel P, Le Bihan D, Dehaene S. Tuning curves for approximate numerosity in the human intraparietal sulcus. Neuron (2004) 44:547–555.[CrossRef][Web of Science][Medline]
Porter R, Lemon RN. Corticospinal function and voluntary movement (1993) Oxford (UK): Clarendon Press.
Price CJ, Zeki S, Ashburner J, Penny W, Friston K, Frith C, Dolan R. Human brain function (2004) London: Academic Press.
Rizzolatti G, Camarda R, Fogassi L, Gentilucci M, Luppino G, Matelli M. Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements. Exp Brain Res (1988) 71:491–507.[CrossRef][Web of Science][Medline]
Rizzolatti G, Craighero L. The mirror-neuron system. Annu Rev Neurosci (2004) 27:169–192.[CrossRef][Web of Science][Medline]
Rizzolatti G, Fadiga L. Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). Novartis Found Symp (1998) 218:81–95.[Web of Science][Medline]
Rizzolatti G, Fadiga L, Matelli M, Bettinardi V, Paulesu E, Perani D, Fazio F. Localization of grasp representations in humans by PET: 1. Observation versus execution. Exp Brain Res (1996) 111:246–252.[Web of Science][Medline]
Saxe R. Against simulation: the argument from error. Trends Cogn Sci (2005) 9:174–179.[CrossRef][Web of Science][Medline]
Sherrington CS. The integrative action of the nervous system (1906) New Haven, CT: Yale University Press.
Shmuelof L, Zohary E. Dissociation between ventral and dorsal fMRI activation during object and action recognition. Neuron (2005) 47:457–470.[CrossRef][Web of Science][Medline]
Shmuelof L, Zohary E. A mirror representation of others’ actions in the human anterior parietal cortex. J Neurosci (2006) 26:9736–9742.
Sirigu A, Duhamel JR, Cohen L, Pillon B, Dubois B, Agid Y. The mental representation of hand movements after parietal cortex damage. Science (1996) 273:1564–1568.[Abstract]
Tunik E, Frey SH, Grafton ST. Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp. Nat Neurosci (2005) 8:505–511.[Web of Science][Medline]
Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G. I know what you are doing. A neurophysiological study. Neuron (2001) 31:155–165.[CrossRef][Web of Science][Medline]
Van Wezel RJ, Britten KH. Motion adaptation in area MT. J Neurophysiol (2002) 88:3469–3476.
Wig GS, Grafton ST, Demos KE, Kelley WM. Reductions in neural activity underlie behavioral components of repetition priming. Nat Neurosci (2005).
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C. Whitehead, J. L. Marchant, D. Craik, and C. D. Frith Neural correlates of observing pretend play in which one object is represented as another Soc Cogn Affect Neurosci, June 17, 2009; (2009) nsp021v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Cattaneo and G. Rizzolatti The Mirror Neuron System Arch Neurol, May 1, 2009; 66(5): 557 - 560. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Majdandzic, H. Bekkering, H. T. van Schie, and I. Toni Movement-Specific Repetition Suppression in Ventral and Dorsal Premotor Cortex during Action Observation Cereb Cortex, March 25, 2009; (2009) bhp049v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Tunik, O.-Y. Lo, and S. V. Adamovich Transcranial Magnetic Stimulation to the Frontal Operculum and Supramarginal Gyrus Disrupts Planning of Outcome-Based Hand-Object Interactions J. Neurosci., December 31, 2008; 28(53): 14422 - 14427. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. Kroliczak, T. D. McAdam, D. J. Quinlan, and J. C. Culham The Human Dorsal Stream Adapts to Real Actions and 3D Shape Processing: A Functional Magnetic Resonance Imaging Study J Neurophysiol, November 1, 2008; 100(5): 2627 - 2639. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||