Cerebral Cortex Advance Access originally published online on February 1, 2006
Cerebral Cortex 2007 17(1):44-62; doi:10.1093/cercor/bhj123
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Large-Scale Visuomotor Integration in the Cerebral Cortex
1 Center for Complex Systems and Brain Sciences, Florida Atlantic University, Boca Raton, FL 33431, USA, 2 Department of Biomedical Engineering, University of Florida, Gainesville, FL 32611, USA, 3 Clinical Brain Disorders Branch, National Institute of Mental Health, Bethesda, MD 20892, USA, 4 Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD 20892, USA, 5 Departament de Tecnologia, Universitat Pompeu Fabra, 08003 Barcelona, Spain
Address correspondence to Steven L. Bressler, Center for Complex Systems and Brain Sciences, Florida Atlantic University, 777 Glades Road, Boca Raton, FL 33431, USA. Email: bressler{at}fau.edu.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
Efficient visuomotor behavior depends on integrated processing by the visual and motor systems of the cerebral cortex. Yet, many previous cortical neurophysiology studies have examined the visual and motor modalities in isolation, largely ignoring questions of large-scale cross-modal integration. To address this issue, we analyzed event-related local field potentials simultaneously recorded from multiple visual, motor, and executive cortical sites in monkeys performing a visuomotor pattern discrimination task. The timing and cortical location of four aspects of event-related activities were examined: stimulus-evoked activation onset, stimulus-specific processing, stimulus category–specific processing, and response-specific processing. Activations appeared earliest in striate cortex and rapidly thereafter in other visual areas. Stimulus-specific processing began early in most visual cortical areas, some at activation onset. Early onset latencies were also observed in motor, premotor, and prefrontal areas, some as early as in striate cortex, but these early-activating frontal sites did not show early stimulus-specific processing. Response-specific processing began around 150 ms poststimulus in widespread cortical areas, suggesting that perceptual decision formation and response selection arose through concurrent processes of visual, motor, and executive areas. The occurrence of stimulus-specific and stimulus category–specific differences after the onset of response-specific processing suggests that sensory and motor stages of visuomotor processing overlapped in time.
Key Words: event-related potentials • local field potentials • prefrontal cortex • timing • visual pathways
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
A major goal of cognitive neuroscience is to understand how the different sensory and motor processes of the cerebral cortex are integrated to achieve goal-directed behavior. An effective method for the study of large-scale sensorimotor integration is the analysis of timing relations of stimulus- and response-related processes in distributed cortical areas. The availability of information on such timing relations has been limited, however, by a tendency for cortical neurophysiological studies to focus on the functions of individual cortical areas in isolation. The relative timing of processes across cortical areas is thus difficult to assess from published studies, and attempts at meta-analysis have been hampered by a lack of uniformity in experimental design, as well as in recording and analysis techniques.
The present study was undertaken to explore the large-scale integration of visual and motor processes in the cortex through analysis of the timing relations of cortical activity recorded simultaneously from visual, motor, and executive areas of the same subject. Three macaque monkeys were trained to perform a visuomotor task, which involved the discrimination of patterned visual stimuli and execution of a 2-choice (go/nogo) response. Local field potentials (LFPs) were simultaneously recorded from distributed sets of cortical locations in a single cerebral hemisphere as the monkeys performed this task.
The study had 3 principal objectives. The first was to investigate the feedforward sweep (Oram and Perrett 1992
; Tovee 1994; Schroeder and others 1998; Lamme and Roelfsema 2000; Foxe and Simpson 2002), a fast, early process triggered by a visual stimulus. The second was to better understand the long recognized, but little understood, early activation of frontal areas by visual stimuli (Boussaoud and others 1993; Schmolesky and others 1998; Thut and others 2000; Saron and others 2001; Foxe and Simpson 2002). The third was to evaluate whether large-scale visuomotor function consists of discrete or overlapping sensory and motor stages (Miller and others 1995; Requin and Riehle 1995; Smulders and others 1995; Spencer and Coles 1999; Bichot and others 2001).
Analysis of the LFP data consisted of 2 main procedures. First, the onset times of average event-related local field potential (ERP) activity were determined at sites in multiple cortical areas across the hemisphere. This procedure was similar to that used in earlier investigations of visual evoked onset latencies of neuronal firing (Robinson and Rugg 1988; Raiguel and others 1989; Nowak and Bullier 1997; Lamme and Roelfsema 2000; Bair and others 2002) and current source densities (Givre and others 1994; Schroeder and others 1998) in monkeys and of visual evoked potentials in humans (Foxe and Simpson 2002; Vanni and others 2004). Our study was unique in that onset latencies were measured from visual ERPs recorded simultaneously from multiple cortical regions, as in human scalp–recorded visual evoked potential studies, but with the spatial precision allowed by direct cortical recording, as is typical in monkey studies.
In the 2nd analysis procedure, the subsequent time course of task-related activity was examined to identify the cortical locations and times that significantly distinguished the different stimulus and response types. This was possible because the monkeys learned to associate a particular visual pattern with either the go or the nogo response, and the stimulus-response contingencies were reversed either within or between recording sessions, forcing the monkeys to change the associations frequently.
Our analysis showed first that within 30 ms of the earliest detected activation in striate cortex (V1), activation also occurred in other visual cortical areas, consistent with the concept of an early visual stimulus–evoked feedforward sweep. Stimulus-specific processing began earliest in V1, within 100 ms of stimulus onset, and rapidly spread to sites in other occipital and temporal cortical areas, the majority of which developed early stimulus-specific differences no later than 110 ms after stimulus onset. Many of the motor, premotor, and prefrontal sites also displayed early activation, some as early as in striate cortex, suggesting that passage of the feedforward sweep through the full cortical visual hierarchy is not obligatory for the activation of frontal cortex by a visual stimulus. The fact that none of these frontal sites showed early stimulus-specific processing indicates that their activation served a function other than sensory discrimination. Sites in all lobes had response-specific differences, starting around 150 ms, which was 50 ms after the start of stimulus-specific processing and more than 120 ms before the average response time. This widespread involvement suggests that perceptual decision and response selection processes arise through the interaction of multiple visual, motor, and executive areas. The stimulus category–specific processing that was observed in 1 monkey occurred at prefrontal sites in a brief period near 200 ms. That this time was well after the onset of response-specific processing is consistent with the notion that the sensory stage of visuomotor processing continues even after the motor stage has begun.
Parts of these results have been published in abstract form (Ledberg and others 2003).
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
Subjects
LFP data from 3 young adult macaque (Macaca mulatta) monkeys (LU, TI, and GE) were obtained from experiments performed at the Laboratory of Neuropsychology at the National Institute of Mental Health during 1984–1988. Animal care was in accordance with institutional guidelines at the time. Data from different experimental sessions of the same monkeys have been used in previous work by our group (Bressler and others 1993; Bressler 1995, 1996; Liang and others 2002; Brovelli and others 2004). The LFP data used in this work will be made available for scientific and educational purposes upon request to the authors.
Electrode Placement
After the monkeys were trained to perform a visuomotor pattern discrimination task (see below), surgery was performed in a 2-stage procedure. All surgeries were performed under sterile conditions with the animal under general anesthesia. In the 1st stage, the electrode locations were marked with steel screws penetrating the skull but not the dura. These locations were based on the sulcal and gyral impressions on the concavity of the skull. Each monkey had up to 35 locations marked, all in the hemisphere contralateral to the hand used in the task. In the 2nd stage, bipolar electrodes were inserted into the cortex at the marked locations. Although some locations were intrasulcal, none of those locations was used in this study. Electrodes were made of Teflon-coated platinum–iridium wire, 0.125 mm in diameter. Each bipolar electrode was inserted with the aim that the less-advanced electrode tip extended approximately 0.5 mm into the dura and the more-advanced tip extended 2.5 mm into the cortex. For more details, see Bressler and Nakamura (1993).
Figure 1 shows the locations of the electrodes used in this work as marked visually during surgery. In monkey GE, the marked surgery positions were verified by postmortem visual inspection. A clear correspondence was established between the marked surgery positions of all 10 electrode locations used for GE in this study with their locations on the surface of the fixed brain, and the positions in the surgery map were determined to be accurate with respect to the landmarks on the map. Because the same surgical procedure was used for the other 2 monkeys, a comparable degree of precision was presumed for them. In any case, interpretation of the results reported in this study does not depend on a high degree of precision in electrode position verification. Electrode locations are designated by arbitrary uppercase letters and also by the regions of their placement.
|
Data Acquisition
For each monkey, LFP data acquisition occurred during a number of sessions, each comprising around 1000 trials. The monkeys performed only 1 session a day. During any particular session, 15 of the electrodes were connected to Grass P511 amplifiers. The particular set of electrodes used for recording could vary between sessions, but all sessions used in this work had fixed sets within each monkey. The data were band-pass filtered from 1 to 100 Hz (–6 dB at 1 and 100 Hz and 6 dB per octave falloff) and digitized at 200 Hz. The amplifiers reduced common signals at the 2 electrode tips of each bipolar electrode by over 10 000 times, thus tending to localize the source of the recorded LFP to the tissue between the tips. Data acquisition started after the monkey initiated the trial (see below) and continued for 900 ms. Intertrial intervals were short, on the order of a second.
Experimental Paradigm
The monkeys were trained to perform a visuomotor pattern discrimination task. Stimuli were presented on a computer screen placed 57 cm in front of the monkey. Presentation time was controlled by a computer-activated piezoelectric shutter. The stimuli were each made of 4 small squares, 2 inner and 2 outer, and their overall size (the distance between the outer squares) was 6 degrees of visual angle. Four different stimulus types were used, all created from subsets of a set of 8 squares in a way that the monkeys had to use the placement of at least 2 of the squares to correctly perform the task. This stimulus design assured that the total area, contrast, edge length, and brightness were constant across stimulus patterns. The stimuli are shown in Figure 2. The stimuli in the left column of Figure 2 are referred to as lines and those in the right as diamonds. As can be seen, the stimuli in the top row are right slanted, and those in the bottom row are left slanted. The right- and left-slanted line stimuli formed 1 stimulus category, and the right- and left-slanted diamond stimuli formed the other. The fact that the superposition of the stimuli from the line category would be identical to that of the diamond category meant that the 2 categories were balanced in terms of their low-level visual features. This aspect of the stimulus set was critical for interpretation of any category-related differences in neural activity (see below).
|
The monkey initiated a trial by pressing a lever with its hand and keeping it pressed. After a random interval (uniformly distributed between 120 and 2200 ms) after pressing the lever, the stimulus appeared for 100 ms. The monkey had 500 ms from stimulus onset to make a response. The release of the lever was the go response, and keeping it pressed the nogo response. The monkey was rewarded by a small amount of water on correct go trials. Correct nogo responses were not rewarded in the sessions used for this work. Figure 3 shows the outline of the task. Two different stimulus-response contingencies were used: in the line-go contingency, monkeys had to make a go response if a line was shown on the display and a nogo response if a diamond was shown; in the diamond-go contingency, the diamond stimulus was the go signal and the line stimulus the nogo signal. For 2 of the monkeys (TI and LU), the contingencies were changed within a session; 60 trials of one contingency were followed by 60 trials of the other contingency throughout the session. For the other monkey (GE), contingencies remained fixed within a session but changed between sessions. This difference in how the contingencies were mixed has implications for the statistical analysis (see below).
|
Data Preprocessing
Before the LFP data were submitted to statistical analysis, they were subjected to artifact correction. The temporal mean during the prestimulus interval was subtracted from each trial, and linear trends were removed. Line frequency contamination was removed using a notch filter. Trials having large variance were rejected to remove muscle and eye movement contamination. For the work presented here, only trials with correct responses (go or nogo) were used. The monkeys were highly trained, and in a typical session, the error rate was less than 20%. Data records from each trial were truncated to have 50 ms prior to stimulus onset and 300 ms after stimulus onset in order to have a fixed prestimulus interval and because of difficulties in interpreting data recorded after the behaviorally controlled part of the trial ended. Two electrode channels in LU, and 5 in GE, were excluded from analysis due to artifacts.
Determination of Onset Times of Event-Related Activity
For each channel, the time point of onset of event-related activity was estimated as the 1st point for which the ERP exceeded (in absolute terms) a baseline threshold level. The threshold was computed as three times the temporal standard deviation of the activity in a 90-ms-long baseline interval extending from 50 ms prior until 40 ms after stimulus onset.
Determination of Stimulus- and Response-Specific Activity
Statistical analysis was performed separately for each monkey because there was no direct correspondence between the electrode placements in the different monkeys. For each monkey, we used a 2-level hierarchical linear model with trial and session as levels. The number of trials per session was between 250 and 600, and the number of sessions was 5, 6, and 7 for LU, GE, and TI, respectively. A separate model (but with the same design matrix) was fitted independently for each channel at each time point using all trials of every session, with stimulus and response type as factors. This approach enabled us to investigate how the ERP from each electrode depended on stimulus and response type at every time point. Multiple comparison corrections were made by a permutation procedure very similar to the one suggested by Blair and Karniski (1993).
Differences between stimulus types and response types were investigated by evaluating the appropriate linear contrasts in the models. In cases where stimulus-related differences were found to be significant, a further test was performed to determine whether differences between stimulus categories (i.e., line vs. diamond) were significant. Category-related differences were determined as stimulus-related differences where the 2 stimuli of the line category significantly differed from those of the diamond category, but the 2 stimuli within each category did not significantly differ from each other. The next section explains the hierarchical linear model in more detail.
The Hierarchical Linear Model
In analyzing data from more than 1 session, it is important to allow for 2 kinds of variability: within- and between-session. These variability sources are encompassed in so-called variance component models (Searle and others 1992; Rao 1997; Raudenbush and Bryk 2002). When data are acquired in hierarchical fashion, hierarchical models are often the most appropriate models for statistical analysis (Raudenbush and Bryk 2002). A 2-level hierarchical model is in this sense equivalent to a variance component model with 2 random effects. Because hierarchical models are relatively uncommon in neurobiological research, we explain the model used in this work in some detail below.
Let yi (t) be a ni x 1 data vector containing the data from a particular channel (electrode) from all trials in session i at time t. The statistical analysis is done for each channel separately so there is no need to index over channels. The 1st level model then becomes
 |
In our application, the first 4 columns of X coded for stimulus type and the fifth for response type; the exact form of the design matrix depended on the details of the experiment. For example, a few lines of X could be
 |
To model the effect of different sessions, the following 2nd level model was used
 |
N(0, 
i (t)Ini) and ui N(0, D(t)), with In denoting the identity matrix of dimension n, D being an arbitrary positive definite p x p matrix, and N(·,·) denoting the normal distribution. Estimation of the parameters (b(t) and its variance) was made using the maximum likelihood principle. Because the models were unbalanced, the parameters were estimated using the expectation maximization algorithm as described in Raudenbush and Bryk (2002, Chapter 14). One of the monkeys (GE) had stimulus/response contingency reversals between sessions, which limited our ability to separate stimulus-related effects from response-related effects. For this monkey, we used a slightly different model for the 1st level, using 4 parameters to code for: 1) difference between right- and left-slanted line, 2) difference between right- and left-slanted diamond, 3) go, and 4) nogo. Therefore, for this monkey, we only report response-related differences.
The main effects of stimulus and response for each time point were tested using the following contrasts (case shown applying to monkeys LU and TI)
 |
Multiple Comparison Correction
Because a test was made for each channel and time point, a correction was necessary for multiple comparisons (number of channels x number of time points). This was done by a permutation procedure (Edgington 1980; Blair and Karniski 1993). In each permutation step, the design matrix for each session (the Xis) was multiplied by a random permutation matrix. Parameter estimation and hypothesis testing were performed as described above. For each channel, the maximum value of the temporally smoothed F-statistic was saved. Thus, the result of 1 permutation step was one value (the maximum F-statistic) for each channel. The permutation step was repeated 500 times creating a distribution of these maximum (F-) statistics. This permutation distribution gave a threshold level for the F-statistics above which the probability of obtaining one false positive was less than 0.1 per monkey. (Note that this was a stringent test of significance: assuring that the probability of obtaining only "one" false positive is <0.1 for a channel means that a significant effect is likely to occur by chance at only a single time point. The probability of finding more false positives is much smaller.) More concretely, for a particular channel to show a significant effect at a particular time point, we required the statistic (at this time point) to be larger than the 2nd largest value (for this channel) in the permutation distribution.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
Behavior
The overall behavior of the 3 monkeys was similar, the error rates within sessions ranging from 6% to 25%. False alarms (i.e., go responses to a nogo stimuli) were by far the most common type of error. The average (over session) percentage of error trials was 16.2, 12.0, and 14.3 for LU, TI, and GE, respectively. Note that only trials with correct responses were included in the data analysis. The average response times for go trials (mean and standard deviation over sessions) were 283 (5), 290 (9), and 249 (9) ms for LU, TI, and GE, respectively.
Timing of the Onset of Event-Related Activity
Results concerning the onset latency of event-related activity are presented in Figures 4–6. All times are reported with respect to stimulus onset (i.e., stimulus onset was set to time 0).
|
|
A number of common characteristics of visual stimulus–evoked cortical activation are evident from these results. First, all onset times of event-related activity, except in somatosensory cortex, occurred prior to 110 ms poststimulus and were thus considered to be the onset times of stimulus-evoked responses. Second, the striate cortex (area V1) in each monkey contained a site having the earliest evoked response for at least 1 stimulus type: a single striate site was earliest in TI and GE, whereas in LU a striate and a premotor site were both earliest (within the limits of the sampling resolution). The earliest onset times were approximately 67 ms in LU, 48 ms in TI, and 55 ms in GE. The site having the earliest average evoked response was in the striate cortex for LU and GE; in TI, one of the striate sites and the motor site were equally early. Third, most other sites had onset times within 30 ms after that of striate cortex. Prestriate, posterior parietal, motor, premotor, and Frontal Eye Field (FEF) sites tended to have earlier onsets than those in temporal and prefrontal areas. In LU, which had both posterior parietal and inferior temporal recording sites, the (anterior) inferior and superior parietal sites both had earlier onset times than the 2 inferior temporal sites, the inferior parietal onset being earlier than the superior parietal one. Also, the 2 inferior posterior parietal sites in GE both had earlier onset times than the 2 inferior temporal sites in TI. Finally, somatosensory event-related activity had the latest onsets (greater than 110 ms on average) and thus appeared to reflect secondary processes occurring consequent to activation of the visual system.
Stimulus- and Response-Related Differences
The following 3 contrasts were evaluated based on hierarchical modeling of the ERP data: stimulus-related, category-related (i.e., line vs. diamond), and response-related (i.e., go vs. nogo). "Stimulus-related" refers to any differences in the evoked activity between the stimulus types. This contrast served as an overall test for stimulus sensitivity. "Category-related", on the other hand, refers to specific differences between the line and the diamond categories that generalized over the stimulus types. Such categorical differences were behaviorally relevant because the decision to respond depended on them rather than on the individual stimuli.
Stimulus-Related Differences
Stimulus-related differences were only investigated for monkeys LU and TI because the 3rd monkey (GE) did not have within-session stimulus/response contingency reversals. An example of average stimulus-related differences is shown in Figure 7A for the prestriate (D) site in LU. The shaded regions indicate the time intervals where the stimulus-related difference is statistically significant. In this particular example, the ERP is different for all 4 stimulus types.
|
The stimulus-related results for all the sites of LU are shown in Figure 8 and for those of TI in Figure 9. They demonstrate pronounced early (starting before 110 ms) differences in the evoked responses to the 4 stimulus types at all striate and prestriate sites in both monkeys. In LU, both inferior temporal sites also had early stimulus-related differences. At all striate, prestriate, and inferior temporal sites in LU, the stimulus-related differences began almost at the onset of the ERP and persisted for at least 100 ms. At both superior temporal sites in LU, the stimulus-related differences also began almost at the onset of the ERP but persisted for less than 100 ms. In TI, the differences at striate and prestriate sites showed similar behavior to that in LU, but the inferior and superior temporal sites which showed stimulus-related differences in TI had late (starting after 110 ms) differences.
|
|
Late stimulus-related differences also appeared at some frontal lobe sites. The FEF site in TI showed large late differences and in LU showed a smaller late difference. Two of the 3 prefrontal sites in TI showed late differences. Even the motor site in LU showed a late difference but only beginning around 260 ms poststimulus. There were no significant differences earlier than 130 ms at any of the frontal lobe sites.
Stimulus Category–Related Differences
For a cortical site to be considered sensitive to differences between the line and diamond stimulus categories, the evoked responses at that site to both line stimuli had to differ significantly from those to both diamond stimuli. Some sites showed a significant difference between responses to, for example, right-slanted line and right-slanted diamond stimuli, but not between responses to left-slanted line and left-slanted diamond stimuli; consequently they were not taken to indicate a category-specific difference.
One prefrontal site (M) in TI showed a significant difference between the line and diamond categories in a brief time window around 200 ms poststimulus. A second prefrontal site (O) in TI showed an appreciably elevated category-related difference also restricted to a brief time window around 200 ms, but the magnitude of the difference for this site was just below the threshold for significance.
Response-Related Differences
Response-related differences are reported for all 3 monkeys. An example is shown in Figure 7B for the prestriate (D) site in LU. The ERPs for the 2 response types are very similar until 150 ms, at which time they separate, and after 160 ms there is a significant difference that persists until 300 ms. This go–nogo difference begins much later than the onset of significant differences due to stimulus type seen in Figure 7A. The response-related results for all the sites of LU are shown in Figure 10, of TI in Figure 11, and of GE in Figure 12.
|
|
|
All motor, premotor, prefrontal, FEF, inferior and superior parietal, and inferior temporal sites in the 3 monkeys showed a pronounced difference between go and nogo conditions, as did 1 superior temporal site (in TI). Of the 8 striate sites, 5 also showed differences between go and nogo conditions, although three of these differences were very brief, lasting only 1 or 2 time points.
In all 3 monkeys, the earliest separation between go and nogo ERPs occurred around 150 ms. The sites at which separation occurred around 150 ms were prestriate, inferior temporal, inferior parietal, motor, and FEF in LU; striate, inferior temporal, superior temporal, premotor, FEF, and prefrontal in TI; and inferior parietal, premotor, and prefrontal in GE. The somatosensory sites (in LU and GE) showed response-related differences only after 200 ms. The sites with significant late differences (after 200 ms) were striate, prestriate, inferior temporal, superior parietal, inferior parietal, somatosensory, motor, premotor, and FEF in LU; striate, inferior temporal, superior temporal, motor, premotor, FEF, and prefrontal in TI; and striate, prestriate, inferior parietal, somatosensory, premotor, and prefrontal in GE.
Time Courses of the Stimulus- and Response-Related Effects
The time periods of significant stimulus- or response-related difference are represented by the shaded regions in Figures 7–12. Significance was determined from F-statistics by the permutation procedure described in Materials and Methods. However, temporal variation in the value of the significant F-statistics is not revealed in these figures because the shading is uniform. This temporal variation in the significant F-statistics is presented for monkeys LU and TI using gray scales in Figures 13 and 14.
|
|
These figures show that the stimulus-specific effects were mainly confined to the occipital and temporal lobes, whereas the response-specific effects were more widely distributed. Most sites with stimulus-specific effects had their highest F-value occurring before 150 ms. The sites with largest early stimulus-specific effects were concentrated in occipital and posterior temporal lobes. By 175 ms, the distribution of sites with stimulus-related differences had shifted further anterior in the temporal lobe. Unlike the stimulus-related differences, no response-related differences had occurred by 100 ms and few prior to 150 ms.
Response-specific effects occurred after 150 ms in the parietal, frontal, and temporal lobes in LU and in frontal, temporal, and occipital lobes in TI (which had no parietal lobe recording sites). The sites having highest F-values for the response-specific differences were in the posterior parietal and motor cortices in LU and in the prefrontal cortex in TI. Most of these high F-values occurred after 250 ms. Some temporal lobe sites had temporally overlapping stimulus and response selectivity in the time period from 150 to 250 ms after stimulus onset. In TI, temporal overlap also occurred between stimulus and response selectivity at some prefrontal sites from 150 to 200 ms.
In general, these results show that the largest ERP differences with respect to stimulus type were in the occipital and posterior temporal cortices before 150 ms and that the largest response-related differences occurred around 100 ms later in the parietal and frontal lobes.
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionsReferences |
|---|
We analyzed ERPs recorded from distributed sets of cortical sites in 3 macaque monkeys performing a visuomotor pattern discrimination task. Unlike most previous studies of the electrophysiology of visuomotor function, we explicitly compared activity simultaneously recorded from both visual and motor areas as well as from executive cortices.
Most sites examined in all 3 monkeys had evoked responses time locked to the visual stimulus, in keeping with earlier studies showing that visually evoked potentials can be recorded in widespread cortical and subcortical structures (Creutzfeldt and Kuhnt 1973). Analysis of the relative timing of onset latencies indicated that within a very short time (30 ms) from the earliest detectable evoked activity in striate cortex, a large number of cortical sites were activated. For many sites, occipital and temporal in particular, the evoked signal depended on the stimulus used, and stimulus-specific differences were often evident from the onset of the evoked response. Many of the same sites also showed significant differences between go and nogo trials, suggesting that they were involved in both sensory and motor processing. Sites showing go/nogo differences were found in all major lobes. Below we discuss each of these findings. However, we first discuss some technical issues regarding the measured LFP signal and the applied analysis.
Precision of LFP Measurements
Origin of the LFP Signal
LFPs are thought to represent the summation of dendritic postsynaptic potentials from large numbers of neurons (Purpura 1959; Klee and others 1965; Elul 1971; Creutzfeldt and Kuhnt 1973; Mitzdorf 1985; Schroeder and others 1995). The LFP signals used in this study were recorded as electric potential differences between the 2 tips of bipolar electrodes, 1 tip located within the deep layers of the cortex (or white matter) and the other in the superficial layers or in the dura. Assuming that the 2 tips had the same impedance, this differential recording resulted in a localized signal mainly reflecting extracellular current flows in the space between the tips. Because the recordings were potential differences between deep and superficial electrode tips, LFP polarity was determined by the relative magnitudes of the deep and superficial potentials, which could be affected by the precise laminar positioning of the tips. Thus, physiological interpretation of the polarity of LFP deflections in this data set was not possible.
We observed that evoked responses from nearby bipolar pairs could be markedly different (e.g., striate (B) and striate (C) in Fig. 9) indicating that the "field of view" of the differential recordings was locally restricted. It is therefore unlikely that recordings from 2 separate sites, at some distance apart, would "see" the same source.
Although the specific neuronal generators underlying the different components of the evoked response are not known in general, the pyramidal cells are considered to be a major contributor to the cortical LFP, regardless of area (Speckmann and Elger 1982). However, in striate cortex the ERP onset to stimuli with a significant luminance increment represents the response of layer 4c stellate cells to afferent inputs from the lateral geniculate nucleus (LGN) (Kraut and others 1985; Schroeder and others 1991; Mehta and others 2000a, 2000b).
Assumptions of Constant Evoked Response
The hierarchical modeling used in this work to identify stimulus- and response-related ERP differences assumed that the evoked response at a particular site was constant across the trials comprising a session, both with respect to amplitude and latency. It was further assumed that the latency was constant between sessions (the amplitude being allowed to vary). Critically, the model was able to account for the sometimes large between-session variability of the ERP amplitude. In this model, inference was made with respect to the session level, meaning that the significant results reported here would very likely be reproduced if the experiment was repeated in the same animals. By contrast, analyses that are only sensitive to within-session variability are not as easily interpreted because their results are not generally reproducible.
On the other hand, evoked responses are also known to be variable within a session, both with respect to latency and amplitude (Zerlin and Davis 1967; Truccolo and others 2002). Our hierarchical modeling did not explicitly take this variability into consideration. The effect of ignoring this variability was most likely a loss in sensitivity, meaning that some small effects might have been missed. There is no generally accepted method for dealing with this within-session, trial-to-trial variability.
Effects of Eye Movements
The monkeys were not restrained from moving their eyes during the task. However, considering that the stimuli were symmetric and spanned a visual angle of 6 degrees, fixation on the center of the screen was important for good task performance. Moreover, the stimuli appeared for only 100 ms, which was too short to allow eye movements to be beneficial in solving the task. Because there is no reason to believe that there were any consistent differences in the propensity to move the eyes for the different stimulus and response types, it seems reasonable to assume that any eye movements that were made had the effect of increasing the between-trial variability of the measurements, but not in any systematic way. The magnitude of this variability was reduced by the preprocessing, rejecting trials having large LFP excursions. Hence, we conclude that the stimulus- and response-related differences reported here are unrelated to eye movements. Note, however, that the onset latency analysis is sensitive to eye movement, and we cannot exclude the possibility that some of the early event-related frontal responses may be eye-movement related.
Onset Latencies of the Evoked Response
Within 30 ms of the earliest detected response in any region, the large majority of recording sites displayed stimulus-evoked activity (Figs. 4–6). In this section, we discuss these findings and relate them to other investigations of onset timing.
The Evoked LFP as an Indicator of Onset Time
A large number of factors are known to influence the onset latency of stimulus-evoked LFPs. These include stimulus intensity (luminance and contrast) and specificity (Regan 1972; Creutzfeldt and Kuhnt 1973; Raiguel and others 1999), psychological factors such as attention (Wilkinson 1967; Regan 1972, Chapter 3), the "background state" of the cortex (Brandt 1997; Haig and Gordon 1998; Tsodyks and others 1999), and the level of anesthesia (Nowak and Bullier 1997). Other factors may affect the detection of onset latency, including physiological factors such as the degree of within-session latency variation and analytic factors such as the criteria for onset detection. A further consideration in the visual system is whether the neuronal population that generates the LFP is dominated by parvocellular or magnocellular input from the LGN. The dorsal visual system, which is dominated by magnocellular input, shows a consistent latency advantage over the ventral system (Schroeder and others 1998).
Given that LFPs reflect the activity of large populations of cortical neurons, and given the known correlation between changes in LFPs and spike rates (Ashford and Fuster 1985; Robinson and Rugg 1988; Coburn and others 1990; Schroeder and others 1991), LFP onset latencies can be viewed as estimates of the average onset latency of the earliest responding neuronal population in a particular area. These responses are stimulus-locked neuronal (dendritic) events that may not generate a significant change in single-neuron spike rates. It has been argued that because such events are detectable in LFPs, LFPs may be a more sensitive measure of onset latency than spike-rate–based measures (Schroeder and others 1998). It seems likely, however, that single-neuron and LFP-based estimates of onset times will be largely similar in cases of stimulus-triggered events. As with detection of onset latency in spike rates, detection in the LFP is affected by the size of the stimulus-related change in comparison with that of the background activity. Some degree of variability in onset latency across cortical sites was expected due to variation in the magnitudes of the evoked signal and the background noise. However, comparison of these values across sites showed that such variation did not significantly affect our analysis.
Variability of Visually Evoked Onset Times in Striate Cortex
In the primary visual cortex, we observed a substantial variability between onset times, both between sites and between different stimulus types at the same site. This variability can be explained as follows. Those sites whose receptive fields contained one of the 4 squares comprising the stimulus most likely had an earlier response than sites with receptive fields more distant from a square. From investigations using voltage-sensitive dyes, it is known that a point-like visual stimulus first activates a small part of striate cortex having that point in its receptive field and that the activity secondarily spreads to a much larger part of the cortex with a speed of 0.15–0.19 m/s (Grinvald and others 1994; Slovin and others 2002). This means that a striate site that is primarily activated from a stimulus square in its receptive field would show an earlier activation than one that is secondarily activated. Given that we report results from a total of 7 sites in the striate cortex, and that the stimuli covered the central 6 degrees of the visual space, the onset variability was likely due to variation in the distances of these sites from sites whose receptive fields contained one of the 4 squares comprising the stimulus.
Comparison with Other Studies of Visually Evoked Onset Times
In most studies of onset times in the visual system, detailed onset time measurements are obtained from 1, or at most 2, areas. The results of many such studies have been laboriously compiled and meticulously compared in meta-analytic reports (Nowak and Bullier 1997; Lamme and Roelfsema 2000). Yet, meta-analysis is greatly hampered by cross-study variability in the many factors known to influence onset latency. The study by Schmolesky and others (1998), which recorded visual flash–evoked response latencies from single units in several areas, was exceptional in its use of partially optimized stimulus parameters to enhance the responses of different cell types, and from its recording of responses from several different areas in the same monkeys. However, interpretation of its latency results is hampered by the facts that the neuronal responses were not recorded simultaneously and that the monkeys were paralyzed and anesthetized. The study by Schroeder and others (1998) also recorded neural responses from sites in different areas of the same monkeys and had the further advantage that the monkeys were awake and fixating their gaze. Comparison across sites was even further facilitated by their use of identical stimuli (supramaximal light flashes) for all recording sites. Their study, though, also suffered from not recording simultaneous responses from different areas. In contrast to previous studies, our work is unique in providing comparison of onset latencies across a large number of sites in visual, motor, and executive cortical areas, simultaneously recorded from the same awake, behaving monkeys.
Despite their limitations, previous studies have established that the earliest onset of evoked activity is in the LGN, followed by V1, and thereafter V2 and areas belonging to the dorsal visual system (posterior parietal, superior temporal, and frontal areas). These areas are activated only some 10 ms after V1. Activations in area V4 and inferior temporal visual areas are delayed more by approximately 15–40 ms. The finding of small latency differences from V1 to other visual areas has led to the concept of the feedforward sweep as an early process that rapidly activates a large number of visual cortical areas following visual stimulus presentation (Oram and Perrett 1992; Tovee 1994; Schroeder and others 1998; Lamme and Roelfsema 2000; Roelfsema and others 2000; Foxe and Simpson 2002).
Our results (Figs. 4–6) are generally consistent with previous studies of activation onset latencies in monkeys (Robinson and Rugg 1988; Raiguel and others 1989; Nowak and Bullier 1997; Lamme and Roelfsema 2000). In visual areas, the earliest onset times that we observed were at the striate sites, followed by prestriate, posterior parietal, and FEF sites some 10 ms after. The inferior temporal sites showed onsets that were some 20 ms after the striate sites, and generally after posterior parietal sites. Many frontal sites also had short onset times relative to the striate sites. In particular, some motor and premotor sites had onset times as early as sites in striate cortex.
Our onset times in striate cortex are somewhat later than the earliest values that have been reported in the literature (Maunsell and Gibson 1992; Schroeder and others 1998). Maunsell and Gibson (1992) recorded single-unit and multiunit responses in striate cortex to bright, high-contrast grating stimuli, whereas Schroeder and others (1998) recorded striate LFP and multiunit responses to bright, diffuse full-field stimuli. The earlier reported onset times in those studies may have been due to the greater luminance of their stimuli. Our estimates may also have been slower because we used a more conservative threshold in our onset detection technique than either of those studies.
Our results, based on LFPs that were simultaneously recorded from different cortical sites, support and extend the conclusions of previous studies that used meta-analysis to show small differences in onset latencies across different visual cortical areas (Nowak and Bullier 1997; Lamme and Roelfsema 2000). The short onset latencies that we observed in higher visual areas imply that information is rapidly transmitted to them after arriving in lower areas, supporting the concept of a fast feedforward sweep of activity from V1 up the visual hierarchy. Our observation of onset times at posterior parietal sites that were generally earlier than at inferior temporal sites is consistent with the idea that early activation of dorsal visual system areas trigger "crossing" (lateral) inputs that modulate later stimulus-specific processing in ventral system areas (Schroeder and others 1998). Our finding in LU of an earlier anterior inferior parietal onset than in superior parietal cortex is similar to that of the human magnetoencephalography study of Nishitani and others (1999).
Although less is known about responses of the frontal lobe (outside the FEF) to visual stimuli, many frontal areas are known to have neurons that respond to visual stimuli with very short (<100 ms) latencies (Boussaoud and others 1993; Lamme and Roelfsema 2000). Onset times in the primary motor cortex have been reported as early as 60 ms poststimulus (Zhang and others 1997) and in the prefrontal cortex between 40 and 300 ms, with a median onset time around 115 ms (Funahashi and others 1990). The frontal cortex of humans is also known to undergo early ERP activation by visual stimuli (Thut and others 2000; Saron and others 2001; Foxe and Simpson 2002).
Our study is the first to compare frontal (including motor) and striate activation onset times measured from simultaneous recordings made directly from both cortical regions. Although previous investigations have shown early onset responses in frontal areas due to visual stimuli (Lamme and Roelfsema 2000; Saron and others 2001), our results show that these onset times are also short in comparison with the striate ones, indicating that activation of frontal cortical areas by a visual stimulus does not require passage of the feedforward sweep through the full hierarchy of visual cortical areas. A likely pathway contributing to the early responses in frontal cortex is through the dorsal visual system (Schmolesky and others 1998).
An important observation is that the initial part of the evoked response at frontal sites did not differ according to stimulus type. This is in marked contrast to the visual sites, where a difference between stimulus types was often evident from the onset of the evoked response. If the early visual evoked response at frontal sites does not lead to stimulus discrimination, what is its function? One possibility is that an early response to visual input serves to prime the motor system to prepare for the response-specific processing that will take place after the stimulus category has been determined (Kwan and others 1985). It is well known that the substantial parietal lobe input received by the cortical motor areas provides visual information for use in action preparation (Rizzolatti and Luppino 2001). In the present experiments, the motor system may well have been conditioned to receive an early priming signal from the visual system by the extensive prior task training undergone by the monkeys. A second possibility is that rapid activation of prefrontal cortical areas triggers them to exert top–down facilitation of visual stimulus discrimination in visual cortical areas (Fuster 1997; Schroeder and others 1998; Corbetta and Shulman 2002; Bar 2003).
Stimulus-Related Differences
In order to correctly perform the task, the monkeys had to distinguish the 4 visual stimulus patterns. These patterns were designed so that correct discrimination required the use of at least 2 of the 4 squares in the pattern. Thus, correct task performance could only be achieved by discrimination of the patterns, and not of any single square. It was expected that visual stimulus pattern discrimination would involve processing in the ventral visual pathway, including striate, prestriate, and inferior temporal cortical areas (Gross 1973; Ungerleider and Mishkin 1982; Tanaka 1996).
Differences related to stimulus categorization were also expected because the task involved discrimination between the line and diamond stimulus categories. From recent evidence (Freedman and others 2001, 2002, 2003; Ashby and Spiering 2004) suggesting that the dorsolateral prefrontal cortex, and not the inferior temporal cortex, is specifically involved in visual stimulus categorization, we anticipated that category-specific processing would be found specifically at prefrontal sites.
Differences in the Ventral Visual Pathway
Significant stimulus-related ERP differences were found at all sites located in striate and prestriate areas and in 3 out of 4 inferior temporal sites (Figs. 8–9 and 13–14). This result is in line with previous investigations of evoked responses to visual stimuli (Lieb and Karmel 1974). Because the cortex on the convexity of the striate cortex is likely to represent the central 10 degrees of the visual field (Van Essen 1985), the stimuli, whose overall size (in visual angle) was around 6 by 6 degrees, should have been represented within the region of the sampled striate sites (assuming that the monkeys were fixating the center of the stimuli). Thus, a main determinant of the evoked response in striate cortex was the position of the recording site relative to the visual field representation. Responses were presumably larger when a site was in the striate representation of one of the squares making up the stimulus pattern, but sites not directly in the stimulus representation could also have a prominent evoked response due to the lateral spread of depolarization (Grinvald and others 1994; Slovin and others 2002).
Although the exact identities of the cortical areas sampled by the prestriate electrodes were not determined, it is likely that the 2 prestriate sites in TI were located in area V4 and that the one in LU was in area TEO (Boussaoud and others 1991) of the ventral posterior inferior temporal (PITv) cortex (also designated PITv by Felleman and Van Essen 1991). Part of the measured difference between stimulus types in V4 may have been due to retinotopic differences as in the striate cortex. The larger receptive fields and crude retinotopy of TEO makes that explanation less likely for the responses recorded in that area. Because many TEO neurons have complex response properties, stimulus-specific LFP signals in TEO could reflect a true representation of the patterns.
Three of the 4 sites labeled "inferior temporal" showed a stimulus-specific response. These sites were in anterior inferior temporal cortex (area TE), an area not thought to be retinotopically organized (Desimone and Gross 1979). The neurons of this region have very large receptive fields and complex response properties and are organized into columns with cells having similar properties (Tanaka 1996). Therefore, it is likely that the observed stimulus dependencies were due to differences in stimulus representation, each representation having a unique spatial distribution in this area.
Differences in Superior Temporal Cortex
Three of the 4 superior temporal sites also showed ERP differences between stimulus types. The exposed cortex of the superior temporal gyrus was traditionally believed to be part of the auditory system (Kaas and Hackett 2000; Poremba and others 2003), but cells in this region have also been shown to be activated by visual stimuli (Baylis and others 1987; Schroeder and Foxe 2002; Schroeder and others 2003). Part of the superior temporal gyrus receives input from cortex in the superior temporal sulcus (Hackett and others 1998) and sends projections to (among other areas) the FEF (Hackett and others 1999). Given the known visual response properties of neurons both in the superior temporal sulcus (Bruce and others 1981; Baylis and others 1987) and the FEF (Mohler and others 1973; Tehovnik and others 2000), the visual responsiveness of the superior temporal region observed in this study is not surprising. The most posterior located site in LU (superior temporal [H]) was located close to the visual areas in the posterior part of the superior temporal sulcus (i.e., middle temporal area/medial superior temporal area) (Desimone and Ungerleider 1986). Prior work (Leinonen and others 1980; Baylis and others 1987; Schroeder and others 2001; Schroeder and Foxe 2002) has shown that parts of the superior temporal cortex comprise a zone of multimodal convergence. Our results clearly indicate that many neurons in this region are not only activated by visual stimuli, but they also selectively discriminate stimulus types. To our knowledge, this selectivity has not previously been reported.
In LU, the evoked responses from superior temporal sites (G and H) behaved like those from visual areas in that they showed stimulus-related differences beginning early after the stimulus. In TI, on the other hand, stimulus-related differences at the superior temporal (H) site developed later. These differences between the monkeys were most likely due to differences in functionality between the posterior and middle superior temporal regions recorded from in LU and the anterior region recorded from in TI.
Differences in Frontal Cortex
The FEF site in TI showed stimulus-related ERP differences occurring late relative to the differences recorded at occipital and temporal sites. The main difference at the FEF site was between the right line and the other stimulus types, and it thus did not discriminate between the line and diamond stimulus categories. The FEF is known to contain visually responsive neurons with large receptive fields (Mohler and others 1973; Tehovnik and others 2000), and single neurons in this region have been shown to respond selectively to colors and patterns (Watanabe 1986). The motor (L) site in LU also showed late stimulus, but not categorical, selectivity.
By contrast, site M in TI, located in the dorsolateral prefrontal cortex, did show category-specific differences in a narrow window near 200 ms. An earlier window of stimulus-related difference near 140 ms at this site did not discriminate between categories. Prefrontal site O in TI also showed a narrow window of category-specific difference near 200 ms, although the difference was not large enough to reach statistical significance. (It did reach significance at P < 0.05, however, when the test was uncorrected for the number o