Cerebral Cortex, Vol. 10, No. 4, 343-358,
April 2000
© 2000 Oxford University Press
Intermodal Selective Attention in Monkeys. I: Distribution and Timing of Effects across Visual Areas
1 Department of Neuroscience and , 2 Department of Neurology, Albert Einstein College of Medicine, Bronx, NY, USA, , 3 Institute for Psychology of the Hungarian Academy of Sciences, Budapest, Hungary, and , 4 Program in Cognitive Neuroscience and Schizophrenia, Nathan Kline Institute for Psychiatric Research, Orangeburg, NY, USA
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionNotesReferences |
|---|
This study quantified the magnitude and timing of selective attention effects across areas of the macaque visual system, including the lateral geniculate nucleus (LGN), lower cortical areas V1 and V2, and multiple higher visual areas in the dorsal and ventral processing streams. We used one stimulus configuration and behavioral paradigm, with simultaneous recordings from different areas to allow direct comparison of the distribution and timing of attention effects across the system. Streams of interdigitated auditory and visual stimuli were presented at a high rate with an irregular interstimulus interval (mean of 4/s). Attention to visual stimuli was manipulated by requiring subjects to make discriminative behavioral responses to stimuli in one sensory modality, ignoring all stimuli in the other. The attended modality was alternated across trial blocks, and difficulty of discrimination was equated across modalities. Stimulus presentation was gated, so that no stimuli were presented unless the subject gazed at the center of the visual stimulus display. Visual stimuli were diffuse light flashes differing in intensity or color and subtending 12° centered at the point of gaze. Laminar event-related potential (ERP) and current source density (CSD) response profiles were sampled during multiple paired penetrations in multiple visual areas with linear array multicontact electrodes. Attention effects were assessed by comparing responses to specific visual stimuli when attended versus when visual stimuli were looked at the same way, but ignored. Effects were quantified by computing a modulation index (MI), a ratio of the differential CSD response produced by attention to the sum responses to attended and ignored visual stimuli. The average MI increased up levels of the lower visual pathways from none in the LGN to 0.0278 in V1 to 0.101 in V2 to 0.170 in V4. Above the V2 level, attention effects were larger in ventral stream areas (MI = 0.152) than in dorsal stream areas (MI = 0.052). Although onset latencies were shortest in dorsal stream areas, attentional modulation of the early response was small relative to the stimulus-evoked response. Higher ventral stream areas showed substantial attention effects at the earliest poststimulus time points, followed by the lower visual areas V2 and V1. In all areas, attentional modulation lagged the onset of the stimulus-evoked response, and attention effects grew over the time course of the neuronal response. The most powerful, consistent, and earliest attention effects were those found to occur in area V4, during the 100–300 ms poststimulus interval. Smaller effects occurred in V2 over the same interval, and the bulk of attention effects in V1 were later. In the accompanying paper, we describe the physiology of attention effects in V1, V2 and V4.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionNotesReferences |
|---|
Sensory information processing is construed as a hierarchical process, whereby successively more information is extracted from the input signal via transformations that occur at each stage of processing (Van Essen et al., 1992
; Maunsell, 1995; Ungerleider, 1995). Selective attention plays an integral role in sensory information processing, enhancing neuronal responses to important or task-relevant stimuli at the expense of the neuronal responses to irrelevant stimuli. This controlled processing is essential to perception because it adapts processing to the goals of the viewer (Harter and Aine, 1984; Hillyard, 1985; Desimone and Duncan, 1995; Maunsell, 1995).
Electrophysiological investigation of the brain mechanisms of attention began with event-related potential (ERP) studies in animals (Hernandez-Peon et al., 1956; Horn, 1965) and humans (Spong et al., 1965; Groves and Eason, 1969; Harter and Salmon, 1972; Hillyard et al., 1973; Naatanen, 1975; Simson et al., 1977). These were followed by single unit recordings in monkeys (Bushnell et al., 1981; Mountcastle et al., 1981; Moran and Desimone, 1985; Richmond and Sato, 1987; Motter, 1988), with the field literally exploding toward the late, 1990s (Motter, 1993; Desimone and Duncan, 1995; Maunsell, 1995; Connor et al., 1996; Treue and Maunsell, 1996; Connor et al., 1997; Luck et al., 1997; Roelfsema, 1997; Vidyasagar, 1998; Ito, 1999; McAdams, 1999; Reynolds et al., 1999; Treue, 1999). The parallel expansion in ERP studies and those incorporating magnetoencephalographic (MEG), metabolic and hemodynamic measures (LaBerge and Buchsbaum, 1990; Corbetta et al., 1991, 1993; Naatanen et al., 1994; Mangun et al., 1993, 1997; Heinze et al., 1994; Aine et al., 1995; Clark, 1996; Worden et al., 1996; Schulman et al., 1997; Woldorf et al., 1997; Culham et al., 1998; Tootell et al., 1998; Brefczynski and DeYoe, 1999; Martinez et al., 1999; Somers et al., 1999) has increased the methodological scope of human attention research; however, the obvious step of directly integrating the findings of human and monkey studies has not been made. One reason is that the measurement methods of monkey and human studies yield data that are not directly comparable. Even when precisely localized to a specific brain region, ERP and MEG components are difficult to relate to local action potential patterns, because such components are generated primarily by synaptically activated transmembrane currents, which precede and are variably related to action potential rates. The relationship between action potentials and the metabolic/hemodynamic measurements is even more difficult to ascertain.
An additional problem for integration of monkey and human research is that the experimental paradigms are rarely directly comparable. Human selective attention has been studied in the contexts of both visual spatial attention (Van Voorhis and Hillyard, 1977; Harter et al., 1982), visual feature attention (Harter et al., 1982; Harter and Aine, 1984; Hillyard, 1985) as well as in a number of auditory (Picton and Hillyard, 1974; Naatanen and Picton, 1986; Woods and Clayworth, 1987; Hansen and Woldorff, 1991) and somatosensory paradigms (Desmedt and Tomberg, 1989). A variety of behavioral paradigms have been used to study ‘state-dependent’ and attention-related discriminative processing in primate visual cortex (Bushnell et al., 1981; Mountcastle et al., 1981; Haenny and Schiller, 1988; Spitzer and Richmond, 1991; Ferrera et al., 1994). With two notable exceptions, however (Roelfsema et al., 1997; Treue, 1999), the specific study of selective attention in monkeys has focused exclusively on the effects of spatial attention (Moran and Desimone, 1985; Motter, 1993; Maunsell, 1995; Connor et al., 1996; Treue and Maunsell, 1996; Luck et al., 1997; Vidyasagar, 1998; Ito, 1999; Reynolds et al., 1999; Seideman, 1999). Even in studying spatial attention, monkey experimental paradigms diverged significantly from those of the preceding human studies, in that they focused on attention shifts over small regions of visual space (Moran and Desimone, 1985; Motter, 1993; Connor et al., 1996; Treue and Maunsell, 1996) rather than across visual hemifields (Harter et al., 1982; Mangun and Hillyard, 1988). In only one case (Luck et al., 1997) was there an explicit attempt to address directly the findings of earlier studies in humans.
For both of these empirical and conceptual reasons, the promise of understanding the physiology of human selective attention through invasive experiments in monkeys has not been realized. In this and the accompanying study (Mehta and Schroeder, 2000), we address methodological integration of monkey and human research directly by analyzing attention effects on processes underlying ERP generation, in relation to attention effects on local action potential patterns. Laminar profiles of ERPs, estimates of the underlying synaptic input patterns and indices of concomitant action potential patterns were sampled during acute penetrations into regions of interest with linear array multielectrodes. Laminar patterns of synaptic input and action potentials were indexed by current source density (CSD) analysis and multiunit activity recording respectively.
We adopted an intermodal (visual/auditory) selection paradigm used by Alho et al. (Hackley et al., 1990; Alho et al., 1992; Woods et al., 1992) to study attention in humans. Monkeys were exposed to concurrent streams of auditory and visual stimuli, interdigitated at random interstimulus intervals, with average combined rate of 4/s. The task required performance of a difficult discrimination within one modality, the other modality being, at that time, irrelevant to the task. Modality was alternated across trial blocks. A constant eye position requirement was in effect across all conditions, so that responses to specific visual stimuli critical to the task (attended) could be compared to the responses to the same stimuli, when looked at the same way, but ignored, during the performance of an equally difficult auditory discrimination. Modifications in the paradigm were made to increase the control over eye position and make it otherwise appropriate for monkeys.
This combined application of this paradigm and recording method was advantageous for several reasons. First, in determining how attention fits into the hierarchical model of visual processing, it is important to know how much influence attention exerts at each stage of processing. This was the first goal of the present study. In monkeys, it is widely observed that attention has a large influence on neuronal responses to visual input at the level of V4 (Moran and Desimone, 1985; Motter, 1993; Desimone and Duncan, 1995; Maunsell, 1995; Connor et al., 1996; Connor et al., 1997; Luck et al., 1997) and MT (Treue and Maunsell, 1996; Treue, 1999), but reports of attentional modulation at lower cortical levels in area V2 (Motter, 1988; Luck et al., 1997), and especially in area V1 (Motter, 1993), until recently, were rare. Recently, reports from several laboratories (Roelfsema et al., 1997; Vidyasagar, 1998; Ito and Gilbert, 1999; McAdams and Maunsell, 1999) have made it clear that, with appropriate stimulus configurations and task demands, attention does indeed modulate processing in V1 (Schroeder, 1995). Similarly, while most human studies have been supportive of attentional modulation in extrastriate cortex (Corbetta et al., 1993; Mangun et al., 1993; Heinze et al., 1994; Clark et al., 1996; Woldorf et al., 1997; Culham et al., 1998), several recent studies in humans also show attentional modulation in V1 (Worden et al., 1996; Schulman et al., 1997; Tootell et al., 1998; Brefczynski and DeYoe, 1999; Martinez et al., 1999; Somers et al., 1999). Numerous predictions have been made concerning attention effects very early in processing in the LGN (Crick, 1985; Sherman and Koch, 1986; Schroeder, 1995), but these have not been tested. In order to define attention effects at different levels of the visual hierarchy, we wanted to use a task which could elicit attention effects as early in the system as possible. The global switching of attention between modalities was designed with this in mind (Alho et al., 1992). To permit direct comparison of effects across brain regions, we used the same behavioral paradigm and stimulus configuration across recording sessions, and in most of the experiments we based cross-regional comparisons upon concurrent dual multielectrode electrode recordings. Using the same stimulus configuration enabled us to equate task conditions and difficulty across recording sites and thereby make direct comparisons. We chose a diffuse light as a stimulus because its evoked response is insensitive to variation in the subject's focus. While the subject's eye position is rather easy to control, focus is not, and this presents a potential confound in comparing visual responses from visual-versus auditory-attend conditions. Diffuse light stimulation avoids this problem, and while not considered an optimal stimulus for visual neurons, does elicit robust neuronal responses throughout the system, as indexed by CSD analysis (Mitzdorf, 1986; Schroeder et al., 1991, 1998; Givre et al., 1994, 1995). We also had to establish a methodology for detecting and quantifying the distribution of attention effects within each visual area. The CSD method provided the basis for detection and quantification of attentional modulation and also provided sensitivity to modulation of activity, which may affect postsynaptic potentials, without a clear manifestation in action potentials (Schroeder, 1995; Schroeder et al., 1995, 1998).
Definition of the temporal pattern of attentional modulation, relative to that of visually evoked activity, can help to elucidate the relationship of attention to the visual processing hierarchy. The time domain has been an important focus in the development of multimodal (ERP, MEG and functional magnetic resonance) neuroimaging in human studies (Heinze et al., 1994), and it has proven to be critical in defining the dynamics of attention (Martinez et al., 1999). However, no previous study in monkeys has performed an explicit analysis of the time course of attentional modulation within a region, such as V4, much less across the different regions comprising the visual hierarchy. Furthermore, while modulation at response onset has been reported for spatial discrimination in V4 (Motter, 1993; Luck et al., 1997), it is not clear to what extent this generalizes over different structures and experimental paradigms. There is evidence, for example, that visual spatial and visual feature selective attention paradigms induce effects in the surface ERP distribution that differ significantly in their onset latencies (Harter et al., 1982). It is also of interest to determine the extent to which attentional modulation builds over time in a manner dissociated from temporal pattern of visually driven activity. Build-up of attention effects over poststimulus time has been observed in studies from several other laboratories, despite differences in recording methods and behavioral paradigms (Maunsell, 1995; Luck et al., 1997; Vidyasagar, 1998; Seideman and Newsome, 1999). The second goal of the present study was to analyze the spatiotemporal pattern of intermodal attention effects across lower visual pathways and across higher dorsal and ventral stream areas of the macaque visual system. That is, we aimed to examine the latency and temporal pattern of modulation within each area, as well as the latency pattern across visual areas. This study allowed us to focus further analyses on areas with the largest, most reliable effects. In the following paper, we then examined the local physiology of attentional modulation, and the mechanisms promoting and coordinating attentional modulation within this subset of visual areas.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionNotesReferences |
|---|
Subjects and Surgical Preparation
All animal care and use procedures were approved by the Institutional Animal Care and Use Committee, and were in accordance with NIH guidelines as outlined in NIH publication no. 86–23 (rev. 1087). Three male macaque (Maccaca fascicularis) monkeys (6.0–9.0 kg) were used as subjects for this study. One subject was used to develop the behavioral paradigm, and electrophysiological data were obtained from the other two. Preparation of subjects for chronic awake recording was performed using aseptic techniques, under general anesthesia, as described previously (Schroeder et al., 1991). To provide access to the brain, groups of 80–100 stainless-steel 18 gauge tubes were glued together in a parallel matrix, positioned normal to the surface of the targeted structures, and sealed with surgical grade silastic. Implantation was guided by stereotaxic transformation of magnetic resonance imaging (MRI) data, which delineated the cortical gyral pattern and optimal penetration angle for the LGN (Schroeder, 1995; Schroeder et al., 1998). Guide tube matrices were positioned over the operculum of striate cortex, the prelunate gyrus, over the projection of a line normal to the lamination pattern at the crown and medial wing of the LGN, and over the projection of a line normal to the superior temporal sulcus.
Stimuli
Visual stimuli were 10 µs flashes generated by one of two Grass PS22 Photo Stimulators projected onto a diffuser, subtending 11.8° of the visual field, at a viewing distance of 43 cm. Filters were inserted in front of each lamp to change the intensity and spectral content of the stimuli. The peak transmittance/half-amplitude bandwidth of the standard stimulus was ~660 nm/141 nm (red), and neutral density filters (Wratten ND 0.1–0.5) could be placed in addition to vary the luminance and peak transmittance generated by either of the lamps. This type of stimulus has been shown to elicit robust CSD responses in a variety of visual areas. (Schroeder et al., 1989, 1991, 1998; Givre et al., 1994, 1995; Schroeder, 1995). In most trial blocks, visual stimuli were of two different intensities, a standard 200 cd/m2 stimulus presented 86% of the time and a deviant dimmer intensity stimulus presented 14% of the time, pseudorandomly. In some trial blocks, the deviant stimulus differed in wavelength. The wavelength of the deviant stimulus was kept the same during the brightness discrimination task, while the luminance decreased from 10% to 50%, depending on task difficulty. During color discrimination tasks, deviant stimuli were either white, blue, green, orange or pink. In an analogous fashion, auditory stimuli consisted of 50 ms, 70 dB, binaural, windowed, pure tones of two different frequencies, a standard low-frequency (2 kHz) tone presented 86% of the time and a deviant higher-frequency tone (2.1–4.0 kHz) presented 14% of the time, through headphones. Only neural responses to visual stimuli will be described in this and the accompanying report (Mehta and Schroeder, 2000).
Behavioral Paradigm
Animals sat in a primate chair in a dark room with their heads fixed in position. As shown in Figure 1, transient auditory and visual stimuli were presented with a random 200–400 ms interstimulus (onset) interval (ISI). Stimuli in each modality were termed ‘standards’ (86% probability) and ‘deviants’ (14% probability). Deviant stimuli differed from standard stimuli in frequency and luminance/peak transmittance for auditory and visual stimuli respectively. Selective attention was controlled by requiring subjects to respond to deviant stimuli in one sensory modality only. If the subject responded by switch release between 120 and 650 ms after presentation of the deviant stimulus in the appropriate modality, a juice reward (0.2 ml) was delivered. Reaction times were in the range of 250–450 ms for visual discriminations and 150–350 ms for auditory discriminations. If the subject released the lever at any time except in response to the target stimulus, a 3 s time-out penalty ensued, with no stimulation and, consequently, no opportunity for reward. After each 3–5 min trial block, we cued subjects to discriminate stimuli in the other sensory modality, so that the visual modality was alternately attended and ignored in successive trial blocks. Cueing was accomplished by presenting stimuli only in one sensory modality at the beginning of a trial block until the animal responded selectively to the deviant stimuli, at which point stimulus presentation began in the other (irrelevant) sensory modality. Each trial block contained 40–60 targets. Animals performed for 2–4 h, yielding 10–20 trial blocks before indicating satiety either by not viewing the stimulus or not pressing the lever. Typically, this occurred suddenly: within a few seconds, animals would go from criterion performance to no performance. Therefore, under most circumstances, there was no systematic change in behavior, independent of task difficulty, as a function of the length of the session.
|
Task Difficulty
Task difficulty was estimated by measuring hits, misses, false alarms and correct rejections. By performing behavioral testing for visual and auditory discriminations, discriminative difficulty could be equated across sensory modality. Subjects were trained on the task 2 months prior to surgical implantation and 2 months after a 2 week recovery period after surgery. Over the course of the 6–12 month recording period, subjects' performance on tasks usually improved. Behavior was maintained near 97% correct, by adjusting the physical difference between standard and deviant stimuli. On a given recording session, this was varied in small steps to test for the effects of difficulty and task performance. If behavior dropped below 92% correct on any trial block, electrophysiological data from that trial block were not analyzed.
Control for Arousal Effects
A critical issue in selective attention studies is the dissociation of the effects of attention from those of arousal. In our paradigm, there are three explicit controls for arousal. First, as mentioned above, task difficulty was equated across the visual and auditory discriminations. Second, although visual and auditory stimuli were interdigitated rather than randomized (to maintain as much temporal separation of the visual stimili as possible), stimulus presentation was extremely rapid (4/s) and the ISI was randomized. At this short, random ISI, human subjects run in this paradigm (Foxe et al., 1994) did not appear to oscillate between vigilance (and arousal) levels as they can at low fixed rates of alternation between modalities of stimulation (Spong et al., 1965; Naatanen, 1975). Significantly, when subjects try to discern a change in the stimulus alternation, they cannot at the same time detect the oddball within the cued (relevant) modality. Because of the alternation, the stimulus sequence itself does not completely rule out the possibility that subjects could oscillate arousal levels between conditions (Naatanen, 1975); however, the high stimulation rate, combined with the randomized ISI, makes it extremely unlikely that arousal levels could oscillate in synchrony with the stimuli. Third, data collection entailed dual multielectrode recordings, and thus we were able to pair the observations in cortical regions showing moderate or large attention effects with simultaneous recordings in lower cortical (V1) or subcortical regions LGN showing little or no attentional modulation. The brainstem/reticular core afferents that mediate the impact of arousal on processing project densely to subcortical as well as cortical visual processing stages (Sherman and Koch, 1986; Schroeder, 1995), which predicts that effects should be seen in LGN; this was not found (Figs 6, 9). The idea that arousal effects compound over successive stages predicts an ascending increase in the amplitude of effects, as well as a ‘feedforward’ (lamina 4 initiated) laminar profile of modulation and a systematic increase in the latency of modulation across successive stages (Figs 9, 10). Our dual multielectrode recordings did reveal an ascending gradient of effects; however, they clearly exhibited a multilaminar ‘associative/feedback’ laminar profile (Mehta and Schroeder, 2000), and the latency pattern was opposite to that predicted by the arousal hypothesis. The latter two patterns of effect across structures, in particular, indicate the operation of a top-down process and argue most strongly against an interpretation of effects in terms of arousal.
|
|
|
Control for Eye Position Effects
In investigating intermodal attention effects on visual processing it is critical for visual stimulation to be constant across visual and auditory attention conditions. The most important issue in this regard is that of eye position. In addition to a rigorous eye position-based data analysis (see below), three components of the behavioral paradigm helped to maximize control of sensory stimulation. First, to minimize eye position biases, efforts were made to produce the same spatial locus of attention in both sensory modalities, by presenting visual stimuli centrally and auditory stimuli binaurally. Second, presentation of all stimuli (auditory and visual) was gated by eye position, and only trials where subjects held their gaze within the eye position window for at least 1 s were analyzed, so that saccade- or blink-related activity was minimized. Finally, the luminance and color discrimination of large, brief, diffuse light stimuli relaxed requirements for focus as well as fixation.
In summary, we made a number of modifications to previously used versions of the intermodal paradigm in animal subjects (Horn, 1965). This was critical for controlling sensory stimulation and arousal, and for knowing whether animals were, in fact, attending and ignoring the appropriate stimuli. (i) To enable control over sensory stimulation, all stimuli (auditory and visual) were gated by eye position, and we used visual discriminations that did not require focus. (ii) To control arousal, equivalent discriminations with similar difficulty were presented in both sensory modalities, and animals controlled their own stimulus environment by pressing a lever and fixating within the eye position window in order to receive any stimulus. Simultaneous recordings across levels of the system also controlled for arousal effects. (iii) To ensure that subjects were ignoring the irrelevant sensory modality, deviants were presented with the same probability in both sensory modalities. If subjects began to false alarm to deviants in the ‘irrelevant’ modality (catch trials), the trial block was aborted, and electrophysiological data from that trial block were discarded.
Signal Recording
For each experimental session, one or two linear array multielectrodes were lowered into the brain and positioned to straddle the laminae of the target regions. Laminar profiles of visual event related potentials (ERPs) and concomitant multiunit activity were recorded (Schroeder et al., 1998). Figure 2 illustrates the recording preparation using an activity profile from V1. To permit direct comparisons across visual areas, we performed concurrent dual-electrode recordings with separate electrode arrays simultaneously sampling laminar profiles in two different brain regions in a subset of recording sessions. Each multicontact electrode contained an array of 14 equally spaced contacts (150 or 200 µm intercontact spacing) and one contact 0.5–1 mm below the lowest channel of the main array. The impedance at each contact was 0.1–0.3 M
. Electrophysiological signals were acquired from the electrode arrays, with each intracortical electrode referenced to an epidural electrode at the frontal midline. Signals from each electrode contact were coupled via unity gain preamplifiers to Grass P5 and P12C amplifiers, at 2000, 5000 or 10 000 gains, and with bandpasses of 1–3000 or 3–3000 Hz. All signals were stored as continuous records onto a PC-based data acquisition system (Neuroscan, El Paso, TX).
|
Visual areas were defined by anatomical location and, in certain cases, by physiological characteristics (Schroeder et al., 1991
, 1998; Givre et al., 1994, 1995; Schroeder, 1995). All identification of areas was subject to histological verification. To optimize the use of one-dimensional CSD analysis in LGN, data were taken from recordings in the crown and the adjacent portion of the medial wing of LGN (Schroeder et al., 1992). Penetrations were made orthogonal to the lamination patterns and P and M laminae were sampled concurrently. P and M laminae in LGN were distinguished as in previous studies (Schroeder et al., 1989, 1990, 1992). V1 recordings were obtained from the lateral surface of the striate operculum. V2 recordings were made primarily in the first belt of cortical tissue underlying the operculum, immediately posterior to the lunate sulcus, by penetrating through V1. V4 recordings were in the prelunate gyrus, or the lateral lip of the posterior superior temporal sulcus (STS). Recordings within STS were grouped into three divisions: (i) STSd (dorsal), primarily areas STP and MST; (ii) STSpv (posterior to the lateral sulcus, ventral bank of STS), including MT and immediate surrounds; and (iii) IT (ventral bank of STS, underlying the lateral sulcus), primarily inferotemporal cortex. Recordings from both banks of the intraparietal (IP) sulcus were grouped together in area IP. Detailed exposition of our procedures for combined application of CSD and action potential profile analysis in areas throughout the cortical and subcortical visual pathways in awake monkeys, can be found elsewhere (Schroeder et al., 1998). After the electrodes were positioned, the eye position monitor (Stoetling, 1° resolution) was calibrated by requiring the subject to perform an LED dimming task, with LEDs located at different positions around the stimulus display. Voltage values corresponding to horizontal and vertical eye positions were entered into the software controlling stimulus presentation (DataWave), so that stimulus presentation could be gated by eye position. While subjects performed the task, we kept a continuous digital record of simultaneous intracranial data, surface data, stimulus events, eye position, and lever responses, so that off-line analysis could be performed to study the relationship among any of these events.
CSD Analysis
The laminar profile of transmembrane current elicited by visual stimuli was assessed by a one-dimensional CSD analysis (Nicholson and Freeman, 1975) of the field potential profile. A three-point, nearest-neighbor approximation for the second derivative was applied using adjacent electrode contacts as the differentiation grid (Schroeder et al., 1991). In keeping with the requirements of one-dimensional CSD analysis, penetrations were made orthogonal to the local lamination pattern, using procedures developed from earlier studies in cortical areas (Schroeder et al., 1991, 1998; Givre et al., 1994, 1995; Schroeder, 1995;) and in the LGN (Schroeder et al., 1989, 1990, 1992, 1998).
Off-line Data Analysis
To obtain quantified attentional modulation out of raw EEG signals at a given intracortical site, off-line analysis was performed in five steps for each recording by: (i) division of EEG signals into ‘single sweep’ responses; (ii) selection of sweeps for analysis; (iii) determination of an eye position range, the ‘off-line’ (post-recording) eye position window; (iv) statistical testing for modulation of neural responses by attention; and (v) quantification of the modulation. Each of these are explained in sequence below.
Generation of Single Sweeps
First, EEG signals were broken up into single sweeps subsuming the interval from 100 ms prestimulus to 500 ms poststimulus. Each sweep represented the electrophysiological response to one visual stimulus in the stimulus stream, a single trial. Each sweep was also tagged by stimulus type (standard/deviant and relevant/irrelevant to discrimination) and x,y eye position for that sweep. Typically, 1000–4000 sweeps were generated in one recording session.
Selection of Sweeps
Analysis was restricted to a subset of all sweeps generated during an experimental session. Sorting procedures were implemented in order to reduce the effects of EMG artifacts, saccades, blinks, eye position biases and dependencies of processing related to the stimulus sequence within a trial block. First, only responses elicited by the standard visual stimuli were selected. This enabled comparisons of larger numbers of observations (frequently presented stimuli), and it also reduced EMG artifacts, since no motor responses to the stimuli were required, whether or not the stimuli were attended. Since the standard visual stimulus was constant across trial blocks, restricting data analysis to standards enabled comparisons across trial blocks. Second, during the experiment, stimulus presentation was constrained such that deviant stimuli were never presented after a target, blink, saccade or false alarm. This constraint on deviant presentation was imposed because the preceding stimulus of a sequence seemed to serve as a comparison stimulus. For example, difficult-to-detect deviant stimuli required a preceding comparison stimulus in order to be detected as such. However, due to this constraint of target presentation, it is possible that the first stimulus of a sequence of stimuli was processed differently from the others, and thus the first sweep in a stimulus sequence was excluded from data analysis. The exclusion from analysis of the first stimulus in a sequence also provided two other controls. The likelihood of ISI effects, where the duration of the preceding ISI can be a factor in the response to a stimulus, was reduced. Also, exclusion of the first stimulus of a sequence ensured that only trials where subjects held their gaze within the eye position window for at least 1 s were analyzed, so that saccade- or blink-related activity was minimized. The remaining steps of off-line data analysis are diagrammed in Figures 3–5 and described below. Data presented in these figures were obtained during one recording session with two electrode arrays simultaneously sampling laminar profiles in V1 and V2. In the V1 site (Fig. 3b–d), attentional modulation was negligible, while in the V2 site (Figs 4, 5) attentional modulation was substantial. For simplicity of presentation, the illustration begins in Figure 3 with the analysis of one CSD waveform from V1, that corresponding to the current sink in layer 4cß. Figures 4 and 5 expand the illustration of this analysis to show how it is applied to the entire CSD profile from the V2 recording site.
|
|
|
Determination of an Off-line Eye Position Window
For each experiment, maps of eye position were constructed by tabulating the number of sweeps for which the eye position fell within each position of a 10 x 10 grid of space, with each position of the grid subtending 1.5 x1.5 of the visual angle. Two such maps were constructed for each experiment, one for ignore-visual trial blocks and one for attend-visual trial blocks, and this is illustrated in the left and right panels of Figures 3a
. Each number within the boxes represents the proportion of times that the eye position fell within that location. As in this case, eye positions usually fell within a 5°2 area, approximately centered upon the visual stimulus display (Fig. 3a, shaded circle). This area was smaller in size than the on-line window that was used to gate stimulus presentation, and was defined as the post-recording session, or ‘off-line,’ eye position window (Fig. 3a, shaded square). The off-line eye position window was subdivided into five sections (Fig. 3a, center panel), in order to sort responses for statistical tests. The same eye position grid applies for the examples shown in Figures 3–5, since these recordings were performed simultaneously. For each attend condition, sweeps were binned on the basis of quadrantic position within the off-line eye position window, and four averaged CSD profiles for each attend condition were computed. Figure 3b shows an overlay of 4cß current sinks that were generated at each of the four positions of the eye position window for ignore visual (left panel) and attend visual (right panel) conditions.
Homogeneity of Neuronal Response within the Eye Position Window
To determine whether the variability of eye position within the off-line eye position window affected the neuronal responses, a test for homogeneity of the neuronal response at different positions in the off-line eye position window was performed. Generally, responses varied little with respect to eye position (Fig. 3b), and usually no significant (t-test; P < 0.05) differences in response at any time or section of the CSD profile as a function of eye position. Occasionally, there were differences in response as a function of eye position. If these differences were small (<1% of the integrated amplitude of the sum activity), they were accepted, but if differences were >1%, smaller off-line eye position windows were used. We decided, a priori, that the minimum number of sweeps within a quadrant be 100 to provide an adequate signal-to-noise ratio for analysis. If the eye position window did not have a homogeneous response with the minimum number of sweeps, data obtained from that recording site were binned separately (see Results section).
Statistical Tests of CSD Modulation at Each Recording Site
For each attend condition, five averaged CSD profiles were generated on the basis of eye position, one from each location depicted in the center panel of Figure 3a. Statistical tests were applied with two constraints. For a modulation of a response to be significant: (i) it must occur when the eye position falls within a very restricted range (position 5); and (ii) the modulation must be independent of the largest acceptable variation of eye position (e.g. 1 versus 4). These tests were applied to each waveform of the profile at each time point, and a Bonferroni correction was applied. To satisfy the first concern, a modulation of the CSD was scored as significant when the P < 0.05 criterion was exceeded in a running t-test for any period of time from onset of activity to 500 ms poststimulus, in any waveform of the averaged profiles generated at the centermost position (position 5) of the eye position window. This comparison and the resultant t-values are shown in the left panel of Figure 3c and in the left two panels of Figure 4. t-values are shown as waveforms where each point represents the t-value of the comparison at a given time point. Shaded boxes superimposed upon the t-value waveforms represent the range of values corresponding to P > 0.05. Thus, any excursion of the t-value waveform out of the shaded boxes indicates a significant difference in the CSD due to attention at that time point for that waveform of the profile.
To ensure that eye position variability did not account for the modulation, a second statistical test was designed. Responses within restricted portions of the eye position window were binned to produce four averages, one from each quadrant (Fig. 3a — center, positions 1–4). As shown in the right panel of Figure 3c, and the right two panels of Figure 4, four difference CSD profiles (attend/ignore) were computed by subtracting averaged responses from different portions of the eye position window in the following manner: attend visual at 1 — ignore visual at 4; attend visual at 2 — ignore visual at 3; attend visual at 4 — ignore visual at 1; and attend visual at 3 — ignore visual at 2, where 1–4 refer to quadrants of the eye position window depicted in the center of Figure 3a. Paired t-tests were used to test for differences between the attend and ignore waveforms in these four conditions in order to define any significant attentional modulation (P < 0.05) which exceeds differences due to eye position (see right-lower panel of Fig. 3c and rightmost panel of Fig. 4). As described in the preceding section, responses were usually homogeneous across eye positions. However, if the response was inhomogenous with respect to eye position, this test would underestimate attentional modulation. This is because the difference in electrophysiological response due to eye position could exceed and obscure the attention effect. Effects that are dependent upon eye position would tend to cancel.
These statistical tests delineated intervals for those waveforms of the profile in which significant modulation of the response occurred due to attention as opposed to small shifts in eye position. This is shown in the left panel of Figure 3d and in Figure 5, where responses to ignored (thin lines) and attended (thick lines) stimuli are overlaid with shaded regions denoting significant differences in response as determined by both tests. We quantified attentional modulation for those intervals and waveforms with significant modulation, as defined by both tests, as shown in Figures 3c and 4.
Quantification
Significant effects outlined by the above analyses were quantified by computing a modulation index (MI) of the CSD. The MI describes the net significant difference due to attention relative to the net physiological response at a given intracortical site. It also enables comparisons of modulation across different sites in different visual areas by normalizing the data. This computation is illustrated in Figure 5. First, the responses to attended and ignored visual stimuli are baseline corrected over the –100 to 10 ms peristimulus interval. Then, both response profiles, as well as the significant difference profile, are full-wave rectified, as shown in the center panel of Figure 5. The rectified CSD responses to attended and ignored stimuli are summed at each CSD channel, and the summed average rectified transmembrane current flow (sAVREC) is then computed. The significant difference average rectified current flow (dAVREC) across channels is also computed for the same location (Figs 3d and 5, right panels). In Figure 3d, the sum is computed over only the one (granular) lamina depicted, while in Figure 5, the sum is integrated over all laminae. Finally, the dAVREC and the sAVREC were each integrated over the interval from response onset to 450 ms after onset and expressed as a ratio, the MI. For the V2 site (Fig. 5), the MI equals 0.387. The MI for waveform corresponding to layer 4cß equals 0.0037. The onset latency was defined as the first point of 16 out of 18 consecutive points in a 2 kHz sampled, 1 Hz high-pass filtered, averaged rectified CSD of grouped attend visual and ignore visual conditions, that exceeded 2 SDs of the peristimulus interval (100 ms before stimulus onset and 10 ms after stimulus onset).
Characterization of the Temporal Pattern of Attentional Modulation.
The MI, as described above, is computed over the entire interval from response onset to 450 ms after response onset Thus, the MI collapses temporal information by reducing measures of activity and modulation into average transmembrane current flow over time. To investigate the time course of attentional modulation, two other computations were performed that retained temporal information. A modulation index may be computed for each time point by obtaining the ratio of the significant difference at time x (in mV/mm2) to the average transmembrane current flow from response onset to 450 ms after response onset (also in mV/mm2). Since this is a normalized measure, modulation waveforms for different recording sites may be averaged together to generate one grand mean waveform for a group of penetration sites.
The grand mean modulation waveform does not account for the stimulus-evoked response at any given time point. Therefore, a small modulation of a large response would be treated in the same fashion as a large modulation of a weaker response. To account for the stimulus-evoked response at a given time point, an interval MI was computed over restricted time intervals. The interval MI was defined as the ratio of the time-integrated significant difference over interval x – x1 (in mV/mm2) to the sum of the transmembrane current flow over interval x – x1 (also in mV/mm2). This measure depends upon the automatic response to the stimulus at a given time point such that small a modulation of a large response will have a smaller interval MI than the same modulation of a weaker neuronal response.
Surface ERP Recording
During each experiment, ERPs were recorded from a posterior site corresponding approximately to electrode position O1 of the International 10–20 System. Signals referenced to the same source as the multielectrode array, were amplified with a gain of 10 000 and a band pass of 1–300 Hz, and averaged together to form surface ERPs in parallel with the intracranial responses to the same stimuli.
Histology
Post-mortem, electrode penetrations were reconstructed histologically to confirm locations of recording sites. In later penetrations, electrolytic lesions were placed at selected sites by passing anodal current (5–10 µA, 10 s) through an electrode contact. Each subject was given a barbiturate overdose and perfused through the heart with 10% buffered formal saline. Parasaggital or coronal sections (80 µm; 5 per well) were cut parallel to the plane of electrode penetrations and stained with cresyl violet to aid in identifying electrode penetrations and electrolytic lesions [further details on histological reconstruction are given elsewhere (Schroeder et al., 1998)].
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionNotesReferences |
|---|
Detection and Quantification of Attentional Modulation across Visual Areas
The MI is the spatiotemporally integrated ratio of the significant difference in response to the sum response to attended and ignored stimuli. Figure 6 shows the distribution and average MI for all recording sites and is categorized by anatomically defined visual areas. Open and closed circles indicate each of the two subjects from which the data were collected. Cross lines are placed at MIs of 0.01, 0.05 and 0.15, which correspond to small, moderate and large modulations. An MI of <0.01, which corresponds to modulation of <2% of the sum response, represents a level of modulation within the range of baseline noise, and thus is considered negligible. The number of penetrations that yielded such ‘near-zero’ modulation values in each of the visual areas is given in parentheses. Although MI values differ appreciably across recording sites within an area, the MI has very small run-to-run variability within each recording site (see below).
Three patterns are evident in this figure. First, the net attentional modulation increases from the LGN to V1 to V2 up to V4. For each of 25 electrode penetrations in V4, 24 in V2, 41 in V1, and 5 in the LGN, the average MI was 0.170 ± 0.019 in V4, 0.101 ± 0.024 in V2, 0.0278 ± 0.0063 in V1 and 0.00156 ± 0.00080 in the LGN. These MIs would correspond to a mean of 41, 22, 8 and 0% modulation of the lower amplitude response respectively. Each of the LGN–V1, the V1–V2 difference and V2–V4 differences is significant (Kruskal–Wallace, P < 0.01). Eye position windows with homogeneous neuronal responses were defined for all LGN and V4 sites (see Materials and Methods). For the subset of V2 (16/24) and V1 (28/41) sites with homogeneous neuronal responses, the average MI was slightly higher than the overall average (0.127 ± 0.032 in V2 and 0.0343 ± 0.0099 in V1).
Second, the distribution also shows that within one cortical area, the modulation ranged from nonexistent to strong. For example, V2 had an average modulation index larger than that of V1, but its distribution was wide, with some sites showing no attentional modulation, and other sites showing large modulation. The third pattern that emerges from Figure 6 is that in the paradigm used here, areas of the ventral stream (V4 and IT) display a greater magnitude of attentional modulation than dorsal stream areas in the STS and in the IP sulcus. Despite this, there are a number of sites in dorsal stream areas where moderate and large amounts of attentional modulation do occur.
Variations in Modulation Amplitude
As shown in Figure 6, the distribution of MIs within each visual area is broad. For example, in V2 approximately half of sites show little to no modulation (MI < 0.05) and the remainder show moderate to large modulation (MI > 0.05). This distribution suggests a ‘patchiness’ in the organization of attentional modulation across sites. To address the possibility that the apparent patchiness reflects simple variability (across trials or recording sessions), rather than an underlying anatomical variation, we conducted two further analyses.
First, dual multielectrode recordings were made simultaneously in different V2 sites. Two such cases, shown in Figure 7, reveal dichotomous modulation patterns (MIs of 0.027 versus 0.183 and 0.011 versus 0.142). Note that the overall stimulus-evoked activity was comparable in magnitude across sites. Second, we assessed test–retest reliability of the attention effect in V1 and V2 (first pair of trial blocks versus last pair of trial blocks), as illustrated for two sites depicted in Figure 8. This analysis revealed small variations in modulation amplitude, but qualitative and quantitative stability of attention effects across trial blocks. Thus, the distribution in the amount of attentional modulation across recording sites within an area is likely to reflect an underlying feature of cortical organization rather than either a variability in our ability to resolve attention effects or day-to-day variability in physiology or behavioral state.
|
|
Temporal pattern of attentional modulation
The temporal activity pattern and the temporal pattern of attentional modulation in each recording site was described AVREC, as shown in Figures 5 and 7. The sAVREC represents the total visually evoked response collapsed across attention conditions, or the temporal activity pattern. The dAVREC is intended to represent the effects of attention, as isolated from visually evoked activity, or the temporal pattern of attentional modulation.
The left column of Figure 9 displays a comparison of attention effects using data from representative single penetrations in each of the areas sampled in the present experiments. The right column of Figure 9 displays the same sAVREC/dAVREC comparison in the form of normalized grand means of the entire data set for all the areas sampled in the present experiments. In order to enable comparison across areas with different numbers of observations, AVREC data from individual penetration sites were normalized on the basis of the average transmembrane current flow from onset to 450ms after onset, prior to computation of the grand means (see Materials and Methods).
These data reiterate the increasing influence of attention levels of the visual hierarchy up to V2 lower visual areas and V4 in the ventral stream. These data also illustrate three points that generally pertain to dorsal and ventral stream areas. First, independent of attention, the shortest latency visually evoked activity is observed in the dorsal stream areas and the rise time to peak response is steeper for dorsal stream than for ventral stream areas, consistent with our previous findings (Schroeder et al., 1998). Secondly, modulation appears larger in ventral stream areas than in dorsal stream areas. Finally, although this analysis concerns neural responses to attended nontargets, the initial portion of the modulation at and above the V2 level occurs before the reaction time range for target stimuli.
In comparing dorsal versus ventral stream modulation, it appears that in some cases (e.g. area IP in Figure 9, left), for a short period (30–75 ms), the average attentional modulation in dorsal stream areas exceeds that of the attentional modulation in ventral stream areas. However, grand means show that at its largest value for this period (50–75 ms) the modulation of the response is only slightly greater than 5% of the average transmembrane current flow over time.
Attentional Modulation over Time
In Figure 9, it is evident that attentional modulation begins in