Cerebral Cortex Advance Access originally published online on January 5, 2005
Cerebral Cortex 2005 15(9):1393-1413; doi:10.1093/cercor/bhi021
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© Oxford University Press 2005; all rights reserved
Neural Correlates of Spatial Judgement during Object Construction in Parietal Cortex
1 Brain Sciences Center, Veterans Affairs Medical Center, Minneapolis, MN 55417, USA, 2 Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA, 3 Department of Neurology, University of Minnesota, Minneapolis, MN 55455, USA and 4 Center for Cognitive Sciences, University of Minnesota, Minneapolis, MN 55455, USA
Address correspondence to Matthew V. Chafee, Brain Sciences Center (11B), Veterans Affairs Medical Center, One Veterans Drive, Minneapolis, MN 55417, USA. Email: chafe001{at}umn.edu.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
We recorded the activity of parietal area 7a neurons in monkeys performing an object construction task. In each trial, a model object consisting of a variable arrangement of squares was presented, followed after a delay by a copy of the model object that was missing a single square. Monkeys replaced the missing square to reconstruct the model configuration. Activity of many 7a neurons varied systematically with the position of the missing square and predicted where monkeys were going to add parts to the object they were building. The location of the missing square was a computed spatial datum important to object construction which did not correlate with the retinal location of a visual stimulus or the direction of the required motor response. The population of cells coding this coordinate was generally inactive when the same spatial locations were made relevant by visual targets to which monkeys either planned saccades or directed attention in other behavioral contexts. The data suggest that some parietal neurons participate in neural representations of space that reflect spatial cognitive as opposed to sensorimotor processing, coding the results of spatial computations performed on visual stimuli to meet cognitive objectives.
Key Words: area 7a • constructional apraxia • primate • spatial attention • spatial cognition
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The spatial functions of posterior parietal neurons in monkeys have often been investigated by recording neural activity while monkeys viewed, attended to or moved toward peripheral visual stimuli (Mountcastle et al., 1975
; Robinson et al., 1978; Hyvarinen, 1981). A principle of parietal neurophysiology deriving from this work has been that many posterior parietal neurons have visual receptive or movement fields — regions of space where visual sensory input or motor output is associated with neural activation. Thus the spatial coordinates coded by parietal neurons have most often represented the location of an attended stimulus or the direction of a planned movement. This has been confirmed in recent experiments examining the relation between neural activity in posterior parietal cortex and various forms of spatial cognition, including spatial attention (Mountcastle et al., 1981; Steinmetz et al., 1994; Gottlieb et al., 1998; Bisley and Goldberg, 2003), spatial working memory (Gnadt and Andersen, 1988; Chafee and Goldman-Rakic, 1998), motor intention (Bracewell et al., 1996; Mazzoni et al., 1996; Snyder et al., 1997, 1998a) and decision processing (Platt and Glimcher, 1999; Shadlen and Newsome, 2001; Sugrue et al., 2004). Typically in these studies, cognitive processing was seen to modulate either the intensity of neural discharge (as in the case of attention or decision processing) or the duration of neural discharge (as in the case of working memory or motor intention) that represented the location of a stimulus or the direction of a movement. Thus cognitive processing has been correlated with the modulation of either the timing or the intensity of sensorimotor representations in parietal cortex. It is an interesting question whether all forms of spatial representation encode the locations of stimuli or movements, or whether the brain can also sustain abstract spatial representations that do not. In order to solve a spatial problem, for example, it may be necessary for cortical neurons to compute and represent abstract spatial parameters that do not correlate with the locations of their visual receptive or movement fields and do not represent the locations of stimuli or the directions of movement. The question seems an important one in assessing the potential variety of neural processes engaged in spatial cognition, and whether and to what degree the neural representation of space can be dissociated from sensorimotor function.
In the present experiment, we investigated neural activity in parietal cortex during object construction. Our choice of this behavior for study was motivated by evidence that parietal damage can interfere with a person's ability to assemble or construct objects, while sparing basic motor function. This defect in the spatial organization of movement is known as constructional apraxia (Kleist, 1934), and follows damage to various cortical areas (Benson and Barton, 1970; Arena and Gainotti, 1978; Carlesimo et al., 1993), but is most common and severe following lesions of the right posterior parietal cortex (Black and Strub, 1976; Villa et al., 1986; Ruessmann et al., 1988). Consequently object construction is likely to depend at least in part on spatial processing normally supported by the intact posterior parietal cortex.
Objects are typically constructed in order to match a desired configuration or ‘blueprint’, and consequently the act of construction involves largely directing the addition of new parts. As the constructive process progresses, a builder has to compare the object that they have constructed so far to a central representation of the object they ultimately want to construct. The difference between these spatial representations determines where new parts should go. This comparative process is typically formalized in clinical tests of constructional ability in which a model object is presented to patients, who are then instructed to produce a copy of it. The task facing the patient in these cases is to draw or assemble the parts of their copy together in such a way that the resulting structure matches the model object. The construction task we developed for monkeys incorporated this feature. Monkeys were trained to construct a copy of a model object, and to do this, they were required to compare the copy to the model and form a spatial judgement regarding the difference between them. If a comparative spatial operation such as this was performed during object construction, it seemed reasonable that this process would be reflected in (correlated with) neural activity in posterior parietal cortex, given the impact of parietal damage on constructional ability.
We found that during object construction, parietal neurons coded a spatial coordinate that correlated with the location of parts that were missing from copy objects in comparison to model objects monkeys were attempting to reproduce. These data related to one of the overall objectives of the work, which was to determine whether spatial information represented in the activity of parietal neurons could be identified as the product of spatial cognitive as opposed to sensorimotor processing. At the center of this distinction was the question of whether or not neural activity in parietal cortex could code spatial information that did not correspond to the location of a visual stimulus or the direction of a pending movement. The present results suggest that area 7a neurons, when confronted with complex and cognitively demanding spatial problems such as object construction, can represent spatial information that does not correlate with sensorimotor variables, but instead reflects the product of a computation performed on visual stimuli to meet cognitive objectives.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Two male rhesus macaque monkeys (weighing 6 and 4 kg) were prepared for single neuron recording using standard aseptic surgical technique under Isoflurane (1–2%) gas anesthesia. In each animal, two recording chambers (7 mm i.d.) were implanted bilaterally, one each above a craniotomy overlying area 7a in the left and right cerebral hemisphere. Four titanium posts were attached to the skull with titanium screws, and a halo was attached to provide an anchor point to stabilize head position during neural recording experiments. Analgesia was provided for a period of several days following surgery (Buprenex, 0.05 mg/kg BID, i.m.). Care and treatment of the animals conformed to the Principles of Laboratory Animal Care of the NIH (NIH publication no. 86-23, revised in 1995). The Internal Animal Care and Use Committees of the University of Minnesota and the Minneapolis Veterans Affairs Medical Center approved all experimental protocols.
Monkeys viewed visual stimuli appearing on a 19'' color computer monitor (Gateway EV900, 60 Hz refresh) centered at eye level, located at a distance of 42 cm. The left foot of each monkey rested on a single response pedal. Extension of the ankle produced pressure on the pedal closing a switch, and the time of switch closure relative to the stimulus sequence in the behavioral task determined whether or not the trial was correct (see below). Correct trials were rewarded with a drop (0.1 ml) of juice. Monkeys worked for either three or four trial sets each day. Each set contained 200–300 trials, requiring 45 min to 1 h to complete.
Data Collection
Eye position was measured using an infrared video eye tracking system sampling eye position at 60 Hz (ISCAN Inc., Burlington, MA). Single neuron activity was recorded with a 16-microelectrode matrix (Eckhorn system, Thomas Recording, GMbH, Giessen, Germany). The electrode signals were amplified (at a gain of 20 000), filtered (bandpass between 0.5 and 5 kHz), and action potentials were discriminated in each channel using a time-amplitude window discriminator (DDIS-1, Bak Electronics, Mount Airy, MD) or waveform discriminator (Multi Spike Detector, Alpha Omega Engineering, Nazareth, Israel). Spike timing was sampled with 40 µs resolution (DAP 5200a data acquisition processor, Microstar Laboratories, Bellevue, WA). Computer files containing the timing of action potentials relative to stimulus and behavioral events were saved for all neurons encountered. Data were obtained in groups of simultaneously recorded neurons (typically 25–30), and neurons whose average discharge rate exceeded 0.5 Hz during the model, delay and copy periods of the construction task were included in subsequent analyses. In both animals, 
15% of isolated neurons failed to meet this spontaneous activity criterion; the remaining 85% of all cells isolated comprise the present database.
We conducted two series of experiments. In experimental series 1, we recorded the activity of 1614 area 7a neurons in monkeys M1 and M2 as they performed the visual object construction task and also the delayed saccade task. In experimental series 2, we recorded the activity of 367 area 7a neurons in monkey M2 performing the visual object construction task, the mixed-choice construction task and the peripheral attention task (see below). The locations of recording sites were estimated from the magnetic resonance (MR) image of a metal microelectrode placed at the center of each chamber entering the inferior parietal gyrus (Fig. 2c). Microelectrode penetrations were restricted to a circular region 4.8 mm in diameter centered on this point and were confined to area 7a in the inferior parietal lobule. A small minority of penetrations encroached on the bounding intraparietal and superior temporal sulci, although neurons within the banks of these sulci were not sampled. MR localization of recording location was confirmed in the first monkey at sacrifice (Fig. 2d). MR and histological localization were in close agreement (compare Fig. 2c,d, ‘Monkey 1’).
|
The Visual Object Construction Task
In the construction task, monkeys reconstructed the configuration of a remembered model object by adding elements to an incomplete copy object. Each model and incomplete copy object was made up of a group of square elements in various configurations. (Each square element was blue in color and subtended 1.4° of visual angle. Adjacent squares were separated by a 0.25° gap.) These stimuli were adapted from those used by Driver and colleagues to demonstrate object-based neglect after parietal lobe damage (Driver et al., 1994). The squares making up each object were placed on a regular five by five grid (8.3° in both width and height). Each model and incomplete copy included an object frame (Fig. 1a), consisting of a base row and central column of square elements forming an inverted ‘T’. The frame provided a base and central axis to the object. Model objects varied in the number and position of square elements added to this frame. Models contained either one (Fig. 1b) or two (Fig. 1c) additional squares on either the left or right sides of the object at various vertical positions, producing a set of 36 different configurations. Monkeys were trained on the task using the complete set of model objects, but a subset was used to record neural activity in order to reduce the number of trials that had to be performed for each set of neurons. We employed two basic variants of the construction task. In one of these, the model and copy objects appeared at the same location in the visual display, both centered on the central fixation target (Fig. 2a, ‘Model’ and ‘Incomplete copy’ panels of ‘Trial 1’). In this case, monkeys performed the task using the subset of eight model configurations shown within the solid square outlines in Figure 1b,c. In the second variant, the model and copy objects appeared at different locations in the visual display. The model object was offset a fixed distance to one side of the fixation target or the other at random, such that the model appeared entirely within either the left or right visual hemifields (Fig. 2a, ‘Model’ panel of ‘Trial 2’). The incomplete copy appeared as before centered on the fixation target (Fig. 2a, ‘Incomplete copy’ panel of ‘Trial 2’). In this case, monkeys performed the task using the subset of 14 model configurations shown within both the solid and dashed square outlines in Figure 1b,c.
|
Monkeys initiated the construction trial sequence by directing their gaze to a small fixation target (a red spot of light) presented at the center of the display. The monkey was required to maintain its gaze position within 1.3–1.7° of the fixation target until the trial was complete (Fig. 2a; the approximate size of the fixation window relative to the model object is indicated by the dotted circle in the ‘Model’ panel of ‘Trial 1’). If at any time before the end of the trial gaze position deviated by more than this distance from the fixation target the trial was terminated. After 500 ms of fixation, a model object was presented for 750 ms (Fig. 2a, ‘Model’) and then disappeared. The disappearance of the model marked the beginning of a 750 ms delay period in which only the fixation target was visible (Fig. 2a, ‘Delay’). At the end of the delay period an incomplete copy object appeared (Fig. 2a, ‘Incomplete copy’). The copy object remained visible for 750 ms, before an array of choice squares appeared (Fig. 2a, ‘Choice array’). Neural activity during the 750 ms period beginning with the appearance of the incomplete copy object and ending with the appearance of the choice array is the focus of this report. Each incomplete copy object was identical to the model object preceding it on a given trial, except that one square element had been removed, referred to as the ‘missing square’ (Fig. 2b). The missing square was always one of the additional squares placed adjacent to the object frame in the model and not one of the squares making up the frame itself. In the case of model objects having two squares on the same side of the object and at the same vertical position (second row of objects in Fig. 1c), the missing square was always the outermost of the two squares added to the frame. In order to successfully complete the trial, monkeys were required to add a new square to the incomplete copy object in order to replace the missing square and so reconstruct the model configuration. This was done by selecting one of the two choice squares in the choice array. One choice square was the correct choice, and was located at the same vertical position and on the same side of the incomplete copy object as the missing square. (In Fig. 2a, ‘Trial 1’, the correct choice is the leftmost of the two choice squares. In Fig. 2a, ‘Trial 2’, the correct choice is the lower of the two choice squares.) The incorrect choice square was a distracter. The array of choice squares randomly appeared in one of two orientations, forming either a horizontal array (Fig. 2a, ‘Choice array’ panel of ‘Trial 1’) or a vertical array (Fig. 2a, ‘Choice array’ panel of ‘Trial 2’). In horizontal choice arrays, correct and incorrect choice squares were located on opposite sides of the copy object. In vertical choice arrays, correct and incorrect choice squares were located on the same side of the copy object. After a variable interval (300–600 ms), one of the choice squares selected at random increased in luminance for a period of 700–1000 ms (Fig. 2a, ‘1st choice’). If the monkey pressed the response key while the first choice square was bright (as in Fig. 2a, ‘1st choice’ panel of ‘Trial 2’), the first choice was selected for addition to the incomplete copy object. If the monkey did not press the response key in this interval, the first choice square dimmed to its original luminance at the same time that the second choice square brightened. The second choice square then remained bright for a period of 700–1000 ms (Fig. 2a, ‘2nd choice’ panel of ‘Trial 1’). If the monkey pressed the response key during this interval, the second choice square in the sequence was selected for addition to the incomplete copy object. In either case, at the time that the response key was pressed, the dim choice square disappeared and the bright hoice square translated smoothly inward toward the model in a horizontal direction, coming to rest adjacent to the outermost square in the copy object to produce a new configuration (Fig. 2a, ‘Complete copy’). The complete copy remained visible for a period of 300 ms before disappearing at the end of the trial. If the monkey selected the correct choice square, its addition produced a complete copy object whose configuration matched the preceding model, and the trial was rewarded after the complete copy period (Fig. 2a; compare ‘Model’ and ‘Complete copy’ panels of ‘Trial 1’). If the monkey selected the incorrect choice square, an erroneous configuration resulted. The complete copy object did not match the preceding model, and the trial was not rewarded (Fig. 2a; compare ‘Model’ and ‘Complete copy’ panels of ‘Trial 2’).
Delayed Saccade Control Task
We recorded the activity of area 7a neurons while monkeys performed a delayed saccade task. After a period of central fixation (500 ms), a saccade target was presented at one of eight peripheral locations (Fig. 5a). Fixation and saccade targets were both red spots. Saccade target locations corresponded to the centers of missing squares and choice squares in the construction task (object squares were not visible in the saccade task but are shown as dotted outlines to illustrate their locations relative to saccade targets). After a fixed delay of 750 ms, the central fixation target disappeared, and the monkey made a saccade toward the peripheral target. The trial was rewarded if that saccade ended within 1.3–1.7° of the peripheral target location and gaze was maintained at that location for an additional 500 ms.
|
Dual Attention–Construction Task
We used a dual task paradigm in order to probe the allocation of visual attention during object construction. In this dual task, all model objects consisted of the frame plus a single additional square (Fig. 1b). Most trials (85 or 75%) were standard construction trials as described above. A minority of trials (15 or 25%) were attention probe trials. The monkey was not cued as to which trial type was in progress — construction and probe trials progressed through model, delay and copy periods presenting identical visual stimuli. On probe trials, 150–750 ms after the incomplete copy object appeared, a visual probe stimulus was flashed for 50 ms and the choice square array did not appear. The monkey's task was to depress the response key as soon as it detected the probe (the key press had to occur between 50 and 600 ms after probe onset). The probe stimulus was a single red square whose contrast was greater than threshold (the monkey successfully detected 93% of probe stimuli). The horizontal position of the probe stimulus was varied relative to the position of the missing square (Fig. 9), and was expressed in degrees offset from that position. The probe was presented either at the position of the missing square (position 0°), at the mirror opposite location within the copy object (position –3.3°), at the location of the correct choice square (position 3.3°) or at the location of the incorrect choice square (position –6.6°).
|
Mixed-choice Construction Task
The horizontal position of the missing square and the correct choice square were dissociated in a mixed choice task. Choice squares were arranged in a vertical array and presented in one of two locations relative to the missing square. On standard trials, the choice array appeared on the same side of the copy object as the missing square, and so the correct choice and the missing square shared the same horizontal position relative to the central axis of the copy object (Fig. 10a, left). On crossed trials, the choice array appeared on the opposite side of the copy object as the missing square, and so the correct choice and missing square had different horizontal positions relative to the central axis of the copy object (Fig. 10a, right). In either case, the correct choice was located at the same vertical position as the missing square. The monkey was cued as to whether a standard or crossed trial was in progress by the color of the fixation target (red indicated a standard trial, green indicated a crossed trial). On crossed trials, the correct choice square translated across the central axis of the copy object and came to rest at the missing square location on the opposite side of the object.
|
Peripheral Attention Task
We recorded the activity of 7a neurons while monkeys directed attention toward a peripheral visual stimulus in order to detect its dimming (Bushnell et al., 1981; Colby et al., 1996). Trials began when monkeys fixated a central fixation target (a blue spot of light). After 500 ms a second blue spot target identical to the fixation target appeared in a peripheral location (Fig. 11a, left). Monkeys were required to maintain fixation of the central fixation target while directing attention to the peripheral target in order to detect a subtle decrease in its luminance that occurred at an unpredictable time. The peripheral target was presented in one of four locations selected at random. Peripheral target locations corresponded to the centers of the four possible locations of the missing square relative to the fixation target in the construction task (the missing square locations are indicated by the dotted square outlines; Fig. 11a, left). After a random interval of between 400 and 1400 ms, the peripheral target dimmed slightly (Fig. 11a, right). The monkey was rewarded for pressing the response key between 50 and 500 ms following the dimming of the peripheral target.
|
Data Analysis
All data analyses were implemented in Matlab (The MathWorks, Inc., Natick, MA). We employed an analysis of covariance (ANCOVA) to assess the influence of task variables on neural activity. The ANCOVA was implemented as a general linear model (Kleinbaum et al., 1988; Howell, 1997) estimated by the REGRESS function in the Matlab Statistical Toolbox. All ANCOVA analyses included two covariates. The first covariate was the baseline firing rate during a 500 ms period while the monkey fixated the central target before the appearance of the model object at the beginning of each trial. The second covariate was the time elapsed since the beginning of data collection for a given set of isolated neurons and the start of each trial in the set. The influence of the independent variables on neural activity was therefore assessed independently of either fluctuation in basal activity across trials or linear trends in activity across the recording session. An 
level of 0.05 was used unless otherwise noted.
In experimental series 1, we analyzed neural activity on the construction and delayed saccade tasks. In the construction task, the dependent variable in the analysis was the firing rate of a given neuron on each trial during the incomplete copy period. The copy period began with the appearance of the incomplete copy object, and ended with the appearance of the choice array 750 ms later. The two fixed factors in the analysis were the horizontal position (left/right) and the vertical position (high/low) of the missing square in the incomplete copy object coded as categorical factors (Fig. 2b). These factors together defined the location of the missing square relative to the central axis (horizontal position) and base (vertical position) of the incomplete copy object. In the delayed saccade task, the dependent variable was the firing rate on each trial during the delay period. This period began with the appearance of the saccade target in peripheral vision, and ended with the offset of the fixation target, which was the go signal to make a saccade to the target. The two fixed factors were the horizontal position (left/right) and vertical position (high/low) of the saccade target, again coded as categorical factors. Trials with near targets (at the locations of removed squares in the incomplete copy object) and far targets (at the locations of choice squares) were pooled (Fig. 5a).
In experimental series 2, we employed an ANCOVA to analyze the activity of neurons in a mixed-choice construction task. The dependent variable in this analysis was neural activity during the copy period. The independent variables were the horizontal position of the missing square (left/right), and the horizontal position of the choice square (left/right), with respect to the central axis of the copy object. A relation between neural activity and the position of the choice square was anticipatory in nature because the neural activity analyzed was measured during the copy period, and the choice array had not yet appeared at this point in the trial (Fig. 2a). Horizontal positions of missing and choice squares were included as two independent factors in this design because these positions were uncorrelated in the mixed-choice task. In experimental series 2, we also analyzed the activity of a group of neurons that were recorded on both the standard construction and the peripheral attention tasks. In this group of neurons, we simplified the ANCOVA by collapsing the coding of position into a single factor with four levels (to code the location both of the missing square and also the peripheral target). This simplified the comparison between tasks to the pattern of significance obtained across two (instead of four) factors, as was the case in analyzing the mixed-choice task data.
We constructed spike density functions (SDF) from the pooled activity of neural populations. First we identified (partially overlapping) populations of neurons whose activity varied significantly either with the horizontal or the vertical position of the missing square as assessed by the above ANCOVA. For the purposes of constructing population spike density functions in Figures 4 and 5, we used a stringent statistical criterion to select cells with the strongest signals in the population (including only those neurons whose firing rates were affected by the relevant spatial variable at P < 0.001). Trials were segregated into preferred and nonpreferred groups as a function of the location of the missing square along the horizontal or vertical axis that was associated with the greatest activity in each neuron. Each trial, represented by a sequence of spike times, was then converted into a SDF evaluated at 1 ms intervals using the KSDENSITY kernel smoothing function in the Matlab Statistical Toolbox. A Gaussian kernel of 
= 15 was used to construct population spike density functions (in the case of the spike density functions constructed from the activity of the single neuron in Figure 3, a kernel of = 20 ms was used). Individual trial SDF were then averaged across the trials in preferred and nonpreferred groups separately for each neuron and normalized (by subtracting the minimum at each point and dividing by the range). These average normalized SDF for each neuron were again averaged across all the neurons in each population to yield the population SDF in Figures 4, 5, 10 and 11. In Figures 4 and 5, the standard error of the mean of the population SDF was computed across neurons and is represented by the region of lighter shading surrounding each SDF. When neural activity was contrasted across tasks, the preferred horizontal or vertical position of each neuron was defined by the activity of that neuron on the standard construction task, and this preferred position was then used to segregate trials in either the delayed saccade task (Fig. 5d), the mixed-choice construction task (Fig. 10d,f) or the peripheral attention task (Fig. 11e) into preferred and nonpreferred groups. The difference between population activity on preferred and nonpreferred trials in the various control tasks therefore illustrates the degree to which neurons in the population maintained a consistent spatial preference across the tasks.
|
|
We recovered the location of the missing square by decoding neural population activity using linear discriminant analysis (Johnson and Wichern, 1998
; Averbeck et al., 2003). Linear discriminant analysis (LDA) is a multivariate statistical technique that classifies observations to categories on the basis of a group of discriminating variables. In the present analysis, observations were trials of the construction task, and each trial was represented by a population activity vector consisting of the mean firing rate of each neuron in a population during the copy period of the task. The population activity vector therefore represented the distribution of firing rates across the neurons in the population on a given trial while the copy object was visible. There were four categories in the analysis, corresponding to the four possible locations of the missing square in the copy object. Classification of trials to categories was carried out using the CLASSIFY function of the Matlab statistical toolbox. A discriminant function for each category was derived by computing the parameters of a multivariate normal probability density distribution whose mean was equal to the mean population activity vector averaged across the trials in a given category and whose covariance was estimated by pooling trials across categories (see equation 3 of Averbeck et al., 2003). Equal prior probabilities were assumed (as the square could be removed from any one of the four potential locations with equal probability). The analysis proceeded by using the discriminant functions to compute a set of scores for each trial associated with its membership in each of the categories. The trial was then classified to the category with the highest score. This was carried out using fivefold cross-validation — the mean and covariance of the discriminant functions for the categories were defined on the basis of four-fifths of the trials and the remaining fifth was treated as being of unknown category and was classified using LDA. The procedure was replicated for successive fifths of the data until all trials were classified. Generally, LDA performance scales with the number of neurons included in the analysis (Averbeck et al., 2003). The present analysis was based on a population of 7a neurons whose firing rate during the copy period varied significantly with the location of the missing square. This population included neurons that belonged to different ensembles recorded at different times (that is not all of the neurons in the population were recorded simultaneously). Neurons recorded at different times were aggregated into this larger population by considering a single trial in the analysis to be a particular repetition of a given pair of model and copy objects. The firing rates of all neurons in the population on a given trial defined in this way were then grouped into a single population activity vector.
In a separate analysis, we recovered the location of the missing square when monkeys chose the incorrect choice square and built the incorrect object. We identified error trials and divided them into four groups. These four groups included trials in which the missing square was located on the left side of the copy but the monkey added a square on the right, and vice versa; and trials in which the missing square was located at the upper position within the copy but the monkey added a square at the lower position, and vice versa. We constructed neural populations for each error type. The neurons included for each error type were those in which (i) activity on correct trials varied significantly with the position of the missing square along the spatial axis (horizontal or vertical) associated with that type of error (ANCOVA; P 
0.05); and (ii) a neuron was recorded while the monkey committed at least 10 errors of that type. Once the populations had been identified, categories for LDA classification were constructed on the basis of the activity in each population on correct trials. In the case of each population and error type, the classification included two categories. In populations recorded during horizontal errors, the two categories corresponded to the left and right side of the copy object. In populations recorded during vertical errors, the two categories corresponded to the high and low positions in the copy object. We then classified error trials to these categories using LDA as above. On error trials, one position in each classification corresponded to the location of the missing square whereas the other corresponded to the location of the square that the monkey erroneously added to the object (on error trials these locations were different).
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
We recorded the activity of 1981 neurons in posterior parietal area 7a (Fig. 2c,d) of two male rhesus macaque monkeys (M1 and M2) as they performed the visual object construction task (1092 of these neurons were recorded within the first monkey, 889 in the second). Of these cells, 1614 were recorded from both monkeys in experimental series 1 (standard construction task plus delayed saccade task). An additional 367 neurons were recorded from M2 in experimental series 2 (standard construction task plus mixed-choice and peripheral attention tasks, see Materials and Methods). Monkey M1 performed 83% of construction trials correctly in which the model object appeared randomly in left or right visual hemifields, and 94% of trials correctly in which model and probes both appeared centered in the display. Overall performance in series 1 was 87% correct. Monkey M2 performed 90% of construction trials correctly in which the model object appeared randomly in left or right visual hemifields, and 91% of trials correctly in which model and probes both appeared centered in the display. In series 2, Monkey M2 performed 97% of mixed-choice construction trials correctly, with comparable performance on standard (99% correct) and crossed (97% correct) trials. Performance was more accurate in monkeys M1 and M2 on trials with vertical choice arrays (88% and 91% respectively) than with horizontal choice arrays (86% and 89% correct respectively), although the difference was slight (only 3% difference in both animals).
Neural Activity in 7a Correlates with Spatial Judgement during Object Construction
In the construction task, monkeys viewed two objects in sequence — first the model object and then, after a delay, the incomplete copy object. Both objects consisted of varying arrangements of identical squares presented on a video monitor. The incomplete copy was identical to the preceding model each trial, except that a single square had been removed (Fig. 2).
We found that after the incomplete copy appeared, 7a neurons became active as a function of the location of the square that had been removed from the incomplete copy object relative to the preceding model. An example of this pattern of activity in a single 7a neuron is shown in Figure 3. During the incomplete copy period (indicated by the vertical gray bar of shading through each raster), neural activity increased on trials in which model and incomplete copy objects together connoted a square removed from the lower left position in the incomplete copy object (Fig. 3b,f,j). This increase in activity was not a function of the pattern of visual input at the time it occurred. On trials in which the model consisted of the object frame plus one additional square, the incomplete copy object consisted of the frame only. In spite of the fact that the incomplete copy object was physically identical across these trials, its appearance evoked markedly different levels of neural activity depending on where the square had been removed from the copy object (Fig. 3a–d). On trials in which the model consisted of the object frame plus two additional squares, the neuron exhibited the same preference, and incomplete copy objects in which squares had been removed from the lower left position excited the neuron (Fig. 3f,j); physically identical copy objects from which squares had been removed at other locations did not (compare copy objects in Fig. 3f and l, and those in Fig. 3j and h). Activity during the incomplete copy period varied across trials with the same model object (compare model objects in Fig. 3e and f, and those in Fig. 3j and k), and so was not likely to represent either a working memory of the model configuration or the reactivation of a visual receptive field stimulated by the model. Activity during the incomplete copy period did not reflect an association between a particular pair of model and incomplete copy objects because it generalized across all object pairs in the set that together defined a missing square at the preferred position (Fig. 3b,f,j).
We assessed the influence of the location of the missing square in the copy object on neural activity during the copy period using a two-way ANCOVA. The two orthogonal fixed factors in the analysis (horizontal and vertical position) jointly determined the location of the missing square (Fig. 2b). The activity of approximately one in four area 7a neurons during the incomplete copy period that met a minimal activity criterion (see Materials and Methods) varied significantly as a function either of the horizontal position (27%) or the vertical position (24%) of the square removed from the incomplete copy object relative to the preceding model (Table 1, ‘Combined’). The proportions of neurons exhibiting these effects were comparable in the two monkeys considered individually (Table 1, ‘Monkey 1’ and ‘Monkey 2’). The group of neurons whose activity was significantly related to either the horizontal or vertical position of the missing square (or both spatial factors) included 43% of all spontaneously active area 7a neurons isolated for study (700 of 1614 cells).
|
Trials in which model and incomplete copy objects appeared at the same or different locations in the display are pooled in Table 1. Trials in Table 2 are restricted to those in which model and incomplete copy objects appeared at different locations (the model object was offset into the left or right visual hemifield while the copy object was centered on the fixation target). The effects of the position of the square removed from the incomplete copy were equally prevalent in this subset of the data. This is evidence that neural activity in 7a coding where the square had been removed from the incomplete copy object did not require that the corresponding square in the model object appear at the same retinotopic location.
|
The set of model objects included some in which two squares were stacked together on the same side and at the same height in the model object (second row of model objects in Fig. 1c). In these objects, the missing square (always the outermost of the two squares added to the frame) was located at a farther eccentricity from the central axis of the object than in other model configurations. We tested the effect of the eccentricity of the missing square on the activity of those neurons coding the horizontal position of the missing square. We found that the activity of approximately one in five of these neurons (62 of 275 neurons coding horizontal position in Table 2) was significantly influenced not only by the side but also by the eccentricity of the missing square relative to the central axis of the object (the activity of these neurons was significantly affected by both the side and eccentricity of the missing square at P
0.05 in a two-way ANCOVA employing side and eccentricity as factors).
We selected neurons in which activity during the copy period varied significantly (P 0.001) with either the horizontal position (Fig. 4a,c; 152 neurons) or the vertical position (Fig. 4b,d; 108 neurons) of the missing square. We then constructed average normalized population spike density functions (SDF) illustrating the time course of activity in these two populations (see Materials and Methods). In each panel (Fig. 4a–d), separate SDF illustrate population activity on trials in which the missing square was in the preferred position (blue SDF), or the nonpreferred position (red SDF) of each neuron included in the population. (The preferred position of each neuron was defined as the position of the missing square associated with the greatest activity during the copy period.) On the subset of trials in which the model object consisted of the frame plus one additional square (Fig. 1b), the copy object consisted of the object frame only (Fig. 1a), and so the visual form of the copy object did not vary. Population activity on these trials clearly differentiated the horizontal (Fig. 4a) or vertical (Fig. 4b) position of the missing square during the copy period (note the separation between blue and red SDF), in spite of the fact the same copy object was presented on every trial. This confirmed at the population level what was seen at the single cell level (Fig. 3) — that neural activity during the copy period was not a function of the form of the copy object. On these trials, we also noted that the population signal coding the position of the missing square emerged early in the trial, shortly after the onset of the model object (Fig. 4a,b; note black arrows pointing at the point of divergence between the blue and red SDF during the model period). We wondered whether the early emergence of this population signal might be due to the fact that on this subset of trials monkeys could anticipate the location of the missing square with certainty on the basis of the model object alone. (Models with one additional square determined the position of the missing square unambiguously because only the one additional square could be removed to produce the copy object, and consequently the missing square could be located at only one position). To examine this possibility, we constructed neural population activity functions (Fig. 4c,d) using trials in which model objects included two squares in addition to the object frame (Fig. 1c). In this case, monkeys could not anticipate the location of the missing square on the basis of the model object alone, because either of the two additional squares present in the model could be removed to produce the copy. On these trials, we found that the emergence of the population signal coding the horizontal position (Fig. 4c) and the vertical position (Fig. 4d) of the missing square was delayed until after the copy object appeared (Fig. 4c,d; note black arrows pointing at the divergence between blue and red SDF during the copy period). Neural population activity therefore varied with the missing square both in space and in time — coding the horizontal and vertical position of the missing square at the point in time that the position of the missing square was fixed by model and copy objects.
Potential Relation of Neural Activity to Oculomotor Factors
It was not likely that neural activity during the incomplete copy period reflected motor planning for the upcoming response. The monkey pressed a single response key with its foot every trial and movement direction did not vary. To examine whether neural activity might reflect oculomotor intention, we tested the activity of 7a neurons on both the construction task and a delayed saccade task. Saccade targets were placed at locations that included those of the missing squares, and also the squares in the choice array (Fig. 5a). Under the hypothesis that neural activity during object construction reflected the oculocentric direction of a planned saccade, the same population of neurons should be active in the two tasks and exhibit the same spatial preference for missing squares and saccade targets. Instead we found that the object construction and delayed saccade tasks recruited largely distinct neural populations. Of the 444 neurons in the sample whose activity varied with the horizontal position of the missing square during object construction (Table 1, ‘Combined’), activity in only 83 neurons (19%) varied significantly with the horizontal position of the saccade target in the delayed saccade task. Likewise, of the 393 neurons in the sample whose activity varied with the vertical position of the missing square during object construction (Table 1, ‘Combined’), activity in only 56 neurons (14%) varied with the vertical position of the saccade target in the delayed saccade task. Therefore 84% of the population whose activity coded the location of the missing square along one or the other spatial axis during object construction failed to code the location of the saccade target along that axis in the context of the delayed saccade task. In the 139 neurons in which activity varied significantly with both the position of the missing square and the saccade target, we calculated two positional preference indices (see Materials and Methods) for each neuron recorded in the two tasks. The index quantified the degree to which each neuron preferred left versus right or high versus low positions. The position preferences of single neurons were weakly correlated between the saccade and construction tasks (r = 0.246, P = 0.003); but there was considerable scatter and the indices were not tightly aligned. Many neurons preferred saccade targets and missing squares at opposite horizontal or vertical positions (Fig. 5b; points in upper left and lower right quadrants).
We selected the subpopulation of neurons whose activity was most strongly related to the horizontal position of the missing square during object construction (Fig. 5c; the neurons included were those in which activity significantly related to the main effect of horizontal position at P 0.001). We then examined the activity of these neurons during the delayed saccade task, segregating trials into preferred and nonpreferred groups according to whether the saccade target was located at the same horizontal position each neuron preferred in the construction task or not. We found that whereas neural activity in this population clearly varied with the horizontal position of the missing square during the construction task (Fig. 5c), population activity did not vary with the horizontal position of the target in the delayed saccade task (Fig. 5d). The directional signal evident in the activity of this neural population when monkeys were building objects was largely absent when monkeys were planning saccades.
Whereas few neurons active during object construction were active during the saccade task, the saccade task in itself was effective in driving a largely distinct population of neurons in area 7a. For example, the activity of 7a neurons varied significantly (P 0.05) as a function of the horizontal position of the saccade target both during the delay period (266 neurons) and the saccade period (271 neurons) of the delayed saccade task. Population activity clearly differentiated the position of the saccade target during both task epochs (Fig. 5e,f; neurons included were those in which neural activity related significantly to the horizontal position of the target at P 0.001; 61 and 48 neurons during delay and saccade periods respectively). Neural activity during the delay period of the saccade task may have been related to the visual stimulation provided by the continuing presence of the saccade target, or to the monkey's directing attention toward the target. Neural activation during the saccade period was largely postsaccadic. Population activity peaked approximately 50 ms after the eyes entered the response window surrounding the saccade target.
During the construction task, neural activity may have alternatively related to a variation in eye position within the fixation window. To test this possibility, we assessed the distribution of sampled eye position within the fixation window as a function of the location of the missing square. We found that the average eye position within the fixation window varied by only 0.1° across trials with different missing square locations (Fig. 6). A negligible variation in firing rate would be predicted by a variation in eye position this small (Andersen et al., 1990b).
|
Neural Activity Predicts Performance in the Construction Task
We decoded neural population activity using linear discriminant analysis (LDA; see Materials and Methods) with fivefold cross-validation (Johnson and Wichern, 1998) to provide a measure of how well population activity represented the location of the missing square. The analysis classified trials to one of four categories corresponding to the four possible missing square locations in the experiment. The classification was based on the pattern of activity observed in a population of 7a neurons. The frequency with which the classification was correct indicated the strength of the prediction about the location of the missing square we could make on the basis of the firing rates of 7a neurons. The results of the trial classification procedure are shown in Figure 7. Trials were first segregated into four groups based on the known position of the missing square. Trials in each group were then classified to four categories, corresponding to the potential locations of the missing square. The heights of the four vertical columns in each copy object (Fig. 7a–d) are proportional to the frequency that trials of each type were classified to each of the four corresponding categories in the analysis. In each case, the tallest column representing the category to which the majority of trials were classified corresponds to the true location of the missing square on each trial. Overall, 85% of trials were correctly classified on the basis of the pattern of population activity.
|
On error trials, the square missing from the copy object and the square added to the copy object were located at different positions. We identified four types of error trials on this basis: trials in which the missing square was on the left side of the object and the added square was on the right side (Fig. 8a), or vice versa (Fig. 8b), and trials in which the missing square was at the high position and the added square was at the low position (Fig. 8c), or vice versa (Fig. 8d). We classified each of these error trials to one of two categories corresponding to the position of the missing square or the added square on each error trial (see Materials and Methods). LDA classified most error trials to the category corresponding to the position of the added square and not the missing square (Fig. 8a–d). This indicated that the population activity pattern predicted where the monkey was going to add a square when it committed an error in construction, establishing an association between neural activity and the animals' behavioral performance. The overall accuracy of classification performance on error trials (Fig. 8) was lower than that on correct trials (Fig. 7). This was likely to reflect the availability of fewer neurons contributing to the population on which the error trial classification was based.
|
