Cerebral Cortex Advance Access originally published online on December 7, 2005
Cerebral Cortex 2006 16(10):1494-1507; doi:10.1093/cercor/bhj082
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Cortical Area MSTd Combines Visual Cues to Represent 3-D Self-Movement
Departments of Neurology, Brain and Cognitive Sciences, Neurobiology and Anatomy, and Ophthalmology and the Center for Visual Science, The University of Rochester Medical Center, Rochester, NY 14642, USA, 1 Current address: Box 673, School of Medicine and Dentistry, University of Rochester Medical Center, Rochester, NY 14642, USA
Address correspondence to Dr Charles J. Duffy, Deptartment of Neurology, University of Rochester Medical Center, Rochester, NY 14642-0673, USA. Email: Charles_Duffy{at}urmc.rochester.edu.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
As arboreal primates move through the jungle, they are immersed in visual motion that they must distinguish from the movement of predators and prey. We recorded dorsal medial superior temporal (MSTd) cortical neuronal responses to visual motion stimuli simulating self-movement and object motion. MSTd neurons encode the heading of simulated self-movement in three-dimensional (3-D) space. 3-D heading responses can be evoked either by the large patterns of visual motion in optic flow or by the visual object motion seen when an observer passes an earth-fixed landmark. Responses to naturalistically combined optic flow and object motion depend on their relative directions: an object moving as part of the optic flow field has little effect on neuronal responses. In contrast, an object moving separately from the optic flow field has large effects, decreasing the amplitude of the population response and shifting the population's heading estimate to match the direction of object motion as the object moves toward central vision. These effects parallel those seen in human heading perception with minimal effects of objects moving with the optic flow and substantial effects of objects violating the optic flow. We conclude that MSTd can contribute to navigation by supporting 3-D heading estimation, potentially switching from optic flow to object cues when a moving object passes in front of the observer.
Key Words: extrastriate • MST • optic flow • vision • visual cortex • visual motion
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Visual cues about self-movement are derived from the patterned visual motion of optic flow that provides information about heading direction and the spatial layout of the environment. In addition, observers can use the visual motion of discrete objects to judge their relative self-movement. The interpretation of object motion is complicated by the independent movement of animate objects (Gibson 1950
). Interactions between neural responses to combined optic flow and object motion could facilitate the recognition of animate motion to enhance the detection of predators and prey.
One difficulty in combining optic flow and object cues about self-movement is their putative analysis in separate visual cortical processing pathways. Optic flow analysis occurs in the dorsal extrastriate "where" pathway for localization, and object analysis occurs in the ventral extrastriate "what" pathway for identification (Mishkin and others 1983). Dorsal stream responses to objects might contribute to the analysis of an object's identifying features (Geesaman and Andersen 1996), implying a breakdown of the dorsal–ventral dichotomy in medial superior temporal (MST). Instead, the dorsal–ventral dichotomy has been extended to subdivide MST (Komatsu and Wurtz 1988a). Dorsal medial superior temporal (MSTd) is thought to be specialized for optic flow analysis related to self-movement perception (Orban and others 1992; Saito 1993; Duffy and Wurtz 1995). Ventrolateral MST is thought to be specialized for processing the motion of discrete objects passing through the visual field (Komatsu and Wurtz 1988a; Tanaka and Saito 1989; Tanaka and others 1989).
The applicability of the dorsal–ventral dichotomy to MST is complicated by MSTd's responsiveness during the pursuit of moving objects (Komatsu and Wurtz 1988b) and during changes in the shape of objects (Sugihara and others 2002). This apparent conflict between the dorsal–ventral dichotomy and MSTd physiology might be reconciled if all of MSTd's responses relate to the processing of relative movement signals, accessing both optic flow and object motion as cues about self-movement.
Such a role in self-movement analysis is consistent with MSTd's access to vestibular signals (Thier and Erickson 1992; Duffy 1998; Bremmer and others 1999) as well as MST's use of eye position (Bremmer and others 1998) and eye movement (Komatsu and Wurtz 1989; Bradley and others 1996; Page and Duffy 2003) signals about gaze changes that alter the observer's view.
We have tested this hypothesis in studies of optic flow and object motion processing in MSTd. We find that MSTd neurons can represent self-movement heading in three-dimensional (3-D) space with a uniform distribution of heading preferences. MSTd's population responses estimate 3-D heading from either optic flow or object motion cues. When optic flow and object motion are combined, an object moving as part of the optic flow field has little effect, whereas an object moving against the optic flow field can dominate the population's response. Thus, MST can combine optic flow and object motion cues about self-movement to support 3-D heading estimation.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Animal preparation
Single-neuron responses were recorded from 5 cerebral hemispheres of 3 rhesus monkeys (2 males, 1 female, ages 4–8 years). Surgery was performed under general anesthesia using inhaled isoflurane, implanting bilateral scleral search coils (Judge and others 1980), a head holder, and bilateral recording cylinders. The recording cylinders were placed over 1-cm trephine holes above area MST (AP, –2 mm; ML, ±15 mm; dorsal–ventral angle, 0°) and encased in a dental acrylic cap. Postoperative analgesia with Banamine (1 mg/kg, intramuscular) was administered as judged appropriate by the veterinary staff. Following recovery from surgery, the monkeys were trained to sit in a primate chair and perform a visual fixation task monitored with magnetic search coils (Robinson 1963). In all trials, the monkey's task was to maintain fixation within an area 2.5° x 2.5° around the centered fixation point. All protocols were approved by the University Committee on Animal Research and complied with the Society for Neuroscience and Public Health Service Policy on the care of laboratory animals.
Visual Stimuli
Stimuli were presented in a pseudorandom sequence with blocking by the stimulus types described subsequently. The stimuli were generated by a personal computer driving a television projector (Electrohome ECP4100) at 60 Hz to cover the central visual field (90° x 90°) for 2 s. The 14 headings (Fig. 1A) simulated observer movement in 8 directions in the horizontal plane and frontoparallel planes with 45° separation within each plane and overlap on the interaural axis.
|
Optic flow stimuli consisted of 350 white dots (0.19° at 2.61 cd/m2) on a dark background (0.18 cd/m2) with dot motion simulating observer movement with respect to a remote frontoparallel surface. Dots moved at an average speed of 40°/s, the middle of the range of preferred speeds for MSTd neurons (Duffy and Wurtz 1997
). The dots were evenly distributed in a smoothed, random pattern in the first frame with lifetimes of 1–60 frames and speed proportionate to twice the sine of angle from the observer's line of sight in centered fixation to the dot in question, a sine 2
function of the viewing angle. Object motion stimuli consisted of an outline of dots forming 3 concentric circles and 4 radial lines with transparent segments between these structural features. The object moved to simulate the image of an earth-fixed landmark, as seen by an observer moving relative to that object. The heading of the simulated observer movement was the basis for naming the corresponding object motion stimulus. For example, leftward observer movement would be accompanied by rightward object motion through the observer's visual field and would be termed a leftward heading. The object stimuli were clearly visible, even on congruent optic flow patterns, because of shape, texture, and density cues.
In addition to translational movement across the screen (40°/s for movement in the frontoparallel plane), when appropriate, the object also changed its size. Size changes ranged from 5° to 20° diameter (averaging 10°) and simulated the observer's approach or recession relative to the object. The frontoparallel plane objects began and ended at the edge of the screen, forward and backward objects were always centered, and oblique horizontal motions ranged from 15° to 40° eccentricity.
Optic flow and object motion stimuli were presented alone and in combination. In same direction combined stimuli, the object moved as an element of the flow field to simulate an earth-fixed object. In opposite direction combined stimuli, the object motion violated the surrounding optic flow field to simulate an independently moving object on a heading 180° offset from the heading in the optic flow. The object did not occlude the flow; that is, pixels within the bounds of an object, which were not illuminated as part of the object, could be illuminated as part of the superimposed optic flow.
In a subset of the neurons studied, we also presented combined stimuli in which the optic flow and object motion headings were offset by ±90° in the simulated plane of self-movement. All combined motion stimuli included the same set of 14 simulated heading directions described earlier. Responses from combined stimuli with 180° or 90° offset heading directions are referred to by, and analyzed relative to, their optic flow direction.
Neural Recording
Tungsten microelectrodes (Microprobe, Inc.) passed through a transdural guide tube to record neuronal activity (Crist and others 1988). A dual-window discriminator digitized discharges that were stored with event markers on the REX experiment control system (Hays and others 1982). MSTd neurons were identified by their physiologic characteristics including large receptive fields (>20°2) containing the fixation point, with direction-selective responses, and a preference for large moving patterns rather than moving bars or spots (Komatsu and Wurtz 1988a; Duffy and Wurtz 1991). The location within MSTd was confirmed with deeper penetration to obtain middle temporal neuron responses. When experiments were completed, electrolytic marking lesions were placed at selected sites in the recording region. Histological analysis confirmed that all recording sites were in the anterior bank of the superior temporal sulcus in the area corresponding to the zone of heavy myelination associated with MSTd.
Data Analysis
Post–stimulus time histograms were generated from spike times smoothed by a 20-ms Gaussian and averaged over 5–7 trials. Control trials included fixation point stimulation only.
Two types of data analyses were used: 1) An analysis of firing rates during periods of significant neuronal responsiveness to visual motion stimuli was used to compare activation by optic flow and by object motion (as in Figs. 1–8). Significant responses were periods of at least 300 ms in which the 1–standard deviation envelope of stimulus-evoked activity was greater than the 1–standard error (SE) envelope of control activity. Firing rate was averaged across significant response period in each cell. This analysis provides a measure of each neuron's level of activation by each self-movement simulation. 2) An analysis of firing rates in all eight 250-ms intervals of each 2-s stimulus was used to characterize interactions between superimposed optic flow and object motion stimuli (as in Figs. 9–16). Firing rate was averaged across all 250 ms of each interval in each cell. This analysis provides a measure of each self-movement simulation's contribution to the activity of each neuron when those stimuli are presented separately or together.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Tuning surfaces derived from the 3-D heading responses were fit to Kent spherical distributions (Fisher and others 1987
). The Kent fits yield unidirectional profiles that can vary from spheres to symmetrically or asymmetrically elongated ellipsoids (as in Fig. 4). These 3-D profiles can rotate in any plane to conform to directional preferences in the responses (as in Fig. 5). The Kent fits can also flatten in any plane to match the cross-sectional asymmetry of some response profiles (as in Fig. 7). The Kent fits produce directional strength, preferred direction, and goodness-of-fit measures for each stimulus type in each neuron. We measured the strength of directional tuning using a directional index [DI = 1 – (Kent fit's amplitude at its antipreferred direction/fit's amplitude at its preferred direction)]. Significant direction selectivity was identified using a modification of Hotelling's test for 3-D distributions (Fisher and others 1987). Two-way, unbalanced-design analysis of variance (ANOVA) was performed on each set of responses with 14 heading directions and eight 250-ms stimulus epochs. Multiple linear regressions characterized the relative influence of optic flow and object motion with combined stimuli. Goodness-of-fit was obtained from residual distances of the data from the fits. Multiple regression using sigmoidal functions did not yield substantially better fits than did the planes fit by linear regression.
Object Motion and Receptive Fields
We obtained hand maps of their receptive fields of 86 neurons. Maps were obtained with both projected white dot patterns covering 30° x 30° and a projected white bar 3° x 7° and moved in the direction that evoked the most distinct responses from the neuron under study. All mapping stimuli were presented during fixation of the red fixation point at the center of the screen. We digitized these maps and smoothed their boundaries using a 5° spatial filter implemented in Matlab to recognize the spatial uncertainty inherent in our estimates of receptive field dimensions based on hand mapping.
We superimposed the course of object motion at 1-ms intervals on each neuron's receptive field maps. We used only the frontoparallel planar directions of object movement because they provide a clear correspondence between the spatial location of the stimulus and defined epochs in the response. We identified the location of all significant object motion responses, as defined in the response duration analysis, and coded those as responsive zones and coded all others as nonresponsive. We then derived correlations between the location of the moving object stimulus during significant responses and the boundaries of the hand-mapped receptive fields using a Pearson's phi correlation coefficient for 2 binary variables.
Population Responses
We followed the approach of Georgopoulos and others (1986) by combining the preferred direction from each neuron into a population response making 3 assumptions: First, each neuron's directional responses can be represented as a net vector that indicates that neuron's preferred heading. Second, a neuron's response to a heading stimulus, minus unstimulated control activity, indicates the size of its effect on the population response. Third, the response vectors of all neurons sum to create a net vector that represents the population response,
 |
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Optic Flow and Object Motion Responses
We presented optic flow and object motion stimuli simulating 14 directions of self-movement in 3-D space (Fig. 1). Visual motion–sensitive neurons characteristic of MSTd (see Methods) were identified using manually controlled stimuli. The responses of 138 MSTd neurons were recorded to both the optic flow and object motion stimuli (Fig. 2).
Responses were defined as 
300-ms segments of the stimulus period in which the SE envelope of evoked activity was above the SE envelope of control activity (Fig. 3A). These criteria were developed by successive analyses in which we compared automated response identification with the results of our inspection of all the single-neuron responses to all the stimuli. A total of 91% (125/138) of the neurons responded to optic flow stimuli and 87% (120/138) to object motion stimuli. A total of 81% (112/138) of the neurons responded to both optic flow and object motion.
Optic flow responses tended to have somewhat larger amplitudes (median = 20.2 spk/s) than object motion responses (median = 16.9 spk/s). However, this distinction was not as marked as might have been expected from the size of the stimuli. In fact, the distribution of amplitudes of optic flow and object motion responses was not significantly different across the sample of neurons (Wilcoxon rank sum P = 0.07) (Fig. 3C).
Responses to the 2-s optic flow stimuli were of substantially longer duration (median = 1949 ms) than responses to the 2-s object motion stimuli (median = 1112 ms). The more sustained responses to optic flow resulted in a significant difference in the distribution of durations of optic flow and object motion across the sample (Wilcoxon rank sum P < 0.001) (Fig. 3B).
We considered that the shorter duration of object motion responses might reflect a correspondence between the location of the object and the boundaries of a neuron's receptive field. To address this issue, we digitized receptive field maps derived from hand mapping with dot pattern and moving bar stimuli (Fig. 4A). We superimposed a map of object motion responsiveness based on the location of the object and whether the neuron yielded a significant response when the object was at that location. Pearson's phi correlation coefficient was derived for each neuron by relating the 2 binary variables of whether there was a significant response at a given location and whether that location was within or beyond the receptive field.
Correlation coefficients were averaged within each neuron for the dot pattern and for the moving bar receptive field maps. The distribution of these values across neurons yields means that are significantly different from zero (P < 0.0001) for both the dot pattern maps (Fig. 4B, mean = 0.21) and the moving bar maps (Fig. 4C, mean = 0.25). We conclude that there is a link between object motion responses and receptive field dimensions, but that link does not fully explain object motion response characteristics.
Thus, MSTd neurons responded to both optic flow and object motion stimuli. We found similar response amplitudes with both types of stimuli. However, responses to optic flow were more sustained than those evoked by object motion. Sustained responses to optic flow may result from interactions between large receptive fields and the large-field–patterned motion of optic flow. Transient responses to object motion may result from interactions between more responsive zones within the receptive field and the limited spatial extent of object stimuli crossing the receptive field. The similarity of the response durations might be more surprising than the differences but for the complex receptive fields properties of MSTd neurons (Duffy and Wurtz 1991; Lagae and others 1994).
Encoding Heading in 3D
Neuronal responses to optic flow and object motion stimuli simulating relative self-movement were plotted in 3-D polar coordinates. The polar coordinate frame is oriented with reference to the monkey, and each response is plotted toward its direction of relative self-movement at a distance from the origin in proportion to the evoked firing rate (Fig. 5A).
Both optic flow and object motion responses were fit by generalized normal distributions in 3-D space (Kent distributions) (Fig. 5B). Good fits (r2 > 0.6) were obtained for most optic flow (66%) and object motion (59%) responses (Fig. 5C). Smaller sampling intervals in heading space would have provided more data for the Kent fits. Nevertheless, the available data suggest that most MSTd neurons can provide a veridical contribution to the neural representation of self-movement heading in 3-D space. This 3-D heading representation can rely on the cues in optic flow or object motion.
We measured the strength of 3-D heading direction selectivity using a conventional directionality index (DI = 1 – anti/preferred). The great majority of optic flow (90%, 112/125) and object motion (82%, 99/120) responses showed strong 3-D direction selectivity (DI > 0.5) (Fig. 6A). Most neurons with significant directionality to both optic flow and object motion (68%, 82/120) showed similar heading preferences to both stimulus sets (median directional difference = 40°) (Fig. 6B).
The similarity in single-neuron–preferred headings to optic flow and object motion represents good agreement between heading representations based on 2 different types of stimuli. An appropriate context for considering the degree of directional agreement is provided by comparison with the chance distribution of 2 directions in 3-D space (Fig. 6C). These findings suggest that MSTd can represent 3-D self-movement heading direction based on either optic flow or object motion. This may be viewed as further evidence of cue invariance (Geesaman and Andersen 1996) in MSTd's contributions to complex motion analysis.
Single-neuron–preferred headings to optic flow and object motion stimuli were uniformly distributed in 3-D space. Although inspection might detect a cluster at 1 location in the distribution or a gap in another, there is no significant unimodality or bimodality (P > 0.05). This suggests that single-neuron heading preferences are not concentrated on any particular heading direction, such as the straight ahead heading, or in any particular plane of self-movement, such as the ground plane (Fig. 7). Thus, MSTd's representation of 3-D self-movement appears to be isotropic, encoding all possible heading directions equally, without evidence of an intrinsic bias about the observer's 3-D heading.
However, there is a second source of potential bias in MST's representation of 3-D heading direction: bias that can be revealed by examining the manner in which the relative strength of heading responses is arranged around each neuron's preferred heading. In fact, the 3-D fits of many optic flow (24%) and object motion (20%) responses are highly asymmetrical, flattened, around the preferred heading (Fig. 8A, left). This flattening is seen in fits to the 3-D response profiles as an elliptical cross section in a plane that is orthogonal to the preferred heading at the 3-D fits widest extent (Fig. 8A, right). The major axis of such flat cross sections can be seen as defining a wedge-shaped section of a plane through the origin of the 3-D space. For such neurons, all strong responses to self-movement headings lay in that plane.
We quantified the flatness of MSTd neuronal 3-D response profiles by comparing the relative length of the major and minor axes of the profile's cross section [flatness = 1 – (minor/major)]. The distribution of the flatness of the response profiles for optic flow and object motion is similar (Fig. 8B). Both distributions show that most profiles have circular cross sections. However, 30% of the optic flow response profiles and 24% of the object motion response profiles have distinctly flat cross sections.
The critical issue is whether the planes defined by the distribution of strong responses might be concentrated in 3-D space. This could mean that the neuronal population's capacity to represent particular headings might not be isotropically distributed, even though the preferred headings of single neurons are isotropically distributed. We examined the distribution of the planes defined by the flatter response profiles by representing each plane as a circle projected onto the unit sphere. The planes defined by both the optic flow and object motion response profiles are distributed uniformly in 3-D space (Fig. 8C, P < 0.05 to reject significant uni- or bidirectionality). This implies that there is no special tendency toward greater responsiveness to movement in any particular plane in 3-D space. In particular, there is no apparent concentration of responsiveness in the ground plane, as might have been expected from greater experience with locomotion in the ground plane.
Combined Optic Flow and Object Motion
In nature, object motion is commonly superimposed on optic flow. This occurs both with earth-fixed objects moving through the visual field as a feature in the optic flow seen by a moving observer and as animate objects moving independently of the optic flow field created by the observer's movement. We superimposed object motion on optic flow in 2 stimulus sets creating a set of same direction combined stimuli simulating the addition of an earth-fixed object and a set of opposite direction combined stimuli simulating the independent movement of an animate object (Fig. 9). The interactions between optic flow and object motion in response to combined stimuli were evident in the responses of most neurons tested (Fig. 10).
Same direction combined stimuli yielded response interactions that resulted from a stronger influence of optic flow. The optic flow responses often masked any clear effects of the object motion stimuli, even though the object motion stimuli evoked clear responses when presented alone (Fig. 11A, top). Opposite direction combined stimuli yielded response interactions that resulted in a stronger influence of object motion (Fig. 11A, bottom). These responses often revealed an effect of object motion that was not evident when the object motion was presented alone.
Response interactions with both the same and the opposite direction combined stimuli created subadditive effects. We measured this subadditivity by first dividing the 2-s stimulus periods into eight 250-ms response intervals (Fig. 11B). This approach incorporates a consideration of the differing time courses of optic flow and object motion responses. To the extent that object motion responses are more transient because of the object's interactions with neuronal receptive field organization, the use of briefer response intervals allows the separate measurement of interactions between optic flow and object motion for every segment of the object's trajectory.
Each 250-ms response interval was used as a separate test of the additivity of response interactions. Additivity was defined as the amplitude of each response to the combined stimuli minus the sum of the amplitudes of responses to the 2 corresponding stimuli when presented alone. Across all intervals from all responses of all neurons, we found that both the same and the opposite direction combined stimuli evoked responses that were, on average, less than the sum of the responses to the separate stimuli (Fig. 11C), that is, subadditive interactions.
Subadditive response interactions suggest the potential applicability of a linear but nonadditive model of interaction between optic flow and object motion responses. Multiple linear regression was used to assess the relative influence of optic flow and object motion on the same direction and opposite direction combined stimuli. In this analysis, we again used response amplitude in the eight 250-ms intervals of all responses. Here, we fit a plane to the surface defined by comparing responses to combined and separate stimuli. The goodness-of-fit of the surface to the data was measured by an r2 statistic, 50% (68/137) of the same direction and 33% (35/107) of the opposite direction fits yielded r2 values >0.5. The relative influence of optic flow and object motion on the combined responses was measured as the ß weight for each: the higher the ß weight, the greater the influence of that type of stimulus; purely additive response interactions would yield ß weights of 1 for both stimuli.
Single neurons typically showed a greater influence of optic flow in responses to same direction combined stimuli. This is reflected in a steeper slope of the fit plane along the optic flow axis and a correspondingly higher regression ß weight for optic flow (Fig. 12A, left). These neurons also showed a relative increase in the influence of object motion in responses to opposite direction combined stimuli. This is reflected in a steeper slope of the fit plane along the object motion axis and a correspondingly higher regression ß weight for object motion (Fig. 12A, right).
We applied this analysis to all the neurons studied with combined stimuli. The ß weights for optic flow commonly decreased, and those for object motion commonly increased, from same direction to opposite direction combined stimuli. This is reflected in the average ß weights for each type of combined stimulus—same direction: ßflow = 0.58, ßobject = 0.28; opposite direction: ßflow = 0.39, ßobject = 0.43 (Fig. 12B). The shift to greater relative influence of the object in opposite direction stimuli is also seen when r2 > 0.5 is required for both the same and the opposite direction fits (same direction: ßflow = 0.87, ßobject = 0.25; opposite direction: ßflow = 0.69, ßobject = 0.45). These findings suggest that an object moving against the optic flow substantially influences neuronal responses, whereas an object moving with the optic flow has relatively little effect.
Population Responses to Heading Stimuli
We used population vector analysis to compare MSTd's composite neural response across stimulus conditions. This approach provides measures of the strength and accuracy of the population vector as relative indices of MSTd's heading estimation in response to each type of stimulus.
MSTd's population response to each type of heading stimulus was derived from the sum of the responses of all neurons. Each neuron's responses were fit with the 3-D Kent function so that all responses contributed to that neuron's net vector. Each neuron's net vector is described by 2 values from the Kent fit: the net vector's direction in 3-D space describes the neuron's preferred direction and the net vector's magnitude describes the strength of that directional preference.
We combined all responses under all stimulus conditions to derive each neuron's contributions to the population responses. This is a more conservative approach than the separate derivation of population responses for each stimulus condition. The use of separate response derivations would assume that the neuron is specifically encoding a particular type of stimulus. Under that assumption, its ability to represent the heading simulated by the stimuli of that type might be considered self-evident. Instead, we accept that the neurons might not have a priori information about the current stimulus, so it would not be able to encode different types of stimuli in a different manner. We assume that every neuron contributes to the encoding of all relevant stimuli. Thus, their ability to represent heading under different stimulus conditions is a better test of its potential contributions to a behaviorally relevant signal about the heading of self-movement in 3-D space.
Each neuron's contribution to the population response to a selected stimulus is a vector having the direction of that neuron's preferred direction and a length having the amplitude of that neuron's response to the heading stimulus under consideration. The 14 heading stimuli yield population net vectors with average resultant lengths that are 60% larger for optic flow (Fig. 13A) than for object motion (Fig. 13B) (260 vs. 163 population spk/s = 1.60) (F1,16 = 30.65, P < 0.0001). Thus, optic flow and object motion can support a veridical population estimate of heading direction, but optic flow yields a more robust signal.
The greater strength of the population response to optic flow is consistent with same direction combined optic flow and object motion stimuli yielding single-neuron responses and population vector resultant lengths that are not significantly different from those evoked by optic flow alone (F1,16 = 0.41, P = 0.53) (Fig. 13C). In marked contrast, responses to opposite direction combined stimuli yield population vector resultant lengths that are much smaller than those evoked by optic flow alone (F1,16 = 195, P < 0.0001) (Fig. 13D).
Thus, the influence of object motion on optic flow responses in combined stimuli seems to depend greatly upon the relative heading directions simulated by the combined stimuli. Same direction combined stimuli yield population responses much like those obtained with optic flow alone. Opposite direction combined stimuli yield responses that are substantially more affected by the addition of object motion.
We tested whether object motion's impact on combined stimuli might vary across the 2-s stimulus period. Population net vectors were derived for the eight 250-ms time intervals of all responses to compare the 4 stimulus conditions. Population responses to optic flow alone showed accurate net vectors that strengthened in the first 500 ms (Fig. 14A). In contrast, population responses to object motion alone built up in the first 1 s and subsided in the second 1 s (Fig. 14B). Population responses to same direction combined stimuli were only slightly less accurate than those evoked by optic flow alone (Fig. 14C). However, population responses to opposite direction combined stimuli were smaller than all the others, with the middle intervals showing variable directions that commonly pointed away from the direction simulated by the optic flow (Fig. 14D).
We further compared the population net vectors from the 4 stimulus conditions by measuring their average directional error: the difference between the heading simulated by the stimuli and the heading estimated by the population. Heading error varied substantially across stimulus conditions and across the 2-s stimulus periods. In the optic flow alone and the same direction combined stimulus conditions, heading error rapidly declined to establish accurate representations of the simulated self-movement direction. Their directional errors averaged less than half the 45° separation of stimulus directions and maintained that level throughout the stimulus period (Fig. 15A,C). Object motion alone yielded larger heading errors but improved across the 2-s stimuli (Fig. 15B).
Opposite direction combined stimuli showed a dramatic increase in heading error during the first 1 s of the stimulus that is followed by decreasing error in the second 1 s (Fig. 15D). This represents increasing deviation from the self-movement heading simulated by the optic flow when the object moved toward central vision (stimulus x time ANOVA of all 4 conditions F3,383 = 61.21, P < 0.016; all post hoc comparisons to opposite direction stimuli P < 0.001). When the object moved outward from central vision, in the course of lateral movement in the frontoparallel plane, heading error declined to resume its veridical representation of the simulated heading in optic flow. This effect was not seen when the object remained in central vision during movement in depth along the horizontal plane.
We considered that the large net vector direction errors seen only with opposite direction combined stimuli might reflect population heading estimates of the object motion's heading. However, data from those studies cannot differentiate such effects from random changes in net vector directions: The object's heading was always the opposite of the optic flow's heading, and both shifts toward the object's heading and random changes would move the net vector away from the heading in the optic flow.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Self-Movement in 3D
The current findings are the first demonstration of 3-D linear translational heading estimation by MSTd neurons (Figs. 2–6
). Previous studies demonstrated that MSTd neurons can represent simulated self-movement in the depth plane (Duffy and Wurtz 1995; Lappe and others 1996) and simulated or real self-movement in the ground plane (Duffy 1998). Those studies tested a narrower range of self-movement directions and suggested that MSTd might overrepresent headings around the direction of gaze (Duffy and Wurtz 1995) or in the ground plane (Maunsell 1988). A variety of simulated 3-D self-movement directions have been used previously (Paolini and others 2000), but the data were not integrated into a test of 3-D heading estimation. Our findings show that MSTd can support a systematic representation of 3-D heading estimation and that it maintains an isotropic coverage of 3-D heading direction preferences (Fig. 7) across all planes in 3-D space (Fig. 8).
The isotropic distribution of single-neuron–preferred 3-D headings is surprising in view of the fact that the headings tested were confined to the frontoparallel and ground planes used previously. One can reasonably expect that more extensive sampling should have yielded still more uniform distributions, but practical considerations limit the number of directions sampled. Our findings suggest that the need to maintain a balanced neuronal population to support unbiased heading estimation may have precedence over the need to concentrate heading estimation in the most common directions and planes of self-movement. MSTd's representation of 3-D self-movement may support its role in controlling pursuit eye movements (Komatsu and Wurtz 1988a; Bremmer and others 1997). There is a clear need for a 3-D relative movement signal to drive the frontal cortical control of 3-D pursuit eye movements (Tian and Lynch 1996; Gamlin and Yoon 2000; Fukushima and others 2002). This function might be served by posterior parietal projections to frontal cortex (Petrides and Pandya 1984) with reciprocal connections (Stanton and others 1995) promoting the use of 3-D pursuit signals to support optic flow analysis during eye movements (Bradley and others 1996; Page and Duffy 1999, 2003; Shenoy and others 2002). A corresponding interaction between 3-D self-movement signals and motor cortical control systems might contribute to guiding limb movements during self-movement (Merchant and others 2001, 2004).
Optic Flow and Object Motion Cues
We find that individual MSTd neurons can access the self-movement cues in both optic flow and object motion. The responses to both cues have comparable firing rates and heading direction selectivities. Furthermore, single neurons show similar preferred heading directions in response to optic flow and object motion stimuli (Fig. 5). Object motion responses were briefer than optic flow responses, creating greater response variability across the 2-s stimuli and potentially accounting for smaller population net vectors compared with optic flow.
Transient responses to object motion may reflect the object's crossing more responsive segments of the receptive field. These transient responses might be more evident in our studies because we presented movements that crossed the central 90° of the visual field. These extensive movements distinguish our stimuli from the 5° to 20° movements more commonly used in studies of MT and MST (Tanaka and others 1993; Recanzone and others 1997). Comparison of object motion responses with hand-mapped receptive field boundaries yielded only modest correlations (Fig. 4), although this may reflect the limits of hand mapping.
Large-scale object motion has been used in studies of the dorsal-anteriorly adjacent areas 7a (Motter and others 1987; Steinmetz and others 1987) and STP (Oram and others 1993). Our findings suggest that MSTd might contribute to 7a and STP responses as a large-scale, direction-selective intermediary between extrastriate visual motion processing and parietotemporal association cortex.
Adding same direction object motion to the optic flow field does not alter human heading estimation (Royden and Hildreth 1996) or MSTd's population responses (Figs. 12–15); in single-neuron responses to combined stimuli, optic flow seemed to mask any effect of same direction object motion. This is consistent with the perceptual phenomenon of inhibitory interaction in which visual or vestibular signals about self-movement can either elevate object motion detection thresholds (Probst and others 1986) or decrease the perceived speed of object motion (Brenner 1991).
A different circumstance arises with object motion that violates the optic flow field and suggests the presence of an animate object. Superimposing such an animate object disrupts human heading estimation (Royden and Hildreth 1996; Royden 2002). MSTd's population responses show a corresponding sensitivity to the frontoparallel motion of animate objects, particularly during movement toward central vision. This might reflect object interactions with MSTd's large and diverse receptive fields (Raiguel and others 1997) that share the unifying property of including the central visual field (Komatsu and Wurtz 1988a). Animate objects might also engage the center-surround organization of some MSTd neurons (Komatsu and Wurtz 1988b) that are activated by opposite directions of central and peripheral motion (Eifuku and Wurtz 1998).
It must be kept in mind that the greater effect of object motion on responses to the opposite direction combined stimuli is not as simple as a hot spot in the center of the visual field. Neither the object moving alone nor the object moving with congruent optic flow shows evidence of such a hot spot. We should infer that the opposite direction optic flow activates specific receptive field properties that result in these effects. Thus, stimulus interactions may shape receptive field properties to create performance characteristics that are suited to a role in complex, naturalistic circumstances.
Navigation and Evasion
Our findings suggest that either optic flow or object motion stimuli can support 3-D self-movement heading analysis for navigation and orientation (Figs. 13–15). Large population vector errors, in the range of 20°, should be considered in the context of our sampling only 
120 neurons. Perceptual performance can be reasonably supposed to rely on neuronal populations that are several orders of magnitude larger than this sample. In addition, perception is likely to reflect the integration of single-neuron responses in a manner that is not considered in our derivation of population responses.
The effects of object motion in the central visual field might be viewed as a failure of heading analysis when an object obscures the observer's field of view. Such circumstances have been thought to impair the perception of relative object motion and self-movement during driving, increasing the risk of collisions (Probst and others 1984). This is consistent with the population net vector amplitudes being much smaller when combined optic flow and object motion stimuli are superimposed to simulate opposite heading directions.
Alternatively, the direction of MSTd's population net vector may be more important than the overall amplitude. In this context we should consider that object motion's effects on MSTd's heading signal might be an adaptive mechanism. When a moving object violates the surrounding optic flow as it approaches central vision, it triggers a transformation in MSTd's population responses so that MSTd encodes a heading much closer to that of the object rather than encoding the observer's heading that is implied by the superimposed optic flow (Figs. 14–16).
The increased effect of the object in opposite direction combined stimuli may be viewed as transforming the representation of observer self-movement from an environmental reference frame to an object-based reference frame. Such a transformation might be useful in evading or intercepting animate objects (McBeath and others 1995) that come to be aligned with central vision during observer self-movement.
|
Acknowledgments |
|---|
We gratefully acknowledge the assistance of William Vaughn, Jennifer Postle, and Sherry Estes in this research and artwork by Teresa Steffenella. We thank Drs Marc J. Dubin, Roberto Fernandez, Michael T. Froehler, Voyko Kavcic, and William K. Page as well as William Vaughn for comments on the manuscript. This work was supported by EY10287 from the NEI to CJD.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Bradley DC, Maxwell M, Andersen RA, Banks MS, Shenoy KV. 1996. Mechanisms of heading perception in primate visual cortex. Science 273:1544–1549.[Abstract]
Bremmer F, Distler C, Hoffmann KP. 1997. Eye position effects in monkey cortex. II. Pursuit and fixation-related activity in posterior parietal areas LIP and 7A. J Neurophysiol 77:962–977.
Bremmer F, Kubischik M, Pekel M, Lappe M, Hoffmann KP. 1999. Linear vestibular self-motion signals in monkey medial superior temporal area. Ann N Y Acad Sci 871:272–281.[CrossRef][Web of Science][Medline]
Bremmer F, Pouget A, Hoffmann KP. 1998. Eye position encoding in the macaque posterior parietal cortex. Eur J Neurosci 10:153–160.[CrossRef][Web of Science][Medline]
Brenner E. 1991. Judging object motion during smooth pursuit eye movements: the role of optic flow. Vision Res 31:1893–1902.[CrossRef][Web of Science][Medline]
Crist CF, Yamasaki DS, Komatsu H, Wurtz RH. 1988. A grid system and a microsyringe for single cell recordings. J Neurosci Methods 26:117–122.[CrossRef][Web of Science][Medline]
Duffy CJ. 1998. MST neurons respond to optic flow and translational movement. J Neurophysiol 80:1816–1827.
Duffy CJ, Wurtz RH. 1991. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J Neurophysiol 65:1329–1345.
Duffy CJ, Wurtz RH. 1995. Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. J Neurosci 15:5192–5208.[Abstract]
Duffy CJ, Wurtz RH. 1997. Medial superior temporal area neurons respond to speed patterns in optic flow. J Neurosci 17:2839–2851.
Eifuku S, Wurtz RH. 1998. Response to motion in extrastriate area MSTl: center-surround interactions. J Neurophysiol 80:282–296.
Fisher NI, Lewis T, Embleton BJJ. 1987. Statistical analysis of spherical data. Cambridge, UK: Cambridge University Press.
Fukushima K, Yamanobe T, Shinmei Y, Fukushima J. 2002. Predictive responses of periarcuate pursuit neurons to visual target motion. Exp Brain Res 145:104–120.[CrossRef][Web of Science][Medline]
Gamlin PD, Yoon K. 2000. An area for vergence eye movement in primate frontal cortex. Nature 407:1003–1007.[CrossRef][Medline]
Geesaman BJ, Andersen RA. 1996. The analysis of complex motion patterns by form/cue invariant MSTd neurons. J Neurosci 16:4716–4732.
Georgopoulos AP, Schwartz AB, Kettner RE. 1986. Neuronal population coding of movement direction. Science 233:1416–1419.
Gibson JJ. 1950. The perception of the visual world. Boston: Houghton Mifflin.
Hays AV, Richmond BJ, Optican LM. 1982. A UNIX-based multiple process system for real-time data acquisition and control. WESCON Conf Proc 2:1–10.
Judge SJ, Richmond BJ, Chu FC. 1980. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20:535–538.[CrossRef][Web of Science][Medline]
Komatsu H, Wurtz RH. 1988a. Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J Neurophysiol 60:580–603.
Komatsu H, Wurtz RH. 1988b. Relation of cortical areas MT and MST to pursuit eye movements. III. Interaction with full-field visual stimulation. J Neurophysiol 60:621–644.
Komatsu H, Wurtz RH. 1989. Modulation of pursuit eye movements by stimulation of cortical areas MT and MST. J Neurophysiol 62:31–47.
Lagae L, Maes H, Raiguel S, Xiao DK, Orban GA. 1994. Responses of macaque STS neurons to optic flow components: a comparison of areas MT and MST. J Neurophysiol 71:1597–1626.
Lappe M, Bremmer F, Pekel M, Thiele A, Hoffmann KP. 1996. Optic flow processing in monkey STS: a theoretical and experimental approach. J Neurosci 16:6265–6285.
Maunsell JH. 1988. Representation of three-dimensional visual space in the cerebral cortex [review] [95 references]. Can J Physiol Pharmacol 66:478–487.[Web of Science][Medline]
McBeath MK, Shaffer DM, Kaiser MK. 1995. How baseball outfielders determine where to run to catch fly balls [see comments]. Science 268:569–573.
Merchant H, Battaglia-Mayer A, Georgakopoulos AP. 2004. Neural responses during interception of real and apparent circularly moving stimuli in motor cortex and area 7a. Cereb Cortex 14:314–331.
Merchant H, Battaglia-Mayer A, Georgopoulos AP. 2001. Effects of optic flow in motor cortex and area 7a. J Neurophysiol 86:1937–1954.
Mishkin M, Ungerleider LG, Macko KA. 1983. Object vision and spatial vision: two cortical pathways. Trends Neurosci S414–S417.
Motter BC, Steinmetz MA, Duffy CJ, Mountcastle VB. 1987. Functional properties of parietal visual neurons: mechanisms of directionality along a single axis. J Neurosci 7:154–176.[Abstract]
Oram MW, Perrett DI, Hietanen JK. 1993. Directional tuning of motion-sensitive cells in the anterior superior temporal polysensory area of the macaque. Exp Brain Res 97:274–294.[Web of Science][Medline]
Orban GA, Lagae L, Verri A, Raiguel S, Xiao D, Maes H, Torre V. 1992. First-order analysis of optical flow in monkey brain. Proc Natl Acad Sci USA 89:2595–2599.
Page WK, Duffy CJ. 1999. MST neuronal responses to heading direction during pursuit eye movements. J Neurophysiol 81:596–610.
Page WK, Duffy CJ. 2003. Heading representation in MST: sensory interactions and population encoding. J Neurophysiol 89:1994–2013.
Paolini M, Distler C, Bremmer F, Lappe M, Hoffmann KP. 2000. Responses to continuously changing optic flow in area MST. J Neurophysiol 84:730–743.
Petrides M, Pandya ND. 1984. Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. J Comp Neurol 228:105–116.[CrossRef][Web of Science][Medline]
Probst T, Brandt T, Degner D. 1986. Object-motion detection affected by concurrent self-motion perception: psychophysics of a new phenomenon. Behav Brain Res 22:1–11.[CrossRef][Web of Science][Medline]
Probst T, Krafczyk S, Brandt T, Wist ER. 1984. Interaction between perceived self-motion and object-motion impairs vehicle guidance. Science 225:536–538.
Raiguel S, Van Hulle MM, Xiao D, Marcar V, Lagae L, Orban GA. 1997. Size and shape of receptive fields in the medial superior temporal area (MST) of the macaque. Neuroreport 8:2803–2808.[Web of Science][Medline]
Recanzone GH, Wurtz RH, Schwarz U. 1997. Responses of MT and MST neurons to one and two moving objects in the receptive field. J Neurophysiol 78:2904–2915.
Robinson DA. 1963. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Biomed Eng 10:137–145.[Medline]
Royden CS. 2002. Computing heading in the presence of moving objects: a model that uses motion-opponent operators. Vision Res 42:3043–3058.[CrossRef][Web of Science][Medline]
Royden CS, Hildreth EC. 1996. Human heading judgments in the presence of moving objects. Percept Psychophys 58:836–856.[Web of Science][Medline]
Saito H. 1993. Hierarchial neural analysis of optical flow in the macaque visual pathway. In: Ono T, Squire LR, Raichle ME, Perrett D, Fukuda M, editors. Brain mechanisms of perception and memory: from neuron to behavior. New York: Oxford University Press. p 121–140.
Shenoy KV, Crowell JA, Andersen RA. 2002. Pursuit speed compensation in cortical area MSTd. J Neurophysiol 88:2630–2647.
Stanton GB, Bruce CJ, Goldberg ME. 1995. Topography of projections to posterior cortical areas from the macaque frontal eye fields. J Comp Neurol 353:291–305.[CrossRef][Web of Science][Medline]
Steinmetz MA, Motter BC, Duffy CJ, Mountcastle VB. 1987. Functional properties of parietal visual neurons: radial organization of directionalities within the visual field. J Neurosci 7:177–191.[Abstract]
Sugihara H, Murakami I, Shenoy KV, Andersen RA, Komatsu H. 2002. Response of MSTd neurons to simulated 3D orientation of rotating planes. J Neurophysiol 87:273–285.
Tanaka K, Fukuda Y, Saito H. 1989. Underlying mechanisms of the response specificity of expansion/contraction and rotation cells in the dorsal part of the medial superior temporal area of the macaque monkey. J Neurophysiol 62:642–656.
Tanaka K, Saito H. 1989. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J Neurophysiol 62:626–641.
Tanaka K, Sugita Y, Moriya M, Saito H. 1993. Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual cortex. J Neurophysiol 69:128–142.
Thier P, Erickson RG. 1992. Vestibular input to visual-tracking neurons in area MST of awake rhesus monkeys. Ann N Y Acad Sci 656:960–963.[Web of Science][Medline]
Tian JR, Lynch JC. 1996. Functionally defined smooth and saccadic eye movement subregions in the frontal eye field of cebus monkeys. J Neurophysiol 76:2740–2753.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Liu and D. E. Angelaki Vestibular Signals in Macaque Extrastriate Visual Cortex Are Functionally Appropriate for Heading Perception J. Neurosci., July 15, 2009; 29(28): 8936 - 8945. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. K. Page and C. J. Duffy Cortical Neuronal Responses to Optic Flow Are Shaped by Visual Strategies for Steering Cereb Cortex, April 1, 2008; 18(4): 727 - 739. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. A. Orban Higher Order Visual Processing in Macaque Extrastriate Cortex Physiol Rev, January 1, 2008; 88(1): 59 - 89. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Takahashi, Y. Gu, P. J. May, S. D. Newlands, G. C. DeAngelis, and D. E. Angelaki Multimodal Coding of Three-Dimensional Rotation and Translation in Area MSTd: Comparison of Visual and Vestibular Selectivity J. Neurosci., September 5, 2007; 27(36): 9742 - 9756. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. J. Dubin and C. J. Duffy Behavioral Influences on Cortical Neuronal Responses to Optic Flow Cereb Cortex, July 1, 2007; 17(7): 1722 - 1732. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||