Cerebral Cortex Advance Access originally published online on July 5, 2007
Cerebral Cortex 2008 18(3):705-717; doi:10.1093/cercor/bhm107
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Natural Vision Reveals Regional Specialization to Local Motion and to Contrast-Invariant, Global Flow in the Human Brain
1 Max Planck Institute for Biological Cybernetics, Department of Physiology of Cognitive Processes, 72076 Tübingen, Germany, 2 Laboratory of Neurobiology, Department of Anatomy, University College London, London WC1E 6BT, UK
Address correspondence to A. Bartels, Max Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076 Tübingen, Germany. Email: andreas.bartels{at}tuebingen.mpg.de.
 |
Abstract |
|---|
 TopNotes Abstract IntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Visual changes in feature movies, like in real-live, can be partitioned into global flow due to self/camera motion, local/differential flow due to object motion, and residuals, for example, due to illumination changes. We correlated these measures with brain responses of human volunteers viewing movies in an fMRI scanner. Early visual areas responded only to residual changes, thus lacking responses to equally large motion-induced changes, consistent with predictive coding. Motion activated V5+ (MT+), V3A, medial posterior parietal cortex (mPPC) and, weakly, lateral occipital cortex (LOC). V5+ responded to local/differential motion and depended on visual contrast, whereas mPPC responded to global flow spanning the whole visual field and was contrast independent. mPPC thus codes for flow compatible with unbiased heading estimation in natural scenes and for the comparison of visual flow with nonretinal, multimodal motion cues in it or downstream. mPPC was functionally connected to anterior portions of V5+, whereas laterally neighboring putative homologue of lateral intraparietal area (LIP) connected with frontal eye fields. Our results demonstrate a progression of selectivity from local and contrast-dependent motion processing in V5+ toward global and contrast-independent motion processing in mPPC. The function, connectivity, and anatomical neighborhood of mPPC imply several parallels to monkey ventral intraparietal area (VIP).
Key Words: contrast • heading • LIP • motion • natural scenes • objects • predictive coding • V5/MT • VIP
|
Introduction |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
There are several types of behaviorally relevant motion in our natural visual environment that are independent. Because of their independence, they need to be segregated by our visual system. These natural dynamics are also captured in feature movies. Even though movies may deviate in some aspects from our real-life visual input, they nevertheless constitute an excellent experimental approximation to it. Because motion processing has been studied in great detail in controlled settings, it seemed worthwhile to characterize at least some of its aspects also during processing of these more natural, complex scenes. The aim of the current study was thus 2-fold: firstly, we wanted to investigate cortical responses to objectively determined dynamic changes in freely viewed natural scenes, due to motion and other factors. Secondly, we wanted to learn whether different aspects of motion, namely simulated observer motion (resulting in global flow fields across the screen) and object motion (resulting in local or differential motion) could also be differentiated cortically (Gibson 1954
; Galletti and Fattori 2003). In contrast to many controlled settings, motion in real life tends to be an attribute of recognizable shapes, provides cues for figure/ground segregation and aides object recognition (Julesz 1971; Braddick 1974; Grossman and Blake 2002; Self and Zeki 2005). This raises the additional question whether motion during natural vision involves only motion-selective regions such as V5+/MT+ or extends to lateral occipital cortex (LOC) involved in object processing.
Motion can be coded in different frames of reference (retina-, world-, or self-centered). How and where what is done is still surprisingly unclear. V5 (MT) and V5A (MST) already code primarily for real (i.e., external object) compared with retinal (i.e., pursuit-induced) motion (Erickson and Thier 1991; Freitag et al. 1998; Thiele et al. 2002; Goltz et al. 2003; Goossens et al. 2006). MSTd responds to head-centered wide-field flow (Desimone and Ungerleider 1986; Tanaka et al. 1986; Duffy and Wurtz 1991; Erickson and Thier 1991; Graziano et al. 1994; Page and Duffy 1999; Thiele et al. 2002). Parietal cortex is less well studied but appears to code primarily in nonretinal coordinates and contains several regions coding for motion and flow: V6 (Galletti et al. 2001; Galletti and Fattori 2003), 7a (Zhang et al. 2004), ventral intraparietal area (VIP; Colby et al. 1993; Zhang et al. 2004) and PEc (Raffi et al. 2002; Merchant et al. 2003). VIP contains the best performing neurons yet characterized, coding for heading with behavioral precision, and in multiple modalities (Zhang et al. 2004; Schlack et al. 2005). Human imaging studies on flow have primarily focused on MST and revealed potential homologues of MSTd and MSTl (Morrone et al. 2000; Dukelow et al. 2001; Goossens et al. 2006; Smith et al. 2006). However, studies on flow going beyond V5+ found strong involvement of medial posterior parietal cortex (mPPC) (Haarmeier and Thier 1998; Peuskens et al. 2001; Tikhonov et al. 2004), in particular, differentiating self-motion from external visual motion in a magnetoencephalography study (Tikhonov et al. 2004). The latter task is selectively impaired in patients with posterior occipitoparietal lesions, demonstrating the existence of neural substrates specific for this task, as their destruction appeared to spare attention, eye movements, and motion processing (Haarmeier et al. 1997; Heide and Kompf 1998).
We were hopeful to be able to identify motion-responsive regions despite the uncontrolled nature of the stimuli, not least because regional responses and their connectivity are at least as specific during natural vision compared with traditional experiments (Bartels and Zeki 2003, 2004, 2005). Feature-driven analyses of natural viewing data revealed a functional segregation of color, faces, and human bodies that closely resembled that known from controlled experiments (Bartels and Zeki 2003). Additionally, blind decomposition techniques (independent component analysis) separated more areas even within the visual cortex in the context of natural viewing compared with controlled experiments due to both the richness of the stimulus and highly characteristic responses (temporal fingerprints) in distinct regions (Bartels and Zeki 2004). Presentation of movies also increased the specificity of functional connectivity (fc) compared with resting state conditions as it led to a decorrelation of cortical signal time courses with the exception of directly connected regions (Bartels and Zeki 2005). It is also worth noting that natural stimuli do not always replicate results obtained in artificial settings. For example, dynamic faces in movies activate large parts of temporal cortex not activated and thus barely studied due to the common use of static pictures in face studies (Bartels and Zeki 2003).
Unlike traditional stimuli, natural scenes are not balanced for visual contrast across space or time. This gave us the opportunity to examine in the same study which of the generally motion-responsive regions respond purely to velocity fields, to their contrast-weighted counterpart, or to contrast changes alone, which has not been studied before in physiology or fMRI. V5 and likely MST are modulated by luminance contrast, complicating estimates of speed or heading (Felleman and Kaas 1984; Sclar et al. 1990; Dobkins and Albright 1994; Tootell and others 1995; Riecansky et al. 2005). Our expectation was that higher order regions, such as those coding for heading, would be less influenced by low-level visual features, such as contrast. Such invariance would also facilitate cross-modal integration and be analogous to the increasing attribute and contrast invariance observed in high-level ventral visual areas coding for objects (Murray and He 2006).
We were thus particularly interested to see whether distinct brain regions responded to global and local motion during viewing of natural movie stimuli. In addition, we examined the contrast invariance of motion processing in distinct regions. Finally, we charted the functional connections of a parietal motion-responsive region and of its lateral neighbor.
|
Methods |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Natural Scene Analysis
The overview provides the necessary understanding of the measures we extracted from the movie stimulus in order to correlate them with blood oxygen level–dependent (BOLD) signals. Further sections provide more detailed descriptions.
Overview
One of the aims was to quantify the changes that account for the visual dynamics in a motion picture and to partition them into different components, namely two motion components (global and local motion) and a residual, nonmotion component - their sum would account for all changes in the movie. Each component consisted of a time course with one value per movie frame, and their hemodynamic response function (hrf)–convolved versions were used as regressors for the fMRI data analysis. We first quantified all our measures in terms of pixel luminance changes (but see below), averaged across the whole display for every movie frame. Luminance (Y) values of every pixel (320 x 240) and for every frame (20175 frames at 15 Hz = 22 min 25 sec) in the movie were calculated using standard methods (CIE XYZitu (D65) standard: Y = r*0.222015 + g*0.706655 + b*0.071330).
dTotal expresses the total amount of pixel-wise luminance change from frame to frame, that is, the absolute of the difference between two successive frames (see Fig. 1e), calculated for every pixel and then averaged, as follows.
 |
|
dTotal was partitioned into the three fractions of interest in this study, one accounting for global motion, one for local/differential motion, and one for residual changes. First, however, dTotal was split into two fractions, one that can be explained by motion (dMotion) and the remainder (dResidual) (Fig. 1f, g). dMotion are pixel changes that can be accounted for by any type of motion detected by our motion detection algorithm (see below). dResidual was the remainder with respect to dTotal and included changes related to illumination, the appearance of new objects, and scene cuts. dMotion was then partitioned into two fractions: one related to global motion flow spanning the whole scene (dMotGlobal), such as induced when the camera moves, pans across, or zooms into a scene (Fig. 1c), and one corresponding to local or differential motion that cannot be accounted for by global flow, that is, incoherent, spatially local, or motion deviating from global flow (dMotLocal) (e.g., see Fig. 2). Note that both global and local motion can coexist and that their sum (dMotGlobal + dMotLocal) equals dMotion for every frame.
|
The advantage of above measures is that all changes, whether related to motion or not, are expressed in the same (luminance) units and that they are thus directly related to visual salience. In order to differentiate between two factors contributing to this salience, namely "flow" (motion vector length, i.e., velocity) and the visual contrast of the underlying visual scene, we quantified two additional measures. Both are alternative measures for the total amount of motion in the scene (like dMotion) and do not differentiate between local or global motion: first, a measure for pure motion flow (the sum of all motion vector lengths of each frame). This measure (flow) would be important for any cortical mechanism estimating global flow fields as it is independent of image contrast that may vary locally across space and over time. Second, a measure for motion energy, that is, the lengths of the flow vectors weighted by the RMS (root mean square) contrasts of the underlying visual scene patches. This measure (flow*RMS contrast) is the contrast-dependent counterpart to pure flow. RMS contrast is a standard measure for visual contrast and defined (for each scene patch, or RF, see below) as std(luminance)/mean(luminance). See Fig. 6a, b for an illustration. The supplementary movie files in the Supplementary Material provide examples of movie segments containing high local or high global motion components, shown with superimposed flow fields.
Motion Flow Extraction
Motion flow fields were estimated using a patch-wise spatial translation method similar to that introduced by Bülthoff et al. (1989) and described as a perceptually plausible flow field estimator capable of reproducing several visual illusions. In our implementation, the visual field (26 x 19°) was divided into an array of M x N square "receptive fields" (RFs; we replicated all our analyses using several RF array resolutions: 10 x 8, 20 x 15, 40 x 30). Each RF of frame(n) was translated within a given radius until its best matching location in frame(n + 1) was found (see Fig. 1a), that is, the location where the difference (see formula for dTotal above) between it and its (translated) location in frame(n + 1) was minimal (=dResidualRF). The difference at its original location is dTotalRF. dMotionRF is then
 |
This patch-wise translation method thus yields a measure for motion in luminance units (dMotion) and also a motion flow field r of M x N motion vectors, that is, for every RF of frame(n) the x and y translation components that result in the best match in frame(n + 1). Computations were kept tractable by restricting the RF translation search space to 12 radial directions and 7 distances for each direction (1–24 pixels = 0.08–1.9° at 15 Hz, i.e., 1.2–29°/s) (see Fig. 1a, d). To reduce artifactual motion vectors, two threshold operations were applied (not present in Bülthoff and Poggio 1989): RF motion vectors in r were set to zero if 1) dMotionRF < threshold(1)*length(vectorRF), that is, if they did not account for a minimal amount of dY relative to their length. This lower limit on motion-related pixel changes avoided arbitrary motion vectors in (noisy) uniform scenes. 2) dResidualRF > threshold(2), that is, if a large amount of pixel change remained unexplained. This upper limit on residual pixel change avoided arbitrary motion vectors due to large nonmotion-related changes (e.g., newly appearing objects or illumination changes). A.B. determined thresholds for (1) and (2) empirically and found that values of 0.08% and 4% of average luminance change per pixel, respectively, reduced artifactual motion vectors efficiently. Finally, artifactual motion vectors at scene cuts were set to zero. They were identified by high-pass filtering the time series of dTotal with a cutoff of 2 Hz (9th-order Butterworth filter) and detecting peaks exceeding a threshold T—a T of 7.4% percent of average luminance change per pixel detected the 428 scene cuts also detected manually. In a test of dynamic noise (a series of frames each containing uniformly distributed noise), the algorithm attributed 0.0% of change to motion (100% to residual changes). With random sequences of frames from the original movie (thus preserving all image statistics except for motion continuity), 10.2% of change was attributed to motion (of which 91% was local motion because it was entirely random). When a large noise field moved continuously from left to right, 100% of change was attributed to motion (100% global motion).
Motion Partitioning into Global and Local Components
To partition motion into fractions related to global and local/differential motion, respectively, we employed a simple 2-step procedure applied to the real flow field (r) of every frame that was determined as described above. The 1st step identified an "ideal" uniform global flow field spanning the whole display (g) (e.g., leftward motion within all RFs) that best matched the actual flow field (r) of this frame, the 2nd step partitioned every RF motion vector of r into fractions accounted for by the "ideal" uniform global field and the remainder—the latter thus constituting local (or differential) motion. According to the relative size of both fractions, dMotion was partitioned into dMotGlobal and dMotLocal.
For step 1, a set of "ideal" uniform global flow fields was generated (124 g's in total: translation fields for 24 directions, 50 centrifugal/centripetal fields [25 evenly spaced points of origin, fugal or petal] and equally many left/right rotational fields; see Fig. 1c). After normalizing the mean vector lengths of both real and ideal fields to one, the (signed) projection lengths of vectors from each g onto r were calculated (Fig. 1b) as follows:
 |
Pgr was then weighted by the pixel-wise changes accounted for by motion of the corresponding RFs (dMotionRF) and summed across all RFs to get a score (dMotGlobal = Pgr*dMotionRF). The g with the highest score was the best matching ideal flow field used for Step 2. (Note: This computationally efficient procedure turned out equivalent to a regression model fitting real flow with a weighted sum of ideal translation, expansion, and rotation fields. The latter was computationally very expensive because it had to be done for each triple of every possible combination of points-of-origin and direction and yielded measures for global and local motion that correlated with r = 0.996 and r = 0.999 with that of the 1st measure described). Step 2: For each RF, dMotionRF was partitioned into a global and a local fraction according to the relative projection length of the global vector onto the actual one (Pgr) and its remainder (Pdr) (Fig. 1b), that is,
 |
This way, we obtained measures for the global and local/differential motion components that together sum up to dMotion, all expressed in terms of the pixel-wise changes they account for in the movie display.
Stimuli and subjects
Informed consent was obtained from all subjects in accordance with the Declaration of Helsinki, and ethical approval was granted by the Ethics Committee of the National Hospital for Neurology and Neurosurgery, London, UK. Eight volunteers (5 female, all right handed, between the age of 24–38 years) viewed the 1st 22 min 25 s of the James Bond movie Tomorrow Never Dies (including the sound track) while BOLD activity was measured using fMRI (Bartels and Zeki 2003, 2004). The movie was viewed through an angled mirror on a translucent screen of 26 x 19° visual angle. It was interrupted every 2.5 or 3 min with a blank period (black screen and no sound) lasting 30 s, in total for 8 times, and the image was switched between achromatic and colored every 30 s, which was, however, barely noticed by the subjects and is not relevant to this study (Bartels and Zeki 2003). All analyses reported here are based only on the movie periods (excluding the 30-s blanks and the 15 s following blank–movie transitions, as the latter lead to nonspecific effects in activation as well as functional connectivity [Bartels and Zeki 2005]).
Acquisition of fMRI Data
T2*-weighted whole-brain images (3 x 3 x 3 mm resolution; 48 slices, 1.8 mm thick with 1.2 mm gap, with a matrix of 64 x 64 pixels) were acquired in a Siemens Vision 2 Tesla MRI scanner, using an echo planar imaging sequence that optimized BOLD contrast, reflecting neural activity of different kinds (Logothetis et al. 2001). Echo time was 40 ms. Data from subjects 1–4 were acquired using a repetition time (TR) of 4.105 s with 324 whole-brain acquisitions in 22 min 12 s; subjects 5–8 were acquired with a TR of 3.649 s and 368 acquisitions in 22 min 23 s.
Data analysis
All data were processed using SPM99 (Friston et al. 1995) (http://www.fil.ion.ucl.ac.uk/spm/) as follows. Whole-brain images were realigned to compensate for head movement, and slices were temporally realigned to compensate for acquisition time lags. Images were spatially normalized to the Montreal Neurological Institute template (approximating to the atlas of Talairach and Tournoux (1988)) and spatially smoothed with a Gaussian kernel of 10 mm full width at half maximum for group analyses and with 6 mm for single-subject analyses. For the GLM analysis, data were high-pass filtered using a 512-s cutoff, and subject-specific realignment parameters were included as regressors of no interest. The computer-derived feature regressors (15 Hz) were convolved with SPM99's canonical hrf and normalized to a range of one and a mean of zero before inclusion as regressors. Data were analyzed in both fixed-effects group and single-subject analyses for display and (2nd level) random effects (RFX) analyses (t-tests on contrast images, reported for regions of interest [ROIs]). To demonstrate the replicability of our results, we repeated analyses for the 1st and 2nd halves of the movie data separately. Results of split analyses are reported in the form of true conjunctions where only voxels survived that passed the given P value in each of the split analyses independently (Nichols et al. 2005). All results reported here were thus replicated in the full-movie analysis as well as in separate analyses of two halves of the movie and, despite the small number of subjects, also achieved significance in 2nd level (RFX) analyses reported for ROIs. Results are shown for the feature measures of the 20 x 15 RF motion extraction and were near identical for the 10 x 8 or 40 x 30 RF analyses. Group results are reported for full-movie analyses with P < 0.05 FWE correction, conjunctions of 1st and 2nd halves for P < 0.05 FDR correction and single-subject examples are shown at P < 0.005 uncorrected; all types of analyses are shown in each figure (Figs 4, 5 and 7).
Movie Statistics
In absolute terms, frame-to-frame pixel-wise luminance changes (i.e., dTotal) were small: the median dTotal per frame was 2.1 luminance units (lower/upper quartiles: 1.0, 4.8; with a luminance range of 0–100). Scaled by each frame's mean or SD of luminance, this amounted to a median of 12.3% or 15.4% luminance change per frame, respectively (quartiles: 6.7, 24.0% and 7.8 or 29.7%). For the midresolution 20 x 15 grid, 73 ± 18% (mean ± SD) of these changes were accounted for by residual factors (e.g., illumination changes), 27 ± 18% by motion, and 12 ± 14% and 16 ± 11% by its global and local fractions, respectively (Fig. 3a). Of changes due to global motion, radial flow as well as translatory motion accounted for most pixel changes (together for over 90%), while rotary motion was negligible (see Fig. 3e). Note that the radial fraction may be slightly overrepresented here due to uneven contrast across the scene, biasing the best matching global field to radial instead of translatory fields. Translatory motion, radial flow, and rotary motion were correlated with r = 0.97, which is why we pooled them into a single measure of "global" motion for further fMRI analyses. Increasing grid resolutions increased dMotion, from 23% (10 x 8) to 33% (40 x 30). This was due to an increase of local motion, whereas global motion was virtually unaffected. Grid resolutions affected mainly the scaling or offset of these measures, thus preserving much of their temporal change (and only this is of interest in correlational fMRI analyses): mean correlations for a given measure, averaged across 10 x 8, 20 x 15 and 40 x 30 resolutions were in average: r = 0.95 before and 0.97 after hrf convolution. Correlations between different features are shown in Table 1. Center and periphery had only minor differences: RFs within the outer quartile of the screen radius had a median of 92% of total pixel changes (quartiles: 57% and 140%) and 83% of motion-related changes (quartiles: 33% and 206%) relative to the central quartile of the screen (100%). Finally, we found a directional tuning of motion vectors for the cardinal, especially horizontal, directions when summed pixel changes (or vector lengths, not shown) were plotted against direction (Fig. 3b). This was due to a higher number of motion vectors in cardinal directions; when directional pixel changes were normalized by their number of occurrence, the direction tuning disappeared (Fig. 3d). The direction bias was mainly due to global translation flow fields induced by horizontal (and some vertical) camera panning (Fig. 3c).
|
|
|
Results |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Responses to Motion during Free Viewing of a Movie
Our first aim was to determine regions involved in motion processing during natural vision and to test their replicability in different movie segments. We thus used a GLM model that included dMotion and dResidual as regressors, and analyzed 1st and 2nd halves of the movie separately. The regressors accounted for changes due to motion or residual factors, respectively. The results demonstrate both feature selectivity and replicability across the two movie halves. Response maps obtained from each half as well as their conjunctions are shown in Figure 4 for both features. Motion correlated specifically with V5+ (MT/MST), closely followed in intensity by medial occipitoparietal cortex ranging ventrally from V3A and V7 and extending dorsally to medial posterior parietal cortex (mPPC).
|
Weaker but consistent motion responses were also observed in ventral lateral occipital cortex (LOv), otherwise primarily involved in object recognition (P < 0.003, RFX 16 hemispheres, 0.5% BOLD signal modulation). With coordinates (±42, –70, –20), it was located 2–3 cm posterior to face- and body-selective activity of our previous high-level feature analysis of these data located at (±44, –46, –22) (Bartels and Zeki 2003
). To exclude the possibility that this activation was due to a coincidental correlation of dMotion with the presence of faces or human bodies, we ran another GLM with face and body regressors included. The same (and sole) region emerged in LOv for dMotion, surviving P < 0.001 in each movie half. Lastly, dMotion also elicited responses in ventromedial cortex (likely corresponding to ventral V3, see Table 2). In contrast to other regions, this region responded equally to dMotion and dResidual. Near-identical results were obtained when different grid resolutions for motion extraction were used or when dMotion was replaced by other measures of motion such as flow or motion energy (i.e., flow*(RMS contrast)).
|
dResidual, a measure for time-varying contrast and luminance changes unrelated to motion, correlated with BOLD signal in occipital and ventral visual cortex corresponding to visual regions V1–V4 (Fig. 4c). Interestingly, however, these regions were not modulated by motion-induced luminance changes. The occipital poles, which contain the foveal confluence (
central 0–2°) of early visual areas V1–V3, were negatively correlated with motion intensity (Fig. 4b). This was restricted to the poles: even at very low thresholds (P < 0.05 uncorrected) there was neither positive nor negative correlation with dMotion in early visual cortices. For further analyses, we identified motion-responsive ROIs, defined as regional peaks of V5+, the most inferior (V3A/V7) and superior (mPPC) peaks in medial parietal cortex, and the occipital poles, indicated by arrows in Figure 4a. Figure 4d shows the parameter estimates (estimated %BOLD signal change) for dResidual and dMotion of the ROIs, illustrating significance at the RFX level (P < 0.0001, n = 16 hemispheres). Figure 4e plots BOLD responses against feature intensities of dMotion and dResidual, revealing linear responses to motion intensity in V5+ and other ROIs, and Figure 4d shows single-subject examples for each of the contrasts. The x, y, z peak coordinates of the motion-responsive ROIs as well as additional regions are given in Table 2.
Global and Local/Differential Motion Reveal Regional Specificity
We next tested whether global and local motion elicited segregated responses. A new GLM was constructed containing three feature regressors (dResidual, dMotGlobal and dMotLocal) that accounted for residual changes, global motion, and local motion, respectively. Note that the latter two sum up to dMotion, and would thus be—in some combination—expected to account for activity revealed in the previous analysis but potentially also for additional regions. Figure 5a shows, firstly, that the same regions accounted for by dMotion were also accounted for by either dMotGlobal or dMotLocal but without involving further regions. Secondly, a clear segregation of the two is apparent, without much overlap: mPPC correlated selectively with global motion, whereas V5+ correlated selectively with local/differential motion. Inferior medial parietal cortex (containing V3A and V7) showed a significant preference for local motion but was laterally surrounded by activity responsive to global motion, spreading ventrally from mPPC, perhaps involving the more laterally located V3B (see inset with reduced thresholds). Results are shown for the full-movie analysis at P < 0.05 (FWE corrected), which looked (like in the previous analysis) identical for the conjunction of 1st/2nd movie halves at P < 0.001 (uncorrected, not shown), demonstrating their replicability across different movie halves. The peaks for global motion were slightly dorsolateral of our independently selected motion-responsive mPPC ROI (right: 14, –74, 48; left: –12, –74, 50).
|
Figure 5b shows estimated %BOLD signal change at RFX level for the independently selected ROIs determined in the previous analysis. V5+ and V3A/V7 responded selectively to local motion, whereas mPPC did so to global motion; the foveal confluence in the occipital poles was negatively correlated with local motion only. The more peripheral representations of V1 in the calcarine sulcus were not involved in either global or local motion. Each ROI responded to its preferred feature with P < 0.0001 at the RFX level, n = 16 hemispheres. A direct comparison of global versus local (and vice versa) confirmed the above preferences for the respective regions with P < 0.002 (RFX, n = 16; mPPC reached P < 0.02; when one outlier subject was omitted, this increased to P < 0.003).
Contrast Invariance of Global Flow–Processing Region mPPC
Finally, we exploited the fact that movie stimuli were not balanced for visual contrast across space or time. This allowed us to test to which extent motion-responsive regions responded to flow velocity, velocity weighted by visual contrast (motion energy), or visual contrast alone, which has not been done before. Figure 6a and b illustrates velocity flow and motion energy on the example of a frame dominated by global motion—flow alone provides a better estimate for heading. We expected that global motion–selective region mPPC may be selective for pure motion flow, whereas V5+ would be biased by underlying visual contrast.
|
We used a new GLM containing flow, RMS contrast and flow*(RMS contrast) as regressors, allowing us to determine responses to flow, contrast and their interaction in our ROIs. mPPC responded selectively to flow but not to flow*RMS contrast, whereas V5+ was selective for flow*(RMS contrast) but not for flow (Fig. 6c). Statistics: response to flow: mPPC, P < 0.006; V5+, n.s.; response to flow*RMS contrast: V5+, P < 0.00006; mPPC, n.s. Contrast flow > flow*RMS: mPPC, P < 0.05; contrast flow*RMS > flow: V5+, P < 0.008; RFX level for n = 16 hemispheres). V3A/V7 responded to both flow and flow*RMS (significant only to the latter).
Independent and Simultaneous Coding of Flow and Global/Local Motion in mPPC and V5+
Note that the measures flow and flow*RMS contrast were independent of those for global and local motion. We wanted to test whether mPPC (global motion and flow responsive) and V5+ (local motion and flow*RMS contrast responsive) coded for each of the measures in addition and independently, that is, whether additional variance in their response was explained by inclusion of the respective other regressor. We thus constructed a GLM with the above-mentioned five regressors but orthogonalized (serially) the last two regressors (flow and flow*RMS contrast) to all preceding ones, just as done for hierarchical forward model selection (Buchel et al. 1998). (We ran this twice, once with flow and once with flow*RMS contrast as last regressor). Thus, any significance in a t-test for flow (orthog) or flow*RMS contrast (orthog) would imply that measures of flow are coded in addition to global/local motion in these regions. Indeed, the results (reported for the weaker statistics of the two GLM permutations) confirmed that V5+ (P < 0.0005), V3A/V7 (P < 0.002) and occipital poles (negatively, P < 0.002) coded independently of all other regressors for flow*RMS contrast (orthog) (RFX level, n = 16 hemispheres), whereas mPPC coded independently of all other regressors for flow (orthog) (P < 0.02). The same held true when dMotGlobal and dMotLocal were orthogonalized to all other regressors. Equally, the specific preferences of each ROI were preserved when all 5 regressors were included in their original form in a GLM, with the same significances as in the separate 3-regressor analyses.
Thus, V5+ coded independently for flow*RMS contrast and for local motion, and mPPC coded independently for global motion and pure (velocity defined) flow. Both are thus sensitive to 2 independent properties of motion in the visual field.
Functional connectivity of Medial and Lateral PPC: Relation to VIP and LIP?
mPPC was located medial to the parietal region of the so-called attention network. The latter consists of a parietal region and frontal eye fields (FEFs) and is invariably activated in studies on eye movements and attentional tasks but was not apparent in our analyses (Corbetta et al. 1998; Culham et al. 1998; Petit and Haxby 1999; Perry and Zeki 2000; Astafiev et al. 2003; Koyama et al. 2004). Monkey parietal cortex has a similar organization, in that the flow-responsive VIP is located directly medial to attention/eye movement–related lateral intraparietal area (LIP). Monkey LIP can be identified anatomically by its strong connections with FEFs (Maunsell and Van Essen 1983) and VIP by its strong connections with V5+ (MT+) (Andersen et al. 1985). We were thus curious to see whether the connectivity of mPPC and of the location lateral to it, proposed as a putative human homologue of LIP (hLIP), may reveal similarities to that of monkey VIP and LIP. We thus ran a GLM with both—BOLD time courses extracted from the global motion–responsive mPPC and hLIP (see Fig. 7). This would reveal fc maps that are specific for each region and likely reflect their anatomical connectivity (Bartels and Zeki 2005). For hLIP, we used 5 mm sphere ROIs around coordinates of Schluppeck et al. (2005) who report the most medial coordinates (and thus closest to mPPC) for the parietal eye fields/hLIP in the literature. Near identical results were obtained for more lateral coordinates in intraparietal sulcus (IPS), for example, taken from Sereno (2001) or Perry and Zeki (2000). The fc maps obtained from hLIP and mPPC were compatible with the differential connectivity from monkey LIP and VIP, respectively: putative hLIP correlated with FEFs as well as prefrontal regions involved in attentional allocation (Maunsell and Van Essen 1983) and also revealed correlations with the object-processing LOC. Global motion–responsive region mPPC, in contrast, correlated specifically with V5+, likely involving MSTd, as well as with regions anterior to it, reflecting the connectivity of monkey VIP (Andersen et al. 1985). These results suggest that global motion–selective mPPC is related in function and in connectivity to a VIP-like region, whereas regions lateral to it to LIP-like regions.
|
|
Discussion |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
We determined brain responses in observers viewing a natural movie stimulus and related them to image changes related to simulated self-movement (global motion) or to object movement (local or differential motion) or due to residual factors. Visual areas V1–V3 and V4 in the fusiform gyrus responded only to nonmotion-related changes in the movie, such as changing illumination, newly appearing objects, or, as demonstrated previously for V4, color (Bartels and Zeki 2003
). Stimulus motion intensity correlated most with V5+, followed by V3A/V7 and medial posterior parietal cortex (mPPC), and also involved a subdivision of the object-processing LOC. Responses to global and local motion were spatially segregated: mPPC, located medial to the putative human hLIP, responded to global motion and was invariant to visual contrast, whereas V5+ (hMT+) responded to local motion in a contrast-dependent manner. A fc analysis of the two directly neighboring regions in parietal cortex mPPC (medial) and hLIP (lateral to mPPC) revealed a connectivity pattern similar to that of monkey VIP and LIP, respectively: mPPC correlated with anterior portions of V5+ (likely hMSTd), whereas hLIP correlated with FEFs. Thus, mPPC shows parallels in function, relative anatomical location, and connectivity to macaque VIP. Predictive Coding and Motion Responses in LOC
Among all quantified luminance changes, only the nonpredictable fraction, that is, residual luminance changes, correlated with activity in early visual areas. The predictable fraction, that is, motion-related changes, did not. Therefore, there was a considerable lack of expected activation in early visual cortex, specific to motion-induced changes. It may be that subjects' eye movements may have compensated for some motion-induced luminance changes, for example, the foveal anticorrelation with motion may result from increased foveal locking of tracked items, where tracking would reduce foveal and enhance (yet in a predictable fashion) peripheral stimulation. However, for the overall stimulus, visual tracking would at most compensate for 20% of motion (namely, global translation) and can thus hardly account for the observed lack of activity in early visual cortex. This finding is consistent with a predictive coding account of visual processing, where predicted low-level changes are "subtracted out" in early visual areas through feedback from higher processing regions (Rao and Ballard 1999). Some of the few controlled motion studies that do comment on V1 responses also report compatible findings, namely higher responses to incoherent than to coherent motion in V1, that is, higher responses to less well-predictable motion (e.g., McKeefry et al. 1997; Braddick et al. 2001). A similar observation has been made for object processing, where recognizable objects led to reduced activity in V1 compared with scrambled counterparts matched for low-level features (Murray et al. 2002). The activation of V4 is in accord with its responses to color intensity in the movie (Bartels and Zeki 2003) and to dynamic illumination changes of static scenes, even if they are achromatic (Bartels and Zeki 2000). Thus, V1–V3 and V4 respond to luminance changes in movies as long as these are not accounted for by motion.
The only exception to classic motion–processing regions activated by motion was a small region in the LOv, otherwise known to be involved in object processing (Grill-Spector et al. 1998). Its response persisted across separate halves of the movie, also in the presence of face and body regressors that correlated with activity 2–3 cm anterior to it (Bartels and Zeki 2003). This finding highlights the tight link between shape and motion in natural conditions, and it is the counterpart to previous studies showing shape-dependent responses in V5+ (Movshon et al. 1985; Stoner and Albright 1992; Huk and Heeger 2002; Kourtzi et al. 2002). It may reveal neural substrates underlying improved recognition of moving objects (Julesz 1971; Braddick 1974; Grossman and Blake 2002; Self and Zeki 2005).
Motion Processing: Real and Retinal Motion
Our analysis here concerned objective stimulus motion, that is, in world/head-centered coordinates. The incessant motion due to eye movements is neither perceived nor behaviorally relevant, for separate reasons: saccades suppress the fast motion–sensitive M-system as early as in LGN, allowing only static percepts mediated through the P-system to arise (Burr et al. 1999; Thiele et al. 2002; Galletti and Fattori 2003; Sylvester and Rees 2006). In contrast, objective stimulus motion (real motion) is perceived as motion and elicits responses in motion-sensitive neurons, also when objects are visually pursued and their image is fixed on the retina: an increasing fraction of such real motion cells respond to real motion rather than to eye movement–induced retinal motion, starting with 10–15% of cells in V1 and V2, increasing to 41% in V3A, with more in V5 and the vast majority in V5A/MST (see for review, Galletti and Fattori 2003). In humans, V5+ responses are enhanced during tracking (minimal retinal flow) compared with central fixation (maximum retinal flow) of a moving random dots stimulus (Freitag et al. 1998); during pursuit, retinally fixed afterimages activated V5+ compared with fixation (Goltz et al. 2003); finally, with equated retinal flow, V5 as well as V5A/MST responses were enhanced during pursuit compared with fixation (Komatsu and Wurtz 1988; Shenoy et al. 1999; Goossens et al. 2006). The predominant response to real rather than retinal motion in V5+ likely explains why our measure for real motion in the display correlated so specifically with regions also evidenced in prior, controlled studies. Unfortunately, we do not have precise eye-tracking data of our subjects. Eye movements would have induced an additional component of global retinal flow on the retina. Although the above literature suggests that most of the mid- and higher level motion-processing regions (i.e., V5+ and mPPC) would correlate predominantly with measures of objective stimulus motion (as done here) rather than with measures of retinal motion, eye-tracking data would have allowed us to quantify their relative contributions for distinct regions. Note that our measure for local motion would not have been affected by taking into account measures of retinal motion because eye movements would have added only a global motion component. For the detection of retinal-flow–responsive regions, any type of eye movement would have been like added noise to our objective measure of flow and thus either weakened or more likely prevented the identification of mPPC if it was predominantly responsive to retinal rather than stimulus motion. We thus believe that mPPC predominantly responded to stimulus flow. Patient studies indicate the presence of a system specifically comparing visual flow with self-induced motion because this is specifically impaired with lesions in parieto-occipital regions, yet sparing motion processing or spatial attention tasks (Haarmeier et al. 1997; Heide and Kompf 1998). mPPC may well be a candidate for this as it responds to objective stimulus flow and with this to flow deviating from that self-induced by, for example, eye movements.
Global Flow and Heading Estimation in MSTd and VIP
Most neurophysiological studies on global flow and heading processing have concentrated on MSTd (Saito et al. 1986; Newsome et al. 1988; Duffy and Wurtz 1991). The same is true for human fMRI studies, which confirmed several properties of monkey MSTd in human MST (V5A): ipsilateral response to motion (Dukelow et al. 2001; Huk et al. 2002), preference to global flow (Morrone et al. 2000; Smith et al. 2006), and use of extraretinal signals for pursuit compensation (Goossens et al. 2006). However, MST is just one stage in heading processing. Only a fraction of MSTd neurons adjust, and then only partially, to the tuning of the focus of expansion to compensate for eye movements (Bradley et al. 1996). Many MSTd neurons have conflicting tuning properties with regard to vestibular signals (Gu et al. 2006) and undergo considerable gain changes during pursuit (Komatsu and Wurtz 1988; Shenoy et al. 1999), also observed in human fMRI (Freitag et al. 1998; Goossens et al. 2006). Information from about 150 MSTd neurons needs to be integrated to achieve heading estimates matching the behavioral level, which is achieved by single neurons in VIP (Ben Hamed et al. 2003; Zhang et al. 2004). The few human studies reporting whole-brain activity on heading revealed pronounced responses in medial parietal regions, consistent with our global flow–responsive mPPC (Peuskens et al. 2001; Tikhonov et al. 2004).
VIP and LIP
In the macaque, two regions in parietal cortex are involved in processing of space around us and are of particular interest here. They have distinct functions: LIP in the IPS is involved in control of eye movements and tightly linked to attention, saliency, and decision making (Duhamel et al. 1992; Gottlieb et al. 1998; Dorris and Glimcher 2004). It has been defined as the region in IPS with the strongest connections with FEFs (Andersen et al. 1985). LIP and FEFs form the "attention network," invariably activated in studies on saccadic eye movements, visual tracking, or attention shifts (covert or overt) in both monkeys and humans (Corbetta et al. 1998; Culham et al. 1998; Petit and Haxby 1999; Perry and Zeki 2000; Sereno et al. 2001; Astafiev et al. 2003; Koyama et al. 2004; Schluppeck et al. 2005, 2006). Despite our expectations, this network was not significantly activated with motion of either type. It is possible that allocation of attention (covert or overt) was driven by many factors (such as motion, but also presence and type of objects and actions), thus preventing a significant correlation with motion alone.
VIP, located medial to LIP, contains the best coding neurons for global flow and heading, some reaching behavioral performance levels (Schaafsma and Duysens 1996; Zhang et al. 2004). It is anatomically closely linked with motion-processing regions V5+/MT+ and was originally defined as the "MT projection zone" (Maunsell and Van Essen 1983). Interestingly, it combines heading information of different modalities (Duhamel et al. 1998; Schlack et al. 2005). On this basis, a human homologue of VIP has been proposed previously (Bremmer et al. 2001). However, the lack of controls for motion (i.e., no stimulation in 2 of 3 modalities), the type of conjunction analysis (allowing regions active in a single modality alone to pass [Nichols et al. 2005]), and the anatomical position leave it open whether that study revealed the parietal part of the attention network (that is apparent including FEFs) evoked by visual responses or a multimodal motion-selective region. The location of our flow-responsive region in mPPC directly medial to that of putative hLIP suggested a potential functional as well as anatomical parallel between humans and monkey. In addition, our fc analysis pointed in the same direction: global motion–responsive mPPC correlated with anterior V5+ (presumably mainly V5A/MST [Dukelow et al. 2001; Huk et al. 2002; Goossens et al. 2006; Smith et al. 2006]), whereas hLIP correlated with FEFs (Corbetta et al. 1998; Koyama et al. 2004). This reflects directly the pattern of anatomical connectivity of VIP and LIP in the monkey (Maunsell and Van Essen 1983; Andersen et al. 1985).
Apparently, thus, the region identified here as responsive to global motion flow matches macaque VIP in several properties: firstly, it is invariant to low-level visual properties such as visual contrast, and is thus compatible with a high-level representation of flow, and also for flow integration across multiple modalities (Duhamel et al. 1998; Schlack et al. 2005). It will be interesting to see which regions in monkey will reveal contrast-invariant flow processing. Secondly, mPPC is located adjacent and medial to a region involved in allocating spatial attention, hLIP (Corbetta et al. 1998; Petit and Haxby 1999; Perry and Zeki 2000; Sereno et al. 2001; Astafiev et al. 2003; Schluppeck et al. 2005, 2006). Thirdly, it is functionally strongly connected to the motion-processing region V5+, whereas its lateral neighbor is connected to FEFs and prefrontal regions. We thus refer to mPPC as to a VIP-like region, noting, however, that human parietal cortex may contain several, yet functionally related, subdivisions with VIP or LIP-like properties, respectively. For example, it is open whether mPPC or another, VIP-like region integrates cross-modal flow information. We also note that several other regions in monkey code for head-centered global flow, among them 7a (Siegel and Read 1997; Phinney and Siegel 2000; Merchant et al. 2003), PEc (Raffi et al. 2002) and V6 (Galletti et al. 2001; Galletti and Fattori 2003). Finally, we note that flow processing, spatial updating and salience mapping may in many situations be related and thus coactivate various regions, as observed in some previous studies (Culham et al. 1998).
The method we have used, that of freely viewing movies that contain many measurable and independently varying attributes, may be a promising way of learning how the brain segregates the processing of a variety of different attributes and determining the fc of the areas and networks involved.
The evidence obtained in this study suggests that we have identified a human flow-processing region in mPPC that shares similarities with macaque VIP and that we differentiated it from putative hLIP, or one of its satellites, based on function, relative location, and connectivity.
|
Supplementary Material |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.
|
Funding |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
This work was supported by the Swiss National Science Foundation, the Wellcome Trust, London, and the Max Planck Society, Germany.
|
Notes |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Funding to pay the Open Access publication charges for this article was provided by the Max Planck Society, Germany.
|
Acknowledgments |
|---|
Conflict of Interest: None declared.
|
References |
|---|
|
TopNotesAbstractIntroductionMethodsResultsDiscussionSupplementary MaterialFundingReferences |
|---|
Andersen RA, Asanuma C, Cowan WM. Callosal and prefrontal associational projecting cell populations in area 7A of the macaque monkey: a study using retrogradely transported fluorescent dyes. J Comp Neurol (1985) 232:443–455.[CrossRef]