Cerebral Cortex Advance Access originally published online on September 8, 2005
Cerebral Cortex 2006 16(6):896-906; doi:10.1093/cercor/bhj033
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Boundary Cue Invariance in Cortical Orientation Maps
McGill Vision Research Unit, Department of Ophthalmology, McGill University, Montreal, QC, Canada H3A 1A1
Address correspondence to Chang'an A. Zhan, McGill Vision Research Unit, Department of Ophthalmology, McGill University, 687 Pine Avenue West, H4-14, Montreal, QC, Canada H3A 1A1. Email: changan.zhan{at}mcgill.ca.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
We effortlessly perceive oriented boundaries defined by either luminance changes (‘first-order’ cues) or texture variations (‘second-order’ cues). Many neurons in mammalian visual cortex show orientation preference to both types of boundaries, but it is uncertain how they contribute to perceptual orientation cue-invariance at the neuronal population level. Using optical imaging in cat A18, we observed highly similar orientation preference maps to first-order and a variety of second-order visual stimuli. Thus the neuronal representation of coarse-scale boundary orientation appears to be invariant to the characteristics (including local orientation) of the fine-scale textures by which those boundaries are defined. A common feature of second-order visual stimuli is that modulation shifts their Fourier energy for boundary orientation to the higher spatial frequencies of their constituent textures — our results suggest a common neural mechanism (demodulation) mediating visual processing of many kinds of texture boundary. The similarity between orientation maps to different stimuli implies that second-order responsive neurons are homogeneously distributed across the cortical surface. Such homogeneously cue-invariant orientation representation could provide a neural substrate for perceptual form-cue invariance, and reflect an optimal organization for encoding orientation information in natural scenes.
Key Words: form-cue invariance • illusory contours • non-Fourier • optical imaging • second-order
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Object boundaries in the natural world are delineated by variations of local properties such as luminance or texture. Figure 1 abstracts some basic cues defining object boundaries, all illustrated here as periodic vertical contours. The sinewave grating in Figure 1A captures the common situation of luminance change along an object boundary. Figure 1B–D depicts three examples of ‘contrast contours’, vertically oriented boundaries induced by variations of texture contrast. The same orientation can also be perceived when rendered by phase transitions of textures, due to factors such as occlusion, as in Figure 1E,F (‘phase contours’). Because the sharply delineated vertical contours we perceive in Figure 1E,F are not present in the stimulus as extended physical luminance boundaries, these stimuli have also been termed ‘illusory contours’ (von der Heydt et al., 1984
; Peterhans and von der Heydt, 1989; Grosof et al., 1993; Ramsden et al., 2001) or ‘subjective contours’(Sheth et al., 1996). The ability of our visual system to extract all these contour orientations regardless of their defining cues may importantly contribute to perceptual form-cue-invariance (Albright, 1992) and thus could be critical precursors for figure–ground segregation and object perception.
|
A key difference between the above visual stimuli is how they activate orientation-selective receptive fields (superimposed ovals in Fig. 1) of early visual cortical neurons (Hubel and Wiesel, 1965
), whose orientation selectivity for local boundaries is thought to contribute to object perception in later visual cortex. Conventional, linearly summating receptive fields (Adelson and Bergen, 1985; Carandini et al., 1997) would respond well to luminance-defined (‘first-order’) contours (Fig. 1A). In the Fourier domain (De Valois et al., 1979; Watson and Ahumada, 1983), first-order energy exists within the neurons' spatial frequency passband (Fig. 1G, bright dots within the circle). However, second-order stimuli whose contour orientations are defined by transitions of texture properties (contrast or phase, Fig. 1B–F) do not provide any net luminance change within such a neuron's receptive field due to the local luminance cancellation. The Fourier energy of these stimuli (Fig. 1H–L) is distributed around the higher spatial frequency of their textures, which is beyond the neuron's spatial frequency passband (outside the circle). This change in the Fourier energy distribution arises from the contrast or phase modulation (Daugman and Downing, 1995) of the fine-grain textures (‘carriers’) by a low-spatial frequency ‘envelope’ (periodic vertical grating). Based on this fundamental difference, stimuli such as in Figure 1A, which provide a net luminance input to a neuron's (linear) receptive field, are considered ‘first-order’; an orientation-selective response to the other (‘second-order’) stimuli would require a spatially nonlinear operation.
Human psychophysical and animal single-unit electrophysiological studies have indicated early visual cortical processing of these stimuli (Chubb and Sperling, 1988; Chubb and Landy, 1991; Wilson et al., 1992; Baker, 1999). Human psychophysical studies also suggested two distinct mechanisms for first- and second-order visual processing (Smith and Ledgeway, 1997; McGraw et al., 1999; Kingdom et al., 2003). Electrophysiology supported this idea by showing that some early visual cortex neurons responded to illusory contours (von der Heydt et al., 1984; Grosof et al., 1993; Sheth et al., 1996; Ramsden et al., 2001) or contrast contours (Zhou and Baker, 1993, 1996; Leventhal et al., 1998; Mareschal and Baker, 1998, 1999). Furthermore, these studies also demonstrated that a given neuron exhibits similar orientation preferences to luminance-defined (first-order) and contrast/texture-defined (second-order) stimuli.
Electrophysiological recording in early visual cortex showed that only about half of the sampled neurons (von der Heydt et al., 1984; Zhou and Baker, 1993) respond to the second-order stimuli, but did not reveal how these neurons are organized. Using optical imaging, Sheth et al. (1996) reported that orientation maps in cat A18 to illusory contours were distinct from those to luminance gratings, suggesting that illusory-contour-responsive neurons might be clustered in patches. Also using optical imaging, Ramsden et al. (2001) found similar orientation maps in macaque V2 to both stimuli, implying a uniform distribution of illusory-contour-responsive neurons across the cortical surface. These differing conclusions might reflect the differing species, or could be due to methodological differences in quantifying the similarity between orientation maps.
In natural scenes, first- and second-order visual information tends to be spatially coincident (Johnson and Baker, 2004), and psychophysical studies indeed demonstrate that both cues working together can facilitate shape perception (Schofield et al., 2005). Therefore a uniform distribution might be more efficient for local neuronal integration of first- and second-order cues. Consistent with this idea, successively encountered neurons along a normal penetration (Zhou and Baker, 1996; Mareschal and Baker, 1999) were not correlated in second-order responsivity. If this uniformity hypothesis is correct, the similarity between neurons' orientation preferences to luminance- versus texture/contrast-defined contours should result in similar cortical orientation maps to first- and second-order stimuli.
Because previous optical imaging studies only compared illusory contours to luminance gratings, it is an open question whether neurons responding to different second-order stimuli are organized in the same way. Since second-order visual stimuli such those in Figure 1 are produced by modulation of texture which is beyond the neurons' frequency passband, the visual cortex might employ a nonlinear demodulation (Daugman and Downing, 1995) to extract the contour orientation. For example a filter–rectify–filter model (F-R-F, lower stream in Fig. 1M; Wilson et al., 1992) could provide a common demodulating mechanism for all these second-order stimuli. If such second-order neurons are uniformly distributed across the cortical surface, we would expect similar orientation preference maps for all these stimuli.
To investigate these issues we have employed intrinsic signal optical imaging (Bonhoeffer and Grinvald, 1996; Zhan et al., 2005) to measure the cortical response in cat A18 to a variety of first- and second-order visual stimuli. Guided by previous studies of these neurons' spatiotemporal properties (Movshon et al., 1978; Zhou and Baker, 1996; Mareschal and Baker, 1999; Issa et al., 2000), we chose stimulus parameters to ensure that the textures of the second-order stimuli would be beyond the luminance spatial resolution of A18 neurons, while the coarse modulation contours and sinewave gratings would be appropriate to the sizes of these neurons' receptive fields. Thus our texture-defined stimuli will only drive genuinely second-order responsive neurons, permitting us to obtain orientation maps based on pure second-order responses.
Our results demonstrate very similar orientation maps for all of our first- and second-order visual stimuli. The high similarity between these maps suggests a common mechanism for processing all second-order visual information, and implies that the second-order responsive neurons are distributed homogeneously across the cortical surface. Such an organization might be optimal in providing a cue-invariant representation of natural scenes.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Animal Preparation and Maintenance
Animal use in this project was in accordance with the guidelines and policies of the Canadian Council on Animal Care, and was approved by the Animal Care Committee of McGill University. Procedures for anesthesia, surgery and paralysis were conventional and have been described previously (Mareschal and Baker, 1999). For optical imaging in A18, a 6 x 8 mm craniotomy was centered at Horsley–Clark coordinates A3/L4 (Tusa et al., 1979) and the dura carefully reflected. A plastic chamber was attached to the skull with dental acrylic, filled with 2% agarose and sealed with a coverslip. The cat's eyes were focused at a distance of 28.5 cm, using appropriate spectacle lenses and artificial pupils. Throughout recording, the animal's physiological state and anesthesia level were constantly monitored (EEG, ECG, expired CO2, temperature, heart rate) and maintained at appropriate levels.
Visual Stimuli
Visual stimuli were computer-generated (Macintosh G4) using custom software based on the Psychophysics Toolbox (Brainard, 1997; Mareschal and Baker, 1999). The stimulus display monitor (NEC FP1350, 20 inch, 1024 x 768 pixels, 85 Hz) was gamma-corrected at a mean luminance of 28 cd/m2. A series of oriented periodic stimuli were used to study first- and second-order visual processing. First-order stimuli consisted of drifting sinewave gratings, i.e. luminance contours (Fig. 1A). Several kinds of second-order stimuli were constructed from a drifting low spatial frequency grating envelope which modulated either the contrast or the phase of a high spatial frequency carrier/texture (e.g. line grating, checkerboard, 1/f noise in Fig. 1B–F). Contrast contours (Zhou and Baker, 1993, 1996; Mareschal and Baker, 1999) modulated the contrast of the carriers (Fig. 1B–D), while illusory contours modulated the phase of sinewave (Fig. 1E; Grosof et al., 1993) or line (Fig. 3F; von der Heydt et al., 1984; Sheth et al., 1996; Ramsden et al., 2001) grating carriers. The contrast of the first-order gratings and the second-order carriers was 70%; modulation depth of the second-order envelopes was 100%. Two to eight equally spaced orientations in a 0–180° range were used to measure orientation responses. The relative orientation (if applicable) between envelopes and carriers was either orthogonal or ±45°. The spatial and temporal frequencies of the first-order sinewave gratings and second-order envelopes were in the optimal range for A18 neurons (0.075–0.15 cpd, 2–4 Hz), while spatial frequencies of carriers (>0.8 cpd) were chosen to exceed the luminance resolution of these neurons, and fall in the range of optimal carrier frequencies for A18 neurons (Movshon et al., 1978; Zhou and Baker, 1996; Mareschal and Baker, 1999; Issa et al., 2000).
|
Optical Imaging Data Acquisition and Processing
Intrinsic signal optical imaging (Bonhoeffer and Grinvald, 1996) was used to measure the cortical response to visual stimuli. Our imaging system (Zhan et al., 2005) consisted of a 720 nm LED illuminator, a CCD camera (2/3 inch format, 30 frames/s, 56 dB; Cohu 4812, CA), a macro zoom lens (focal length 18–108 mm, f/2.5) or tandem-lens formed by two Nikor lenses (focal length 50 mm, f/1.2), a digital video processor (12-bit precision, 640 x 480 pixels; DVP-32, InstruTech, NY) and a host computer (PIII, 800 MHz). Raw image data was online spatiotemporally binned to 320 x 240 pixels at 
2 fps. Each recording trial lasted 35 s (6.4 s pre-stimulus baseline, 8.5 s stimulus duration, 3.7 s post-stimulus period, and the remaining inter-trial interval).
For quantitative comparison of orientation maps it is crucial to have reliable optical signals, in the face of noise originating from diverse sources (Bonhoeffer and Grinvald, 1996; Sirovich and Kaplan, 2002). Also it is reasonable to expect somewhat weaker signals from second-order stimuli, due to the lower incidence of responsive neurons and their lower firing rates to these stimuli (von der Heydt, 1984; Zhou and Baker, 1993; Leventhal et al., 1998; Mareschal and Baker, 1999). We employed several measures to improve the signal-to-noise ratio of response patterns, and to evaluate the reliability of the optical signal at each pixel. (i) To minimize effects of inter-condition interference, we used a long inter-stimulus interval of 27 s (inter-trial interval together with pre- and post-stimulus periods). (ii) Pixels with large cross-trial variability were considered unreliable. We estimated this variability for each pixel to construct a variability map (see below) for each dataset. We then formulated an objective criterion to threshold the map and obtained a mask that excluded unreliable pixels from further quantitative analysis. (iii) Pixels in pinwheel centers (Crair et al., 1997) and fractures (Bonhoeffer and Grinvald, 1993), where orientation preference changes very rapidly, contribute disproportionately to the measurement variance in the orientation maps. The pinwheel center pattern of a full-orientation map was dilated with a 5 pixel radius disk; its union with the fracture pattern (gradient of orientation change >15°/pixel) was used to form a binary mask to remove these noise-sensitive pixels. (iv) As detailed below, we also assessed each pixel's reliability in representing the stimulus orientation across trials, and excluded pixels with low t-scores. (v) A very conservative image filtering (bandpass, Gaussian kernel, 
L = 54 µm, H = 810 µm) was applied, to minimize the impact of noise at low spatiotemporal frequencies (e.g. vasomotion) and at high frequencies (e.g. from the CCD camera).
We evaluated pixel reliability across recording trials from pre-stimulation images. To exclude the effects of slow drifts (e.g. due to progressively increasing in opacity of the agarose), we first did a trend-removal by subtracting a best-fitting quadratic polynomial (
equation 1) from each pixel's response profile (I) across trials. The intrinsic variability of each pixel was captured by the variance (s2, equation 2) of regression residuals (i.e. the variance of the difference between the original data and the fitted function):
 | (1) |
 | (2) |
Figure 2A shows one frame of raw image data from one dataset; Figure 2B is the variance map for the same dataset, where brighter pixels indicate larger variance across trials. This variance map is highly correlated with the blood vessels, which objectively indicates that data from blood vessel areas are not reliable. It also captured unreliable pixels beyond blood vessels (indicated by arrows in Fig. 2B). Based on the histogram of the variance map (Fig. 2D), we determined an objective threshold to segregate the blood vessels and high variance pixels. Because the distribution of the square-rooted pixel values in Figure 2B seems to be heavy-tailed (Fig. 2D), we fit it with a sum of Gaussian and power law functions. Pixels from the blood vessel-free areas are described by the Gaussian function N(µ,). The blood vessel pixels form a fractal-like (self-similar) pattern, and their contribution to the pixel histogram can be described by a power law function [f(x) = Ax–
, x > , where A and are fitted parameters]. The inset of Figure 2D shows an enlarged version of the tail for illustration. The resultant blood vessel map, segregated with the threshold , is shown in Figure 2C. Figure 2E shows a raw difference map to orthogonal orientations (averaged across 30 trials), where light versus dark pixels represent greater response to horizontal versus vertical orientations. Figure 2F shows this difference map masked by the variance map (Fig. 2C), demonstrating complete removal of the blood vessel residues.
|
In addition we tested the statistical significance of optical signals due to stimulus activation, by evaluating the optical signal reliability based on trial-by-trial difference maps. In theory, each pixel has zero-mean in the difference maps across repetitions (i.e. µ = 0) unless it is reliably activated by the visual stimuli. Thus a pixelwise t-test
was conducted to examine this hypothesis, where 
is the mean intensity and Si the standard deviation of ith pixel across the n repetitions. Figure 2G shows the absolute values of the orientation-difference values in Figure 2E, and Figure 2H shows a map of their t-score values (thresholded at t > 1.7, df = 29, P < 0.1); this relatively high P-value is chosen to avoid unduly excluding excessive numbers of pixels in these raw difference images which have not yet been spatially smoothed. Notice that although pixels in some blood vessel regions show large difference-map values, their t-scores are small (e.g. arrows in Fig. 2G,H). The pixel variance map (Fig. 2B) indicates spontaneous variation, primarily due to blood vessels, while the t-score map (Fig. 2H) shows variability of stimulus-driven responses. We combine these two kinds of information to create a mask for unreliable pixels (‘URP-mask’), which is formed by the union of blood vessel pixels (bright in Fig. 2C) and low t-score pixels (dark in Fig. 2H). Pixels within this URP-mask will be excluded from subsequent quantitative analysis.
Other data analyses were conventional and are described briefly below. Raw image data were first averaged across 20–40 repetitions, then normalized by a pure blank (average of pre-stimulus images) to correct for uneven illumination. Averaged images for orthogonal conditions were subtracted to calculate raw difference images. A polygonal region of interest (ROI), defined based on the responsiveness of difference images and their t-score maps. Optical response strength was quantified by the standard deviation of pixel values within the URP-masked ROI (Schuett et al., 2001; Zhan et al., 2005). Image filtering was implemented by interpolation of blood vessel masked pixels, followed by spatial bandpass filtering (as described above). As discussed previously (Zhan et al., 2005), this approach removes high-frequency noise (e.g. from the CCD camera) and lessens the impact of low-frequency artifacts (e.g. vasomotion), while minimizing the effects of blood vessels and other unreliable pixels during filtering. For subsequent quantitative analyses, the URP mask was again imposed to exclude the interpolated pixels.
Full orientation maps (Fig. 3G–I) were constructed by pixel-wise vectorial summation (Bonhoeffer and Grinvald, 1996) or curve-fitting to each pixel's orientation response (Swindale et al., 2003), which gives similar results. Fractures were derived following Bonhoeffer and Grinvald (1993), based upon a map of the magnitude of orientation gradient:
 | (3) |
: a pixel is considered to be a pinwheel center if the sum of orientation differences (wrapped into –9090°) between its four counterclockwise neighbors is ±180°.
|
Three methods were employed to quantitatively assess the similarity of maps for different kinds of visual stimuli. To compare the similarity between pairs of full-orientation maps, we constructed differential angle maps, in which each pixel's value is the (wrapped) difference in preferred orientation for the two maps. For an overall comparison between difference images, we first used a simple pixel-wise correlation coefficient. However many of our second-order stimuli contain distinct carrier and envelope orientations; to quantitatively assess the degree to which a second-order orientation-difference map represents the envelope rather than the carrier orientation, we employed a ‘response profile’ analysis (Basole et al., 2003
). For each difference map, the average values of pixels enclosed by adjacent iso-orientation contours (derived from full-orientation maps for luminance-defined gratings) were plotted against the eight corresponding orientation values to form a ‘response profile’. The orientation best represented by the neuronal population in this difference image was estimated by fitting the response profile with one full cycle of a cosine function, O(
) = B x cos [2
( + p)/180], where O is the response strength as a function of , the iso-orientation value (0 –157.5° in 22.5° steps). Thus each response profile could be parameterized with the two fitted parameters: B, the response amplitude and p, the orientation best represented by the optical response. |
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Orientation Maps to First- and Second-order Stimuli
We chose stimulus parameters to optimally isolate first- and second-order responses, guided by previous neurophysiological studies on spatiotemporal tuning properties of cat area 18 neurons using gratings (Movshon et al., 1978; Issa et al., 2000) and contrast contours (Zhou and Baker, 1996; Mareschal and Baker, 1999). First-order stimuli were sinewave gratings at a spatial frequency (SF) of 0.15 cpd, which was optimal for many cat area 18 neurons (Fig. 3A, as illustrated with superimposed receptive field). The spatial frequency of second-order carriers (SFcarr) was higher than 0.8 cpd (typically 1.3 cpd), which was beyond these neurons' luminance resolution and thus unable to activate cortical responses when presented alone (as verified by control experiments, e.g. part of Fig. 5). Low spatial frequency sine- or square-wave gratings were used as envelopes (SFenv = 0.075–0.15 cpd) to modulate the carrier's contrast or phase for generating contrast contours (Fig. 3B) and illusory contours (Fig. 3C). Because of the multiplicative nature of these modulations, these second-order stimuli did not contain Fourier energy falling in the SF passband of these neurons (Fig. 1H-L), i.e. they were beyond the luminance-based resolution limit of these neurons. This property is the key difference between first- and second-order visual stimuli. When presented with back-and-forth motion, these first- and second-order stimuli activated very reliable intrinsic signal optical responses, giving similar orientation-difference maps (Fig. 3D–F). The optical responses to contrast- and illusory contours are highly correlated with that to luminance contours (correlation coefficients of 0.84 and 0.86, respectively). The full-orientation maps also exhibit very similar ‘pinwheel’ patterns (Fig. 3G–I).
|
To quantitatively assess the similarity of each second-order full-orientation map to the first-order map, Figure 4A shows a differential orientation map in which each pixel represents the difference between the orientation preference of corresponding pixels in the sinewave grating and contrast contour maps (Fig. 3G,H). The pixel value histogram (Fig. 4C) of this differential angle map has a pronounced peak centered near zero, with 89.6% of the pixels differing <30° in orientation preference, indicating a high degree of similarity. The same analysis for illusory contours versus sinewave gratings (Fig. 4B,D) shows pixels similarly clustered near zero, with 93.1% of pixels differing less than 30°. Thus only a minority of pixels show substantial differences in preferred orientation for first- and second-order stimuli.
Those pixels differing substantially (>30°) in orientation preference for first- and second-order stimuli might be due to measurement noise, such as blood vessel pulsation (Fig. 2B) or spurious effects of fractures and pinwheel centers. Fractures consist of image regions with large variations in orientation preference (Bonhoeffer and Grinvald, 1993); pinwheel centers are surrounded by pixels with very different orientation preferences spanning a full 180° range (Crair et al., 1997). Therefore these areas may be especially sensitive to measurement variability, and thus inflate the differences between measured orientation maps. To investigate this possibility, we first localized the fractures (Fig. 4E) and pinwheel centers (Fig. 4F) in the first-order orientation map (Fig. 3G; see Materials and Methods), and applied this mask (union of Fig. 4E,F) to the differential angle map (e.g. Fig 4B), to obtain a corresponding map only for pixels near pinwheel centers and fractures (Fig 4G). The histogram of pixels within the mask (but excluding pixels in the URP mask) is much less sharply concentrated near zero. This histogram divided by that shown in Figure 4D yields a ratio histogram (Fig. 4H); if pixels' differential angles were uncorrelated with whether they fell in fractures or pinwheel centers, this ratio histogram would be uniform. A unitary ratio for a specific differential angle indicates corresponding pixels in Figure 4B are completely captured by the fracture and pinwheel center mask. Thus the V-shaped ratio histogram in Figure 4H indicates that pinwheel center and fracture areas disproportionately account for the pixels having large differences in first- and second-order orientation preference. After masking, the pixel histogram is shown in Figure 4J, where the percentage of pixels with differential angles exceeding 30° (pixels in Fig. 4B beyond the mask) increased to 95.7%. A similar analysis for Figure 4A indicated an increased percentage (92.7%) for contrast-contour versus luminance response. These results after exclusion of fractures and pinwheel centers are very consistent across all the examined animals: on average, 91.9% (SD = 2.27, n = 4) of the pixels are within 30° difference in orientation preference for contrast contours versus sinewave gratings, and 93.3% (SD = 3.01, n = 4) for illusory contours. For the contrast- and illusory-contour data taken together, the pinwheel center/fracture mask captures 69.8% of the pixels (SD = 4.89, n = 8) with orientation differences >50°. These results indicate that orientation preference maps are nearly identical for first- and second-order visual stimuli, with most of the differences arising from a minority of pixels which are most vulnerable to measurement noise.
Carrier Orientation Invariance
The similarity between first- and second-order responses might indicate that the measured cortical area can invariantly signal coarse-scale orientations (envelope orientations for second-order) regardless of how they are rendered. However, Figure 3 only showed responses to second-order stimuli whose carriers were perpendicular to their envelopes. Thus it is open to question whether these second-order maps are immune to variations of carrier orientation. Therefore we need to examine whether carrier orientations are represented in the second-order responses. To disambiguate carrier versus envelope effects, we obtained difference maps for second-order stimuli with carrier orientations differing by ±45° from their envelope orientations. The central insets of Figure 5A,B show stimulus images in which the envelopes in each row are orthogonal while the carriers are identical; in each column the envelopes are identical while the carriers are orthogonal. If the optical response is driven by envelope orientation, each row would give a robust differential map, and if optical response is independent of carrier orientation, the two rows would give very similar difference maps. As shown above or below each row, very similar patchy patterns are evident for contrast- and illusory-contour pairs, even though carrier orientation for each row is different (the superimposed colored contours will be explained below). The corresponding differential maps for carrier orientation pairs (left and right columns of stimulus images), however, do not show clear and robust orientation domains. Therefore it seems that the neuronal population response represents the orientations of envelopes rather than carriers.
To verify this impression, we quantified the similarity between difference maps for envelopes and carriers by calculating their correlation coefficient (r). The r-values for pairs of difference images to envelopes (above and below the stimuli) were 0.85 for contrast contours and 0.88 for illusory contours; the r-values for difference images to carriers (left and right of the stimuli) were –0.24 (Fig. 5A) and –0.07 (Fig. 5B). These results clearly show robust difference maps for envelopes but not for carriers. However, the high correlation coefficients between difference responses cannot characterize which orientations are represented in each difference map, in particular whether the difference maps really represent the envelope orientations. Based on population coding theory (Pouget et al., 2000), we applied a response profile analysis (Basole et al., 2003) to derive the best single luminance orientation which would theoretically produce that map. Firstly, iso-orientation contours (color-coded curves for eight equally spaced index orientations, 0–157.5° with 22.5° steps, Fig. 5) derived from a first-order full-orientation map were used to partition each difference map into subregions containing pixels with similar orientation preference. In Figure 5A, the top-left graph plots mean pixel values for each subregion against its orientation, for the difference map shown in the top-center. Notice that the pixels in darker regions respond strongly to the preferred orientation and those in brighter regions respond better to the orthogonal orientation in a difference map; the response profile reaches its minimum (maximum) when the stimulus orientation is identical (orthogonal) to its preferred orientation. To the extent that a given orientation difference map represents envelope orientations and corresponds to one obtained with luminance gratings, the response profile will be periodic with a trough at zero (and a peak at 90°). In this case (upper-left graph) the best-fitting cosine function had a trough at 
= 2.45° (curve-fit r2 = 0.96). A similar analysis was applied to the difference image (bottom-center in Fig. 5A) for orthogonal envelope orientations but identical carriers of –45°. The response profile (bottom-right) again reached a trough near zero (cosine-fit: = 1.22°, r2 = 0.99).
However, difference maps to orthogonal carrier pairs (left and right panels in middle row) did not exhibit similarly robust profiles. Their corresponding response profiles (upper-right and lower-left) were about an order of magnitude smaller than those for envelopes, and did not show any relation to the carrier orientations. Figure 5B shows the same analysis for difference maps to illusory contour stimuli, with very similar results: (i) similar cosine function minima for envelope pairs ( = 5.98°, r2 = 0.98 for top-left; = 1.57°, r2 = 0.98 for bottom-right); (ii) very small magnitudes and different response profiles for differential maps to the carrier pairs. These measurements are consistent across all the examined datasets for contrast- and illusory-contours (n = 8). The average values of the minima for the envelopes with +45° carriers (upper) and –45° carriers (lower) in Figure 5 were 1.57° (SD = 2.83) and 0.81° (SD = 1.89), respectively, which were not significantly different (two-tailed t-test, P = 0.38). In contrast, profiles for orthogonal carriers were consistently small in magnitude and variable in form across the different datasets. These results therefore demonstrate that the differential optical maps in Figure 5 robustly represent the orientations for the envelopes rather than those for the carriers. The results indicate that carriers themselves did not activate any specific pattern, consistent with their spatial frequency being beyond the passband of the examined cortical area. Thus these difference optical images for envelopes reflect genuine second-order responses.
Carrier Structure Invariance
The second-order visual stimuli used above contained only one carrier orientation in each case. To assess the generality of the findings, we designed a broader range of second-order visual stimuli with different carriers to compare whether the population-level orientation responses to envelopes are independent of their carrier patterns. In Figure 6 the left column shows the visual stimuli, and the second column shows their corresponding Fourier spectra. Figure 6A–C shows contrast contours with different carriers (sinewave gratings, regular checkerboards and high-pass 1/f noise); Figure 6D,E shows illusory contours produced by phase modulations of sinewave and line grating carriers, respectively; and Figure 6F shows first-order sinewave gratings for comparison. Although these stimuli are very different in appearance, local features and Fourier energy distribution (second column), their differential optical responses to envelope orientation (third column) appear very similar to one another. Their correlation coefficients to a response pattern template (measured to vertical–horizontal first-order orientation stimuli), are 0.785, 0.812, 0.812, 0.809, 0.729 and 0.939, respectively. These values indicate that the second-order responses are highly similar to the first-order response, and imply a mutual similarity between different second-order responses (see online supplementary material).
|
Population response profiles (rightmost column), constructed as in Figure 5, quantitatively show clear systematic dependence on the envelope orientations for all the second-order stimuli or the first-order orientations. Cosine function fits showed all minima close to 0° (
= 2.45, –1.33, 3.30, 5.98, –2.96, 1.47 from top to bottom; r2 > 0.95). Similar results were found in all four tested animals (the [mean, SD] values of the s are [1.12, 1.86], [0.57, 2.01], [1.36, 2.43], [2.56, 2.99], [–0.37, 2.19] and [0.13, 1.56], respectively). |
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
How the nervous system represents its input at a neuronal population level (Pouget et al 2000
) is fundamental to understanding biological computation. Our optical imaging results have shown that orientation maps in cat visual cortex (A18) are almost identical for contours defined by differences in luminance (first-order) and for a broad range of texture (second-order) cues. These results bear upon two important issues regarding early visual cortex: the mechanisms for second-order visual processing, and the organization of second-order, orientation-selective neurons.
One of the common properties of these second-order visual stimuli is that the Fourier energy of the low spatial frequency contours is shifted to the high spatial frequency domain of the carriers due to the modulation. Thus the Fourier energy of these stimuli lies outside a neuron's classical spatial frequency passband — the luminance changes in the visual stimuli are beyond the spatial resolution of neurons' receptive fields. These stimuli are different in nature from the ‘random field of iso-orientation bars’ used by Basole et al. (2003), whose results can be explained by a linear filter model (Adelson and Bergen, 1985; Baker and Issa, 2005; Mante and Carandini, 2005) responding to their first-order Fourier energy. Such a linear model cannot explain our second-order responses, which imply processing mechanisms exhibiting carrier-invariance for envelope orientation signaling.
If the orientation-selective responses to texture contours resulted from logical operations on categorical features (e.g. Marr, 1982), a multitude of mechanisms might be necessary to handle the different combinations of texture attributes. For example an illusory contour (Fig. 6E) formed by abutting gratings might be detected by an AND-gating of end-stopped cells oriented orthogonally to the contours (Peterhans and von der Heydt, 1989); however, such a model has difficulty in explaining neural response to contours defined by different textures (e.g. Fig. 6B–D).
Alternatively a common neural mechanism might underlie a more versatile, filter/energy-based, second-order visual processing, such as an F-R-F scheme (Wilson et al., 1992; Kingdom et al., 2003), in which neurons with small receptive fields would respond to high carrier spatial frequencies (filter F1 in Fig. 1M) and the profile of their rectified responses would correspond to the envelope waveform. [These neurons must be orientation selective (Mareschal and Baker, 1998), suggesting they either reside in A17 or originate from extrastriate feedback, e.g. A21a (Morley and Vickery, 1997; Wang et al., 2000).] These responses would be summed by a coarser-scale receptive field which would provide orientation-selectivity to the envelope (filter F2 in Fig. 1M). Our second-order stimuli were generated by modulation of carrier properties (contrast or phase) by low spatial frequency envelopes (contours), which could be recovered or demodulated by the F-R-F operation. The demodulated envelopes produced by such a model are devoid of carrier-related information, consistent with our optical imaging data demonstrating orientation preference maps which are invariant with the textures (cues) inducing them or with the type of modulation.
Single-unit studies have shown that only about one-half of cortical neurons respond to illusory contours in monkey V2 (von der Heydt et al., 1984) and contrast contours in cat A18 (Zhou and Baker, 1993; 1996; Mareschal and Baker, 1999). Conceivably these neurons might be segregated into anatomical compartments, or alternatively they could be distributed uniformly across the cortex. Using optical imaging, Sheth et al. (1996) previously reported a spatially clustered distribution of illusory contour responsive neurons in cat A17/18. Such an inhomogeneous distribution might seem reasonable since neurons preferring different orientations, directions of motion, spatial frequencies and ocular dominance are organized in columnar structures. However the line inducers for illusory contours used by Sheth et al. (1996) were 0.45 cpd, a spatial frequency that is resolvable by cat A18 (as well as A17) neurons (Movshon et al., 1978; Issa et al., 2000). Therefore their data could be a mixture of first- and second-order responses and not appropriate for comparing the first- and second-order maps. The carrier spatial frequencies we have used for second-order response mapping were always higher than the luminance resolution of A18 neurons. We verified that our carriers themselves did not activate any cortical responses in area 18, and also confirmed that our data is actually from A18 using the optical imaging method described by Ohki et al. (2000).
The uniform distribution revealed by our experiments may be more developmentally plausible and functionally efficient than a clustered arrangement. It has been suggested that neurons receiving correlated inputs tend to ‘wire together’ in development (Fox, 2000; Lendvai et al., 2000). Such a developmental principle might result in connectivity between retinotopically corresponding neurons which have similar orientation preference to first- and second-order orientation stimuli, because both kinds of orientation information tend to be spatially coincident in natural scenes (Johnson and Baker, 2004). In that case the wiring length would be best economized by physical proximity of first- and second-order neurons having consistent preferred envelope orientations. Thus a uniform distribution of second-order responsive neurons across the cortical surface might reflect an optimal neuronal wiring strategy.
While some second-order responsive neurons showed selectivity to carrier orientation (Mareschal and Baker, 1999), our results show that at a population level, orientation columns appear to be genuinely carrier- (cue-) invariant. Higher-level neurons which locally pool such neurons could then provide an explicit neural substrate for form–cue invariance, which has been suggested by human psychophysics (McGraw et al., 1999).
The amplitudes of second-order responses in our experiments were only about 1/5 of those to first-order stimuli (Fig.6), consistent with their distinct contrast response functions (Ledgeway et al., 2005) and the percentage of second-order responsive neurons (von der Heydt et al., 1984; Zhou and Baker, 1993). However the smaller response to second-order visual stimuli does not necessarily imply less importance of second-order information. For example, if the early visual cortex employs temporal coding (Fiser et al., 2004; Yu et al., 2005), response strength might not be the only determinative factor in coding visual information. The importance of second-order response may also depend on its interaction with first-order response — for example, a second-order cue significantly facilitates depth perception when it occurs in appropriate phase with luminance information (Schofield et al., 2005). In any case, our results reveal that at a neuron population level, orientation-preference maps are invariant for all the first- and second-order visual stimuli which we tested. These results suggest the early visual cortex can provide a neural basis for boundary cue-invariant visual processing.
|
Supplementary Material |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.
|
Acknowledgments |
|---|
We also thank Lynda Domazet, Aaron Johnson and Yuning Song for assistance with the experiments. This work was supported by a Canadian Institutes of Health Research grant MA-9685 to C.L.B.
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionSupplementary MaterialReferences |
|---|
Adelson EH, Bergen JR (1985) Spatiotemporal energy models for the perception of motion. J Opt Soc Am A 2:284–299.[Web of Science][Medline]
Albright, TD (1992) Form–cue invariant motion processing in primate visual cortex. Science 255: 1141–1143.
Baker CL Jr (1999) Central neural mechanisms for detecting second-order motion. Curr Opin Neurobiol 9:461–466.[CrossRef][Web of Science][Medline]
Baker TI, Issa NP (2005) Cortical maps of separable tuning properties predict population responses to complex visual stimuli. J Neurophysiol 94:775–787.
Basole A, White LE, Fitzpatrick D (2003) Mapping multiple features in the population response of visual cortex. Nature 423:986–990.[CrossRef][Medline]
Bonhoeffer T, Grinvald A (1993) The layout of iso-orientation domains in area 18 of cat visual cortex: optical imaging reveals a pinwheel-like organization. J Neurosci 13:4157–4180.[Abstract]
Bonhoeffer T, Grinvald A (1996) Optical imaging based on intrinsic signals – the methodology. In: Brain mapping: the methods (Toga AW, Mazziotta JC, eds), pp. 55–97. San Diego, CA: Academic Press.
Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436.[Web of Science][Medline]
Carandini M, Heeger DJ, Movshon JA (1997) Linearity and normalization in simple cells of the macaque primary visual cortex. J Neurosci 17:8621–8644.
Chubb C, Landy MS (1991) Orthogonal distribution analysis: a new approach to the study of texture perception. In: Computational models of visual processing (Landy MS, Movshon JA, eds), pp. 291–301. Cambridge, MA: MIT Press.
Chubb C, Sperling G (1988) Drift balanced random dot stimuli: a general basis for studying non-Fourier motion. J Opt Soc Am A 5:1986–2007.[Web of Science][Medline]
Crair MC, Ruthazer ES, Gillespie DC, Stryker MP (1997) Ocular dominance peaks at pinwheel center singularities of the orientation map in cat visual cortex. J Neurophysiol 77:3381–3385.
Daugman JG, Downing CJ (1995) Demodulation, predictive coding, and spatial vision. J Opt Soc Am A 12:641–660.[Web of Science][Medline]
De Valois KK, De Valois RL, Yund EW (1979) Responses of striate cortex cells to grating and checkerboard patterns. J Physiol 291:483–505.
Fiser J, Chiu C, Weliky M (2004) Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature 431:573–578.[CrossRef][Medline]
Fox K (2000) A moving experience. Nature 404:825–826.[CrossRef][Medline]
Grosof DH, Shapley RM, Hawken MJ (1993) Macaque V1 neurons can signal ‘illusory’ contours. Nature 365:550–552.[CrossRef][Medline]
Hubel DH, Wiesel TN (1965) Receptive fields and functional architecture in two non-striate visual areas (18 and 19) of the cat. J Neurophysiol 28:229–289.
Issa NP, Trepel C, Stryker MP (2000) Spatial frequency maps in cat visual cortex. J Neurosci 20:8504–8514.
Johnson AP, Baker CL Jr. (2004) First- and second-order information in natural images: a filter-based approach to image statistics. J Opt Soc Am A 21:913–925.[CrossRef]
Kingdom FA, Prins N, Hayes A (2003) Mechanism independence for texture modulation detection is consistent with a filter-rectify-filter mechanism. Vis Neurosci 20:65–76.[CrossRef][Web of Science][Medline]
Ledgeway T, Zhan C, Johnson AP, Song Y, Baker CL Jr. (2005) The direction-selective contrast response of area 18 neurons is different for first- and second-order motion. Vis Neurosci 22:87–99.[CrossRef][Web of Science][Medline]
Lendvai B, Stern EA, Chen B, Svoboda K (2000) Experience-dependent plasticity of dendritic spines in the developing rat barrel cortex in vivo. Nature 404:876–881.[CrossRef][Medline]
Leventhal AG, Wang Y, Schmolesky MT, Zhou Y (1998) Neural correlates of boundary perception. Vis Neurosci 15:1107–1118.[CrossRef][Web of Science][Medline]
Mante V, Carandini M (2005) Mapping of stimulus energy in primary visual cortex. J Neurophysiol 94:788–798.
Mareschal I, Baker CL Jr (1998) A cortical locus for the processing of contrast-defined contours. Nat Neurosci 1:150–154.[CrossRef][Web of Science][Medline]
Mareschal I, Baker CL Jr (1999) Cortical processing of second-order motion. Vis Neurosci 16:527–540.[CrossRef][Web of Science][Medline]
Marr D (1982) Vision: a computational investigation into the human representation and processing of visual Information. San Francisco, CA: WH Freeman.
McGraw PV, Levi DM, Whitaker D (1999) Spatial characteristics of the second-order visual pathway revealed by positional adaptation. Nat Neurosci 2:479–484.[CrossRef][Web of Science][Medline]
Morley JW, Vickery RM (1997) Spatial and temporal frequency selectivity of cells in areas 21a of the cat. 501:405–413.
Movshon JA, Thompson ID, Tolhurst DJ (1978) Spatial and temporal contrast sensitivity of neurones in areas 17 and 18 of the cat's visual cortex. J Physiol 283:101–120.
Ohki K, Matsuda Y, Ajima A, Kim D-S, Tanaka S (2000) Arrangement of orienation pinwheel centers around area 17/18 transition zone in cat visual cortex. Cereb Cortex 10:583–601.
Peterhans E, von der Heydt R (1989) Mechanisms of contour perception in monkey visual cortex. I. Lines of pattern discontinuity. J Neurosci 9:1731–1748.[Abstract]
Pouget A, Dayan P, Zemel R (2000) Information processing with population codes. Nat Rev Neurosci 1:125–32.[CrossRef][Web of Science][Medline]
Ramsden BM, Hung CP, Roe AW (2001) Real and illusory contour processing in area V1 of the primate: a cortical balancing act. Cereb Cortex 11:648–665.
Schofield A, Hesse G, Georgeson M (2005) Texture amplitude is a cue to perception of shape from shading. 15th Annual Meeting of Vision Sciences Society, Abstract no. 524.
Schuett S, Bonhoeffer T, Hubener M (2001) Pairing-induced changes of orientation maps in cat visual cortex. Neuron 32:325–337.[CrossRef][Web of Science][Medline]
Sheth BR, Sharma J, Rao SC, Sur M (1996) Orientation maps of subjective contours in visual cortex. Science 274:2110–2115.
Smith AT, Ledgeway T (1997) Separate detection of moving luminance and contrast modulations: fact or artifact? Vis Res 37:45–62.[CrossRef][Web of Science][Medline]
Sirovich L, Kaplan E (2002) Analysis methods for optical imaging. In: In vivo optical imaging of brain function (Frostig RD, ed.), pp. 43–76. New York: CRC Press.
Swindale NV, Grinvald A, Shmuel A (2003) The spatial pattern of response magnitude and selectivity for orientation and direction in cat visual cortex. Cereb Cortex 13:225–238.
Tusa RJ, Rosenquist AC, Palmer LA (1979) Retinotopic organization of area 18 and 19 in the cat. J Comp Neurol 185:657–678.[CrossRef][Web of Science][Medline]
von der Heydt R, Peterhans E, Baumgartner G (1984) Illusory contours and cortical neuron responses. Science 224:1260–1262.
Wang C, Waleszczyk WJ, Burke W, Dreher B (2000) Modulatory influence of feedback projections from area 21a on neuronal activities in striate cortex of the cat. Cereb Cortex 10:1027–1232.
Watson AB, Ahumada AJ (1983) A look at motion in the frequency domain. In: Motion: perception and representation (Tsotsos JK, ed.), pp. 1–10. Baltimore, MD: ACM.
Wilson HR, Ferrera VP, Yo C (1992) A psychophysically motivated model for two dimensional motion perception. Vis Neurosci 9:79–97.[Web of Science][Medline]
Yu Y, Romero RD, Lee TS (2005) Preference of sensory neural coding for 1/f signals? Phys Rev Lett 94:108103 (1-4).
Zhan CA, Ledgeway T, Baker CL Jr (2005) Contrast response in visual cortex: quantitative assessment with intrinsic optical signal imaging and neural firing. Neuroimage 26:330–316.[CrossRef][Web of Science][Medline]
Zhou YX, Baker CL Jr (1993) A processing stream in mammalian visual cortex neurons for non-Fourier responses. Science 261:98–101.
Zhou YX, Baker CL Jr (1996) Spatial properties of envelope-responsive cells in area 17 and 18 neurons of the cat. J Neurophysiol 75:1038–1050.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  B. Dillenburger and A. W. Roe Influence of Parallel and Orthogonal Real Lines on Illusory Contour Perception J Neurophysiol, January 1, 2010; 103(1): 55 - 64. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. O. Li, S. K. Khuu, and A. Hayes Background motion and the perception of shape defined by illusory contours J Vis, June 9, 2009; 9(6): 5 - 5. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. P. Issa, A. Rosenberg, and T. R. Husson Models and Measurements of Functional Maps in V1 J Neurophysiol, June 1, 2008; 99(6): 2745 - 2754. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. A. Zhan and C. L. Baker Jr Critical Spatial Frequencies for Illusory Contour Processing in Early Visual Cortex Cereb Cortex, May 1, 2008; 18(5): 1029 - 1041. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. D. Lu and A. W. Roe Optical Imaging of Contrast Response in Macaque Monkey V1 and V2 Cereb Cortex, November 1, 2007; 17(11): 2675 - 2695. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Z.-M. Shen, W.-F. Xu, and C.-Y. Li Cue-invariant detection of centre surround discontinuity by V1 neurons in awake macaque monkey J. Physiol., September 1, 2007; 583(2): 581 - 592. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. G. Appelbaum, A. R. Wade, V. Y. Vildavski, M. W. Pettet, and A. M. Norcia Cue-invariant networks for figure and background processing in human visual cortex. J. Neurosci., November 8, 2006; 26(45): 11695 - 11708. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||