Cerebral Cortex Advance Access originally published online on December 15, 2005
Cerebral Cortex 2006 16(11):1556-1565; doi:10.1093/cercor/bhj091
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Oscillatory Beta Activity Predicts Response Speed during a Multisensory Audiovisual Reaction Time Task: A High-Density Electrical Mapping Study
1 The Cognitive Neurophysiology Laboratory, Program in Cognitive Neuroscience, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY 10962, USA, 2 Program in Cognitive Neuroscience, Department of Psychology, The City College of the City University of New York, New York, NY 10031, USA
Address correspondence to Dr John J. Foxe, PhD, Nathan Kline Institute for Psychiatric Research, Program in Cognitive Neuroscience and Schizophrenia, 140 Old Orangeburg Road, Orangeburg, NY 10962, USA. Email: foxe{at}nki.rfmh.org.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
Bisensory redundant targets are processed faster than the respective unisensory target stimuli alone as evidenced by substantially faster reaction times (RTs). This multisensory RT facilitation has been interpreted as an expression of integrative processing between the different sensory modalities. However, the neuronal mechanisms underlying the RT facilitation effect are not well understood. Oscillatory responses in the beta frequency range (13–30 Hz) have been related to sensory-motor processing. Here, we investigated whether modulation of beta responses might also underlie the faster RTs seen for multisensory stimuli. Using high-density electrical mapping, we explored the association between early (50–170 ms) multisensory processing in the evoked beta response and RTs recorded during a simple RT task. Subjects were instructed to indicate the appearance of any stimulus in a stream of auditory-alone (A), visual-alone (V), and multisensory (AV) stimuli by a button press. Beta responses were analyzed using Morlet wavelet transformations. Multisensory interactions were found over frontal, occipital, central, and sensory-motor regions. Critically, beta activity correlated with mean RTs over all stimulus types. Significant negative correlations were found for frontal, occipital, and sensory-motor scalp regions. We conclude that the association between oscillatory beta activity and integrative multisensory processing is directly linked to multisensory RT facilitation effects.
Key Words: cross-modal • EEG • ERP • motor system • sensory integration
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
The integration of inputs from multiple sensory modalities can facilitate motor behavior. For example, a tennis player can estimate the speed, direction, and spin of a ball by integrating visual and auditory information when his or her opponent hits the ball. The visual inputs may allow the player to estimate the direction of the ball, whereas the sound can give him or her further information about the speed and spin of the ball. One illustration of the advantage incurred by multiple sensory inputs comes from studies showing that multisensory stimuli can be processed faster than the corresponding unisensory targets (Schröger and Widmann 1998
; Molholm and others 2002, 2004; Teder-Sälejärvi and others 2002; Murray and others 2005; Senkowski and others 2005; Talsma and Woldorff 2005). Although the reaction time (RT) facilitation effect is a robust finding and can be attributed, at least in some cases, to the occurrence of multisensory interactions (e.g., Molholm and others 2002; Patching and Quinlan 2004), the underlying neuronal mechanisms remain to be elucidated.
One mechanism that might be associated with the multisensory RT facilitation effect is neuronal processing in the beta frequency range (about 13–30 Hz). Studies using intracranial (Crone and others 1998) and scalp-surface recordings in humans (Classen and others 1998; Pfurtscheller, Graimann, and others 2003), as well as studies investigating local field potentials in monkeys (Liang and others 2002), have indicated that oscillatory beta activity is closely related to sensory-motor processing. Classen and others (1998) reported an increase of interregional coherence in the beta frequency over different scalp areas, including regions that have been associated with activity from the visual and motor cortices, when subjects performed a tracking task that included visual feedback. The authors suggested a functional relationship between beta activity and large-scale motor network processing.
Although a number of studies have examined the association between beta activity and motor behavior for unisensory stimuli, to our knowledge this has not been done for multisensory stimuli. One study did investigate multisensory integration effects for beta activity (among other frequency bands), but this was recorded while subjects passively experienced unisensory auditory, unisensory visual, and multisensory audiovisual stimuli (Sakowitz and others 2005). Because of the lack of a task and the fact that only 11 scalp channels were recorded, it is not possible to draw conclusions regarding the functional significance of the beta activity or the intracranial sources of the underlying generators, except in the most general terms. Nevertheless, in this study, audiovisual integration effects in the beta frequency range from data recorded over central and parietooccipital scalp regions were found, starting as early as 50 ms after stimulus onset. These showed increased activity for the multisensory compared with summed unisensory responses. Thus, the Sakowitz and others (2005) study revealed that multisensory interactions for auditory and visual stimuli do occur in the beta frequency range and that these interactions onset early in the cortical information-processing hierarchy.
The link between oscillatory beta activity and sensory-motor processing and evidence of multisensory effects on beta activity suggested that such activity may play an important role in the RT facilitation effect for multisensory stimuli. We tested this hypothesis by examining the correlations between mean RTs and evoked beta activity in high-density electrical recordings from a data set for which the event-related potentials (ERPs) have been previously reported (Molholm and others 2002). In this study, a response-time advantage was found for auditory and visual stimuli when presented together compared to when presented alone during a simple RT task in which the 3 stimulus types were presented in a randomly interleaved order. Significantly, examination of the ERPs elicited by these stimuli showed a number of multisensory integration effects, revealing a widespread network of multisensory activity, which began at the early latency of 46 ms over the right parieto-occipital scalp (Molholm and others 2002).
For the present analysis, we focused on activity that occurred prior to 200 ms, under the assumption that the multisensory effects on beta activity would have to occur relatively early poststimulus onset to affect motor behavior. We expected that putative beta effects would be generated, at least in part, in cortical regions that were involved in the sensory processing of the auditory and visual inputs. There is ample evidence from both human (Giard and Peronnet 1999; Foxe and others 2000; Calvert 2001; Foxe and Schroeder 2005) and nonhuman (Schroeder and others 2001, 2003) primate investigations of cortical multisensory interactions that occur relatively early in time and in cortical regions previously thought to be "unisensory" (for a recent review, see Schroeder and Foxe 2005); and what's more, a number of studies have shown evoked oscillatory beta activity over scalp regions that are traditionally associated with projections from sensory specific cortex (Tzelepi and others 2000; Haenschel and others 2000; Sakowitz and others 2005). In addition, because recent studies have indicated an association between frontal beta activity and sensory-motor processing (Liang and others 2002; Alegre and others 2004), we also inspected beta activity over frontal scalp sites and sensory-motor areas.
To summarize, the primary objective of the present study was to examine the relationship between multisensory interactions in the beta frequency range and motor processing. We therefore tested for correlations between RTs and evoked beta activity. We expected that higher amplitude beta activity would correlate with faster RTs primarily for those areas for which multisensory interactions were observed. Significantly, this addressed a mechanism by which the RT facilitation for bisensory redundant targets may be achieved. The positive results that we found support the notion that multisensory effects on beta activity may contribute to the RT facilitation effects that are commonly observed for multisensory stimuli.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
Subjects
Twelve neurologically normal, paid volunteers participated in the study (5 female and 11 right handed, mean age 23.8 ± 2.69). All subjects reported normal hearing and had normal or corrected-to-normal vision. Data from 2 additional subjects were excluded due to excessive eye movements and failure to maintain central fixation. The Institutional Review Board of the Nathan Kline Institute for Psychiatric Research approved the experimental procedures, and each subject provided written informed consent.
Stimuli and Procedure
The auditory stimulus consisted of a 1000-Hz tone (60-ms duration, 75-dB sound pressure level, 5-ms rise/fall times) presented from a single speaker located atop the monitor on which the visual stimuli were presented. The visual stimulus consisted of a red circular disk subtending 1.2 degrees in diameter presented on a black background for 60 ms. The location of the circle was individually adapted after completion of a pretest session in which the circle was presented from 16 different spatial locations (1.6 or 2.6 degrees laterally from fixation and 1.6 or 2.0 degrees vertically from fixation). One focus of the ERP study of Molholm and others (2002) was on the analyses of early integration effects on the visual C1 component in ERPs. This component is usually observed after about 50 ms poststimulus (e.g., Clark and Hillyard 1996; Foxe and Simpson 2002). For this reason, the spatial location for which the highest C1 amplitudes were observed was chosen for visual stimulation. In 11 of the 12 subjects, the highest C1-P1 amplitudes were found in the upper visual field at 1.6 degrees lateral left (n = 8) or lateral right (n = 3). Only 1 subject showed the highest C1-P1 amplitudes in the lower left visual field (2.6 degrees left, 2 degrees below fixation). Subjects were instructed to maintain central fixation at all times and to make a speeded button response with their right index finger when a stimulus in either sensory modality was detected (unisensory or multisensory stimuli). Sensory stimuli could appear in 3 different conditions: unisensory auditory (A), unisensory visual (V), or multisensory audiovisual (AV). The 3 stimulus conditions were presented with equal probability in a random order. Interstimulus interval varied randomly between 750 and 3000 ms. Each block consisted of 150 trials, and each subject completed a minimum of 18 blocks.
Analysis of RTs
For individual subjects, the response-time distributions were recorded, and mean RTs were calculated for each of the stimulus conditions. Response-time facilitation for the bisensory stimulus was followed up with a test of the so-called "Race model" (e.g., Miller 1982, 1991). The Race model tests whether simple statistical facilitation can account for the observed effects. This analysis has been reported in detail in Molholm and others (2002) and will be briefly recapitulated here.
Calculation of the Race Model
The Race model places an upper limit on the cumulative probability (CP) of RT at a given latency for stimulus pairs. For any latency, t, the Race model holds when this CP value is less than or equal to the sum of the CP from each of the single stimuli minus an expression of their joint probability (CP(t)simultaneous 
((CP(t)alone 1 + CP(t)alone 2) – (CP(t)alone 1 x CP(t)alone 2))). For each subject, the RT range within the valid RTs (100–800 ms) was calculated over the 3 stimulus conditions and divided into quantiles from the first to the hundredth percentile in 5% increments (1%, 5%, ..., 95%, 100%). T-tests comparing the actual (CP(t)simultaneous) and predicted ((CP(t)alone 1 + CP(t)alone 2) – (CP(t)alone 1 x CP(t)alone 2)) facilitation were performed to assess the reliability of violations of the Race model. Any violation of the Race model would indicate that the RT facilitation was at least partially due to interactions between the auditory and visual inputs.
The main assumption of the Race model is that each stimulus of a given stimulus pair (e.g., the auditory and visual stimulus of a multisensory audiovisual stimulus) independently competes for response initiation and that the faster of the two will trigger the response on a given trial. Due to the mechanism of 2 independently competing unisensory processing streams, mean RTs to multisensory stimuli will, by definition, be faster than mean RTs to unisensory stimuli alone, even without multisensory interactions. To draw a parallel, a bisensory stimulus is akin to having 2 rolls of a dice to hit a "six" rather than one, simply due to probability summation. The Race model allows one to test whether RTs to multisensory stimuli are faster again than the predictions of simple probability summation.
Data Acquisition
The electroencephalogram (EEG) was recorded from 2 electrooculogram (EOG) and 126 scalp electrodes (impedances < 5 k
), referenced to the nose, and band-pass filtered from 0.05 to 100 Hz at a sample rate of 500 Hz. In addition, data were filtered off-line with a 60-Hz notch filter (57.74–62.25 (–3 dB)). The purpose of this filter was to suppress the 60-Hz high-frequency noise. Averaging epochs for EEG beta activity lasted from 400 ms before to 800 ms after stimulus onset. Prior to averaging, baselines were computed for each trial from –300- to –100-ms prestimulus time interval. For artifact suppression, trials were automatically excluded from averaging if the standard deviation within a moving 200-ms time interval exceeded 30 µV in any one of the EEG channels and 40 µV at the EOG channels in a time interval between –300 and 500 ms. The reader should note that throughout this paper we will use the familiar nomenclature of the modified 10–20 electrode system to refer to the positioning of electrode sites (American Electroencephalographic Society 1991). Because our montage contains considerably more scalp sites than this nomenclature allows for, in some cases, we will be referring to the nearest neighboring site within the 10–20 system.
Morlet Wavelet Transformation
For the analysis of oscillatory beta activity, a wavelet transform based on Morlet wavelet was employed (Herrmann and others 1999; Senkowski and Herrmann 2002). An established method to examine multisensory interactions in ERP is the comparison of audiovisual (AV) responses with summed auditory plus visual (A + V) responses (Giard and Perronet 1999). However, as illustrated in Figure 1, the event-related oscillatory responses to unisensory auditory and unisensory visual stimuli (left panel) cannot simply be summed up after the wavelet transformation (right panel). As such, a comparison of the summed induced unisensory responses (i.e., activity that is stimulus related but not necessarily phase locked to the onset of events over trials) is not valid except in the very unlikely scenario that these responses are in perfect phase across all epochs. Thus, the lack of additivity for the (AV – (A + V)) comparison prevents us from considering induced beta responses for the analysis of multisensory interactions.
|
The additivity problem, however, can be solved for the evoked responses as follows: One first computes the sum of the event-related responses to unisensory auditory and unisensory visual stimuli and then calculates a wavelet transformation for these summed event-related responses (Fig. 1, lower panel left). The result of this transformation is the evoked activity (i.e., activity that is strictly phase and time locked to the onset of events) of the summed unisensory auditory plus unisensory visual (A + V) stimuli. The evoked beta activity of summed A + V responses was compared with the evoked beta activity of multisensory AV responses.
The frequency for the wavelet transformation was individually adapted for each subject using the frequency with the highest amplitudes in the early evoked oscillatory responses in the beta frequency range (13–30 Hz) to multisensory stimuli. This estimate was made over right frontal scalp (in the region of the F6' electrode site) because the maximum beta amplitudes to multisensory stimuli were observed in this channel. The average of individually adapted evoked beta activity was 17.8 Hz (range 15–23 Hz). The duration (2
t) of wavelet transformations with central frequencies of 18 Hz (f0 = 18) was 111 ms, resulting in a spectral bandwidth (2f) of 5.74 Hz.
Regions of Interests
To avoid the loss of statistical power inherent when repeated-measures analysis of variance (ANOVA) is used to quantify multichannel EEG data, our tests were limited to activity within 6 regions of interest (ROIs) (Fig. 2). We delineated a pair of bilateral occipital ROIs to reflect activity from occipital areas and 1 central ROI to reflect processing in auditory regions. Furthermore, because previous studies have shown a close association between beta responses and frontal scalp regions (Liang and others 2002; Alegre and others 2004), we also computed beta activity for a pair of bilateral frontal ROIs. Finally, we analyzed 1 scalp region consisting of channels located over the left sensory-motor cortex. This ROI was chosen to reflect processing of the sensory-motor system because subjects used their right index finger to respond to all stimulus presentations.
|
Time–Frequency Planes of Evoked Beta Activity
Figure 3 shows multisensory (AV) and summed auditory plus visual (A + V) ERP responses of a single subject. An event-related oscillatory pattern, which starts at about 70 ms after stimulus onset, with a frequency of about 19 Hz, can be observed over multiple scalp regions and particularly strongly in the right frontal scalp site (F6'). It should be noted that the oscillatory beta pattern is riding on lower frequency activity (in particular at occipital electrode sites) and therefore is not as clearly visible in the time-locked ERPs as it might be. Morlet wavelet transformations were calculated separately for each frequency in a range between 13 and 30 Hz for a frontal right (F6') and a frontal left (F5') electrode and plotted into time–frequency representations. Figure 4 displays the grand average time–frequency representations for the 12 subjects showing anterior effects that begin at about 70 ms poststimulus.
|
|
Analysis of Multisensory Interactions in the Evoked Beta Activity
For each of the 6 scalp ROIs, the mean values (over electrodes) of maximum evoked beta amplitudes in a time interval between 50 and 170 ms were computed and analyzed. First, we performed a single global ANOVA using the within-subject factors "ROI" (6 ROIs) and "stimulus type" (multisensory AV beta activity and auditory plus visual (A + V) beta activity) to establish whether a multisensory beta effect was evident. In the second phase of our analysis, we performed separate repeated-measures ANOVAs for the frontal and occipital ROIs, using the within-subject factors stimulus type (multisensory AV beta activity and auditory plus visual (A + V) beta activity) and "hemisphere" (left and right). Lastly, for the central ROI and for the left sensory-motor ROI, ANOVAs with the factor stimulus type (multisensory AV beta activity and auditory plus visual (A + V) beta activity) were performed.
Analysis of Beta Activity and RTs
The relationships between beta activity and RTs over the 3 stimulus types (unisensory A, unisensory V, and multisensory AV) were investigated by calculating their correlations (Pearson correlation) at the 6 ROIs. In addition, we calculated the RTs for single trials (separately for each subject and each modality) and compared the beta responses of the fastest 50% trials with the slowest 50%. This was done using an ANOVA with factors of "RTs" (fastest 50%, slowest 50%), "stimuli" (unisensory A, unisensory V, multisensory AV), and ROI (6 ROIs). The lack of additivity for the induced responses in the AV – (A + V) comparison does not preclude studying the relationships between induced beta responses and RTs. The comparison of beta response and RTs was therefore performed for both evoked and induced responses. In case of nonsphericity, all calculated ANOVAs in this work were adjusted with the Greenhouse–Geisser epsilon correction.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
Behavioral Responses
Fastest responses were observed for multisensory AV stimuli (255 ± 35 ms) followed by auditory (297 ± 48 ms) and visual stimuli (305 ± 31 ms). Multisensory stimuli were processed significantly faster than both unisensory auditory (t11 = 7.97, P < 0.001) and unisensory visual (t11 = 11.06, P < 0.001) stimuli. In addition, robust violations of the Race model were also observed such that fully 25% of all RTs to AV stimuli were faster than the predictions of simple probability summation (described in detail in Molholm and others 2002, p. 119). This violation of the Race model verifies that nonlinear multisensory integration processes must have taken place.
Beta Activity and Multisensory Integration
Evoked beta responses to visual stimuli peaked over occipital brain regions and beta responses to auditory stimuli over central regions (Figs. 5 and 6). Furthermore, for multisensory stimuli, we observed the strongest beta responses over frontal brain regions. Interestingly, the maximum differences between multisensory and summed auditory plus visual grand mean beta activity peaked between 90 and 130 ms across the ROIs. The pattern of beta activity showed general enhanced responses to multisensory as compared with summed auditory plus visual stimuli. We tested the differences between multisensory and summed auditory plus visual beta activity at the 6 ROIs in repeated-measures ANOVAs using the factor stimulus type (maximum of multisensory AV beta responses and summed (A + V) beta responses in a time interval between 50 and 170 ms). In addition, the factor hemisphere (left, right) was included in the ANOVAs for frontal and occipital ROIs.
|
|
For the global ANOVA, where all 6 ROIs were included, a significant main effect of the within-subject factor stimulus type was observed (F1,11 = 33.87, P < 0.001). This effect showed that evoked beta responses were greater for multisensory AV stimuli (0.89 µV) as compared with summed A + V stimuli (0.65 µV). No further effects were observed in this ANOVA. Then, planned follow-up analyses of individual ROIs were performed.
For the frontal ROIs, we found a significant main effect of the within-subject factors stimulus type (F1,11 = 7.36, P < 0.03) and hemisphere (F1,11 = 6.50, P < 0.03). Higher beta amplitudes were observed for multisensory AV stimuli (0.88 µV) as compared with summed A + V stimuli (0.66 µV) (Fig. 7). Furthermore, beta amplitudes were higher over the right (0.84 µV) than over the left frontal ROI (0.69 µV). No significant interaction between the factors stimulus type and hemisphere was found. Separate analyses of multisensory AV versus summed beta responses for the left and right hemisphere revealed significant effects of the factor stimulus type for the left (F1,11 = 8.35, P < 0.02) and the right (F1,11 = 5.11, P < 0.05) ROIs due to higher evoked beta amplitudes in response to multisensory AV stimuli.
|
The ANOVA for the occipital ROIs only approached significance for the within-subject factor stimulus type (F1,11 = 4.66, P < 0.06), suggesting a trend toward higher amplitudes for multisensory (0.98 µV) as compared with summed A + V stimuli (0.75 µV). No significant effect of the factor hemisphere or other effects were observed. We conducted a pair of post hoc ANOVAs to further investigate potential occipital effects. These separate analyses for the left and right occipital ROIs revealed a significant effect of the factor stimulus type for the right ROI (F1,11 = 8.83, P < 0.02) but not for the left ROI (F1,11 = 1.80, P = 0.21).
Next, multisensory interactions were investigated for the central ROI. The ANOVA for this ROI revealed a significant effect of the factor stimulus type (F1,11 = 11.21, P < 0.01) due to higher amplitudes for multisensory (0.85 µV) as compared with summed A + V beta responses (0.56 µV).
Finally, multisensory interactions over the left sensory-motor region were analyzed. The ANOVA revealed a significant main effect of the within-subject factor stimulus type (F1,11 = 21.32, P < 0.001), indicating higher amplitudes for multisensory (0.75 µV) as compared with summed A + V beta responses (0.53 µV).
Beta Activity and RTs
Figure 8 shows the correlations between RTs over the stimulus 3 types with evoked beta activity at the 6 ROIs. Significant negative correlations between the RTs and beta activity were observed for the left frontal, right frontal, right occipital, and left sensory-motor ROIs (left frontal: r = –0.44, P < 0.008; right frontal: r = –0.38, P < 0.03; right occipital: r = –0.44, P < 0.008; left sensory-motor: r = –0.346, P < 0.04; N = 36). These correlations show a clear relationship between higher amplitude beta activity and faster RTs. The correlations between the left occipital and central beta activity with RTs, however, did not approach significance (left occipital: r = –0.25, P = 0.15; central: r = –0.25, P = 0.15; N = 36).
|
The ANOVA for the comparison of the evoked responses of the 50% fastest RTs with the evoked responses of the 50% slowest RTs revealed a significant main effect of the within-subject factor RTs (F1,11 = 5.04, P < 0.05) due to higher evoked beta responses to fast (0.62 µV) as compared with slow (0.55 µV) RTs. This finding further supports the assumption of a direct link between RTs and evoked oscillatory beta activity. No other effects or interactions were observed in relation to the factor RTs.
The same analyses were performed for induced beta responses. For these responses, no correlations with RTs were found. In addition, there were no differences in induced beta responses between trials with the 50% fastest and trials with the 50% slowest RTs. We performed a further analysis to test whether induced responses differ between trials with the 25% fastest and the 25% slowest RTs. Again, this analysis did not reveal any differences between induced responses to fast and slow trials.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
Here we show robust audiovisual multisensory interactions in the evoked oscillatory beta band, which occur early during sensory processing. These interactions were mainly seen over scalp regions also activated by the unisensory stimuli and peaked substantially before the overt motor response. Consistent with the proposed link between multisensory interactions for beta activity and response-time facilitation, correlation analysis demonstrated that the evoked beta responses are predictive of RT speed over the different stimulus types. We discuss these findings in detail below.
Multisensory Effects on Beta Activity
Multisensory effects were found to be widespread, with enhanced beta activity for multisensory compared with summed unisensory evoked responses over a number of ROIs. These were specifically over the bilateral frontal ROIs, the central ROI, the right occipital ROI, and the left sensory-motor regions. The multisensory interactions occurred in the same general time frame across the different ROIs, with differences between the multisensory and summed responses peaking around 120 ms, as illustrated in the wavelet-transformed evoked beta activity depicted in Figure 5. Thus, the data reveal a network of multisensory interactions occurring within a relatively restricted time range in the stimulus-evoked oscillatory beta activity.
The multisensory effects appeared to be largely due to the amplification of unisensory activity. More specifically, the responses to auditory stimuli when presented alone showed peak beta activity largely over central scalp regions (as in Haenschel and others 2000; Makinen and others 2004), and there was an increase in this activity when the same auditory stimulus was paired with the visual stimulus. The auditory stimuli also resulted in clear activity focused over right frontal scalp and relatively lower amplitude activity focused over left frontal scalp. Again, across these regions, this activity was greater when the auditory stimulus was paired with the visual stimulus. Similarly, the beta responses to visual stimuli presented alone resulted in peak beta activity largely over right occipital scalp, and there was an increase in this activity when the same visual stimulus was paired with the auditory stimulus. In addition to these effects over sensory regions, evoked beta responses were also enhanced over the sensory-motor scalp region, further supporting the assumption that the multisensory interactions reported here are likely to be linked to sensory-motor–related processing.
The multisensory interactions in the evoked beta response showed some similarities to the ERP results reported by Molholm and others (2002). As in the present study, Molholm and others reported a multisensory interaction over right frontal scalp, and this was found within the same time frame (starting at about 100 ms). However, the frontal interaction in the ERP appeared to be subadditive (i.e., the audiovisual (AV) ERP was lower in amplitude than the summed (A + V) ERP), whereas the interaction seen in the frontal beta activity was superadditive (i.e., audiovisual (AV) beta activity was higher than the summed (A + V) beta activity). Although "subadditivity" in the ERP does not necessarily indicate a diminution of activity within a generator configuration (a new generator with opposite scalp polarity could result in such subadditivity), this does raise the possibility that the frontal effects on evoked beta activity and ERPs actually reflect distinct neuronal processes that may be involved in different aspects of cortical multisensory processing.
Finally, it is of note that the multisensory interactions and correlations to behavior were observed over the right but not over the left occipital ROI. This observation is in line with previous ERP studies that have reported stronger early-latency integration effects over right occipital than over left occipital cortical regions (Giard and others 1999; Fort and others 2002; Molholm and others 2002) and may suggest some degree of hemispheric specialization for early audiovisual multisensory integrations.
Oscillatory Beta Activity and RT Performance
Correlation analysis revealed that higher amplitude–evoked beta activity was significantly related to faster RTs for bilateral frontal, right occipital, and sensory-motor ROIs. This suggests that early evoked beta activity is indeed related to motor behavior and that multisensory interactions in the beta frequency range can result in faster RTs. Further supporting a link between evoked beta activity and RTs, higher evoked beta activity was associated with trials with short RTs compared to trials with long RTs.
It is also important to consider that even in the absence of a motor task, such beta activity is present and modulated by multisensory inputs (Sakowitz and others 2005) and that further, for the present study, not all beta activity that was analyzed was shown to correlate with RTs (i.e., the central ROI did not). As such, we argue that these multisensory effects in the evoked beta responses are likely associated with subsequent cortical mechanisms directly involved in sensory-motor processing but that not all the beta activity examined here directly reflects the motor processes themselves. However, overall, the present correlation analyses revealed a robust pattern of results that suggests a close association between early evoked beta responses and RTs at the same ROIs for which we observed multisensory interactions.
It could be argued that the present results can be explained in terms of overall stimulus energy rather than a multisensory integration effect per se. For example, one might posit that the same effect could be found if 2 visual stimuli were used simultaneously or 2 auditory stimuli rather than stimuli across 2 sensory systems. However, the extant literature argues otherwise. That is, the violations of the Race model seen for multisensory stimuli are not observed for multiple copies of unisensory stimuli. For example, Murray and others (2001) assessed RT effects as a function of 1 versus 2 copies of a visual stimulus (both within and across hemifields) and although RTs sped up (the so-called redundant target effect), they did so in keeping with a simple probability summation account—that is, there was no nonlinear shortening of RT for unisensory pairs of stimuli. This has been a consistent finding in the literature (e.g., Meijers and Eijkman 1977; Reuter-Lorenz and others 1995; Corballis 1998). In addition, we observed higher beta activity for trials with short RTs as compared with trials with long RTs in an overall analysis including the 3 stimulus conditions. This further demonstrates the direct association between evoked beta activity and RTs, which cannot be explained by differences in stimulus energy across conditions.
Comparison with Beta Activity Reported in Other Studies
There are some major differences between the beta responses observed here and the beta responses that have been reported in a number of previous studies (e.g., Pfurtscheller and Lopes da Silva 1999; Feige and others 2000; Pfurtscheller, Woertz, and others 2003). In part, this is due to different approaches to data analysis. For instance, in some of these studies, beta activity has been investigated during the execution (Feige and others 2000; Pfurtscheller, Graimann, and others 2003) or observation (Babiloni and others 2002) of a motor response, whereas here we investigated early activity that was observed clearly before the motor response. In these studies looking at induced beta activity, a common finding is a decrease in induced responses during and after the execution of a movement, and this is typically seen over sensory-motor areas (Neuper and Pfurtscheller 2001; Babiloni and others 2002). The absence of a relationship between early-induced beta responses and RTs in our study, however, suggests that these early responses are not substantially linked to the response-time facilitation effect observed for multisensory stimuli. Similarly, it has been previously shown that early latency effects of different experimental manipulations such as attention and memory are primarily reflected in the evoked and not in the induced responses (Senkowski and Herrmann 2002; Herrmann and others 2004; Busch and others 2005; Senkowski and others 2005).
Evoked beta responses have been examined with respect to RTs, with several studies reporting an increase in evoked beta activity after the offset of a movement (Pfurtscheller 1992; Pfurtscheller, Woertz, and others 2003). The observation of postmovement-evoked beta responses led Neuper and Pfurtscheller (2001) to hypothesize that these responses may reflect a decreased excitability of cortical networks in the motor system. However, the beta responses in the present study were observed prior to the behavioral motor responses, and it was higher beta amplitude that was related to faster RTs. As such, our study appears to be at odds with their prediction. Rather, our results suggest that evoked beta activity prior to a motor response is more likely to reflect increased activation.
Finally, intercortical coherence in beta activity has been associated with multisensory processing. One study showed that intercortical beta coherence was increased between posterior regions of parietal and temporal scalp for stimuli representing objects, whether presented as images, spoken words, or written words (von Stein and others 1999). This activity was interpreted as reflecting supramodal aspects of the stimuli, further supporting the assumption that beta activity plays an important role in multisensory audiovisual processing.
This work focused specifically on the analysis of beta activity in the context of the RT facilitation effect on multisensory stimuli. The motivation for choosing beta responses for the present analysis was that this activity has been most frequently related to sensory-motor processing and RTs (e.g., Crone and others 1998; Liang and others 2002; Pfurtscheller, Woertz, and others 2003) and was also very recently related to audiovisual multisensory processing (Sakowitz and others 2005). We should point out though that it is also possible that activity within other frequency bands, not investigated here, may also be involved in these processes. For example, multisensory interactions have also been observed in the gamma and theta bands (Sakowitz and others 2000, 2001; Senkowski and others 2005) and another frequency band that has been consistently related to behavioral performance is the alpha band (e.g., Foxe and others 1998; Klimesch 1999; Fu and others 2001; Klimesch and others 2003). As such, future studies should address how oscillatory activity in these frequency bands might also relate to multisensory facilitation effects.
The Use of Wavelet Analysis to Investigate Multisensory Processing
Previous studies have shown that ERPs can be displayed as a superposition of electrical responses in different frequency ranges (Karakas and others 2000; Basar and others 2001), and as such, multisensory processing within a specific frequency may be obscured in the time-locked ERP. One reason for this is that evoked beta responses will partially cancel out in the grand average ERPs when they differ in phase over subjects. Another reason is that event-related oscillatory responses (in particular those in higher frequencies) are often superimposed by activity in lower frequency ranges. The Morlet wavelet transformation that was employed here for the analysis of multisensory interactions, however, is insensitive to phase differences of beta activity over subjects and allowed us to specifically extract beta responses in the time and the frequency domains. Thus, the application of the wavelet transformation might allow a more accurate examination of multisensory interactions in the beta frequency range than the analyses of ERPs alone does. Here we show that examination of evoked oscillatory activity can be highly useful in understanding the relationship between cortical multisensory processing and behavior.
|
Summary and Conclusions |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
To our knowledge this is the first study to describe a close association between early evoked beta activity and RTs during multisensory processing. We observed early multisensory interactions in the beta frequency range over multiple scalp areas, suggesting a widespread network of cortical regions that begins to oscillate at a similar latency after the onset of an audiovisual stimulus. Consistent with a link between multisensory interactions for beta activity and response-time facilitation, beta activity at most of the regions that showed multisensory interactions also predicted the average RTs across the different stimulus types. We propose that evoked beta activity plays an important role in early multisensory integrative processing and that such processing is linked to the sensory-motor system.
|
Acknowledgments |
|---|
This work was supported by grants from the National Institute of Mental Health (MH65350) and the National Institute on Aging (AG22696) to JJF. We would also like to thank two anonymous reviewers for their helpful comments on this article.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSummary and ConclusionsReferences |
|---|
Alegre M, Gurtubay IG, Labarga A, Iriarte J, Valencia M, Artieda J. (2004) Frontal and central oscillatory changes related to different aspects of the motor process: a study in go/no-go paradigms. Exp Brain Res 159:14–22.[Web of Science][Medline]
American Electroencephalographic Society. (1991) American Electroencephalographic Society guidelines for standard electrode position nomenclature. Clin Neurophysiol 8:200–202.
Babiloni C, Babiloni F, Carducci F, Cincotti F, Cocozza G, Del Percio C. (2002) Human cortical electroencephalography (EEG) rhythms during the observation of simple aimless movements: a high-resolution EEG study. Neuroimage 17:559–572.[CrossRef][Web of Science][Medline]
Basar E, Schurmann M, Demiralp T, Basar-Eroglu C, Ademoglu A. (2001) Event-related oscillations are ‘real brain responses’—wavelet analysis and new strategies. Int J Psychophysiol 39:91–127.[CrossRef][Web of Science][Medline]
Busch NA, Schadow I, Frund I, Herrmann CS. ( October 20, 2005) Time-frequency analysis of target detection reveals an early interface between bottom-up and top-down processes in the gamma-band. Neuroimage [Epub ahead of print].
Calvert GA. (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123.
Clark VP and Hillyard SA. (1996) Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. J Cogn Neurosci 8:387–402.
Classen J, Gerloff C, Honda M, Hallett M. (1998) Integrative visuomotor behavior is associated with interregionally coherent oscillations in the human brain. J Neurophysiol 79:1567–1573.
Corballis MC. (1998) Interhemispheric neural summation in the absence of the corpus callosum. Brain 121:1795–1807.
Crone NE, Miglioretti DL, Gordon B, Sieracki JM, Wilson MT, Uematsu S, Lesse RP. (1998) Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization. Brain 121:Pt 122271–2299.
Feige B, Aertsen A, Kristeva-Feige R. (2000) Dynamic synchronization between multiple cortical motor areas and muscle activity in phasic voluntary movements. J Neurophysiol 84:2622–2629.
Fort A, Delpuech C, Pernier J, Giard MH. (2002) Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031–1039.
Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE. (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10:77–83.[CrossRef][Medline]
Foxe JJ and Schroeder CE. (2005) The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16:419–423.[CrossRef][Web of Science][Medline]
Foxe JJ and Simpson GV. (2002) Flow of activation from V1 to frontal cortex in humans. A framework for defining "early" visual processing. Exp Brain Res 142:139–150.[CrossRef][Web of Science][Medline]
Foxe JJ, Simpson GV, Ahlfors SP. (1998) Cued shifts of intermodal attention: parieto-occipital 
10 Hz activity reflects anticipatory state of visual attention mechanisms. Neuroreport 9:3929–3933.[Web of Science][Medline]
Fu KG, Foxe JJ, Murray MM, Higgins BA, Javitt DC, Schroeder CE. (2001) Attention-dependent suppression of distracter visual input can be cross-modally cued as indexed by anticipatory parieto-occipital alpha-band oscillations. Brain Res Cogn Brain Res 12:145–152.[CrossRef][Medline]
Giard MH and Peronnet F. (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490.[CrossRef][Web of Science][Medline]
Haenschel C, Baldeweg T, Croft RJ, Whittington M, Gruzelier J. (2000) Gamma and beta frequency oscillations in response to novel auditory stimuli: a comparison of human electroencephalogram (EEG) data with in vitro models. Proc Natl Acad Sci USA 97:7645–7650.
Herrmann CS, Mecklinger A, Pfeifer E. (1999) Gamma responses and ERPs in a visual classification task. Clin Neurophysiol 110:636–642.[CrossRef][Web of Science][Medline]
Herrmann CS, Munk MH, Engel AK. (2004) Cognitive functions of gamma-band activity: memory match and utilization. Trends Cogn Sci 8:347–355.[CrossRef][Web of Science][Medline]
Karakas S, Erzengin OU, Basar E. (2000) The genesis of human event-related responses explained through the theory of oscillatory neural assemblies. Neurosci Lett 285:45–48.[CrossRef][Web of Science][Medline]
Klimesch W. (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res Brain Res Rev 29:169–195.[CrossRef][Medline]
Klimesch W, Sauseng P, Gerloff C. (2003) Enhancing cognitive performance with repetitive transcranial magnetic stimulation at human individual alpha frequency. Eur J Neurosci 17:1129–1133.[CrossRef][Web of Science][Medline]
Liang H, Bressler SL, Ding M, Truccolo WA, Nakamura R. (2002) Synchronized activity in prefrontal cortex during anticipation of visuomotor processing. Neuroreport 13:2011–2015.[CrossRef][Web of Science][Medline]
Makinen VT, May PJ, Tiitinen H. (2004) Human auditory event-related processes in the time-frequency plane. Neuroreport 15:1767–1771.[CrossRef][Web of Science][Medline]
Mejers LM and Eijkman EG. (1997) Distributions of simple RT with single and double stimuli. Percept Psychophys 22:41–48.
Miller J. (1982) Divided attention: evidence for coactivation with redundant signals. Cognit Psychol 14:247–279.[CrossRef][Web of Science][Medline]
Miller J. (1991) Channel interaction and the redundant-targets effect in bimodal divided attention. J Exp Psychol Hum Percept Perform 17:160–169.[CrossRef][Web of Science][Medline]
Molholm S, Ritter W, Javitt DC, Foxe JJ. (2004) Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14:452–465.
Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ. (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115–128.[CrossRef][Medline]
Murray MM, Foxe JJ, Higgins BA, Javitt DC, Schroeder CE. (2001) Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study. Neuropsychologia 39:828–844.[CrossRef][Web of Science][Medline]
Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ. (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 7:963–974.
Neuper C and Pfurtscheller G. (2001) Event-related dynamics of cortical rhythms: frequency-specific features and functional correlates. Int J Psychophysiol 43:41–58.[CrossRef][Web of Science][Medline]
Patching GR and Quinlan PT. (2004) Cross-modal integration of simple auditory and visual events. Percept Psychophys 66:131–140.[Web of Science][Medline]
Pfurtscheller G. (1992) Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalogr Clin Neurophysiol 83:62–69.[CrossRef][Web of Science][Medline]
Pfurtscheller G, Graimann B, Huggins JE, Levine SP, Schuh LA. (2003) Spatiotemporal patterns of beta desynchronization and gamma synchronization in corticographic data during self-paced movement. Clin Neurophysiol 114:1226–1236.[CrossRef][Web of Science][Medline]
Pfurtscheller G and da Silva FH Lopes. (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857.[CrossRef][Web of Science][Medline]
Pfurtscheller G, Woertz M, Supp G, da Silva FH Lopes. (2003) Early onset of post-movement beta electroencephalogram synchronization in the supplementary motor area during self-paced finger movement in man. Neurosci Lett 339:111–114.[CrossRef][Web of Science][Medline]
Reuter-Lorenz PA, Nozawa G, Gazzaniga MS, Hughes HC. (1995) Fate of neglected targets: a chronometric analysis of redundant target effects in the bisected brain. J Exp Pyschol Hum Percept Perform 21:211–230.
Sakowitz O, Quian Quiroga R, Schurmann M, Basar E. (2005) Spatio-temporal frequency characteristics of intersensory components in audiovisual evoked potentials. Brain Res Cogn Brain Res 23:316–326.[CrossRef][Medline]
Sakowitz OW, Quian Quiroga R, Schurmann M, Basar E. (2001) Bisensory stimulation increases gamma-responses over multiple cortical regions. Brain Res Cogn Brain Res 11:267–279.[CrossRef][Medline]
Sakowitz OW, Schurmann M, Basar E. (2000) Oscillatory frontal theta responses are increased upon bisensory stimulation. Clin Neurophysiol 111:884–893.[CrossRef][Web of Science][Medline]
Schroeder CE and Foxe JJ. (2005) Multisensory contributions to low-level, ‘unisensory’ processing. Curr Opin Neurobiol 15:454–458.[CrossRef][Web of Science][Medline]
Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC. (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327.
Schroeder CE, Smiley J, Fu KG, McGinnis T, O'Connell MN, Hackett TA. (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int J Psychophysiol 50:5–17.[CrossRef][Web of Science][Medline]
Schröger E and Widmann A. (1998) Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 35:755–759.[CrossRef][Web of Science][Medline]
Senkowski D and Herrmann CS. (2002) Effects of task difficulty on evoked gamma activity and ERPs in a visual discrimination task. Clin Neurophysiol 113:1742–1753.[CrossRef][Web of Science][Medline]
Senkowski D, Talsma D, Herrmann CS, Woldorff MG. (2005) Multisensory processing and oscillatory gamma responses: effects of spatial selective attention. Exp Brain Res 166:411–426.[CrossRef][Web of Science][Medline]
Talsma D and Woldorff MG. (2005) Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J Cogn Neurosci 17:1098–1111.[CrossRef][Web of Science][Medline]
Teder-Sälejärvi WA, McDonald JJ, Di Russo F, Hillyard SA. (2002) An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res 14:106–114.[CrossRef][Medline]
Tzelepi A, Bezerianos T, Bodis-Wollner I. (2000) Functional properties of sub-bands of oscillatory brain waves to pattern visual stimulation in man. Clin Neurophysiol 111:259–269.[CrossRef][Web of Science][Medline]
von Stein A, Rappelsberger P, Sarnthein J, Petsche H. (1999) Synchronization between temporal and parietal cortex during multimodal object processing in man. Cereb Cortex 9:137–150.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Gingras, B. A. Rowland, and B. E. Stein The Differing Impact of Multisensory and Unisensory Integration on Behavior J. Neurosci., April 15, 2009; 29(15): 4897 - 4902. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||