Cerebral Cortex Advance Access originally published online on December 15, 2005
Cerebral Cortex 2006 16(11):1566-1570; doi:10.1093/cercor/bhj093
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The Independence of Memory Traces of Attended and Unattended Stimuli
Brain & Behaviour Research Institute and Department of Psychology, University of Wollongong, Wollongong 2522, Australia
Address correspondence to David N. McKenzie. Email: dnm54{at}uow.edu.au.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
In a dichotic paradigm, duration-deviant mismatch negativity (MMN) largely reflects standard stimuli presented to the same ear as the deviant, suggesting an independent representation for each ear. We sought to assess this representation independence. Twenty-two participants, presented left (all 2000 Hz, 70-dB sound pressure level [SPL]) and right (all 800 Hz, 85-dB SPL) ear sounds, attended to 1 ear per block. A series of successive trains of 6–30 sounds were presented to each ear. Although 1 ear received short (40 ms) standard sounds, the other received an equal number of long (120 ms) standards. In the next train, the standard durations were switched between the ears. Stimulus onset asynchrony ranged from 120 to 440 ms. Duration deviants (240 ms) replaced the final standards of some trains. MMN latency simultaneously moved earlier for the ear changing from long to short standards and later for the ear changing from short to long standards. Based on a simple linear model, more than 80% of the memory trace reflected within-channel standards. We conclude that independently changeable memory traces underlie dichotic presentation. Separation of the representation for the attended subset of stimuli is proposed as core to the mechanism of sustained attention.
Key Words: attention • dichotic presentation • duration MMN • MMN amplitude • MMN latency
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
When a stimulus of novel duration (deviant) follows a train of same duration stimuli (standards), it generates the mismatch negativity (MMN) component of the event-related potential. Duration MMN is elicited by both duration increases and decreases (Näätänen and others 1989
). Its amplitude reflects the magnitude of duration change, implying that a representation of prior stimulus durations underlies its generation (Näätänen 1990).
MMN manifests as a negative voltage at frontal sites but is positive at sites below the Sylvian fissure when referenced to the nose (Alho and others 1986). This characteristic inversion is often used in the identification of the MMN. Data are typically collected at the mastoids for this purpose. MMN from these sites is also largely free of the overlapping fronto-central negative component, N2b, generated to attended (and sometimes unattended) stimuli (Näätänen and others 1982).
Because duration deviance cannot occur prior to the offset of the shorter stimulus, duration MMN latency may be manipulated by altering the stimulus durations. Displacement of MMN peak latency well beyond the latency range considered here has been demonstrated (Näätänen and others 2004).
Using this manipulation, Winkler and others (1996) presented participants with six 450-ms standards immediately followed by zero, two, four, or six 150-ms standards to their right ear. Subjects performed a visual tracking task and ignored the stimuli. A subsequent 300-ms deviant stimulus, also to the right ear, produced evidence of 2 consecutive MMNs. As the number of intervening short standards increased from 0 to 6, the amplitude of the short latency MMN increased while the longer latency MMN amplitude decreased. This suggests that 2 preceding-stimulus representations were maintained, both contributing to the MMN.
In a similar study with a different outcome, Ritter and others (2000) presented participants with 100-ms left ear standards, 300-ms right ear standards, and 200-ms duration deviants (to either ear) while they read self-selected literature. Left ear deviants elicited short latency MMN but no evidence of a longer latency MMN. Right ear deviants produced a longer latency MMN without evidence of an earlier MMN. That is, although 2 stimulus representations were again maintained, MMN for deviants to either ear reflected only the standards presented to that ear. The departure of this study from earlier results was explained in terms of streaming early in auditory processing.
In an unpublished study, we replicated the results of Ritter and others (2000) but also found that participants reported left ear sounds to be more intrusive than right ear sounds. This is consistent with the left ear advantage for pure tones in the general population (Pendse 1977). The conjunction of Ritter and others' use of a 10-dB left ear intensity advantage and a task for participants that allowed unallocated auditory attention to be captured by sounds to the preferred ear produced, we argue, de facto control of attention. Consistent with the suggestion that attention affects streaming (Carlyon and others 2001), we believe that this attention differential is responsible for Ritter and others' result: attended MMN reflects attended standards and unattended MMN reflects unattended standards.
Regardless of their source, 2 distinguishable stimulus representations are necessary for the 2 MMNs. However, the purity of these representations remains unclear. Do they reflect only the standard stimulus delivered to that ear, or is there a contribution from standards presented to the opposite ear? We sought to estimate the contribution of within-channel standards to the within-channel representations.
A rapid train of short-duration sounds was presented to 1 ear and, simultaneously, an equal number of long-duration sounds to the other ear. Without pause, these standard durations were then swapped between the ears. The sound sequence switched from short- to long-duration sounds in 1 ear and from long to short sounds in the other. Each ear received an equal number of the new sounds before the durations were swapped back. This swapping continued throughout the block. Participants responded to duration deviants presented immediately prior to the duration swapping. MMN latency was predicted to reflect only the preceding standard sounds to the same ear—that is, MMN latency should simultaneously decrease in 1 ear and increase in the opposite ear.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Participants
Twenty-two undergraduates participated for either course credit or cash payment (A$ 50). One was discontinued after she repeatedly exhibited drowsiness. The remaining 21 participants were predominantly female (17 females, 4 males), young (average 19.9 years), and right handed (2 left, 1 ambidextrous). All reported normal hearing and gave written informed consent for participation consistent with the University of Wollongong Human Research Ethics Committee guidelines.
Stimulus Sequence
All stimuli were pure tones with 8-ms linear rise and fall times. Left ear sounds were all 70-dB SPL and 2000 Hz, whereas right ear sounds were all 85-dB SPL and 800 Hz. Standard stimulus durations of 40 (shorts) and 120 ms (longs) were used for both ears. The deviant for both ears was 240 ms.
Within either ear, standard stimuli of a single duration were arranged into trains of between 6 and 30 presentations. Whereas 1 ear was presented short standards, the other ear received an equal number of long standards. Total train duration was made identical in both ears by using the same set of randomly selected stimulus onset asynchronies (120–440 ms in 1-ms steps, rectangular distribution), randomly ordered for each ear.
A train of longs immediately followed the short standard train, whereas the other ear switched from long stimuli to shorts. First stimuli of the new trains commenced at the same time in both ears. Average stimulus rate was identical in both ears.
The last stimulus of a randomly selected half of left ear short trains was replaced with the duration deviant, whereas the simultaneous right ear long train remained unchanged. The other half of left ear short trains remained unchanged, whereas the last stimulus of the accompanying right ear long train was replaced with the deviant. A similar pattern of replacement was applied to the left ear long trains and their accompanying right ear short sequences, as illustrated in Figure 1.
|
Within a block, each ear received 432 sounds as 24 trains. Average train length across the entire block was forced to 18 sounds. Half the trains of each ear contained a deviant stimulus in the final position (i.e., 12 deviants and 204 standards) to give a ratio of 1:17 for deviants to standards (5.6%). The other half of the trains added a further 216 standards to give a grand total of 420 standards for each ear per block. Each block took
2 min. Thirty-two blocks were presented in total. Experimental Procedure
Participants, seated in front of a personal computer screen in a brightly illuminated and sound-attenuating booth, were read instructions for the task. Two practice blocks were administered. Throughout each block, a large arrow on the screen indicated the ear to attend, and a brief sentence described the target sound.
Participants pressed a button with their right index finger to deviant stimuli in the designated ear. A response between 200 and 1000 ms after a target elicited a brief display of a green tick on the screen. A response at any other time elicited a red cross.
Data were collected from 19 sites of an electrode cap, the earlobes (A1 and A2), both mastoids (M1 and M2), and the tip of the nose. Horizontal and vertical electrooculograms (HEOG and VEOG, respectively) were recorded. The right ear served as the reference. All channels were sampled at 512 Hz.
The data were re-referenced to the nose, band-pass filtered between 0.5 and 25 Hz at a 24-dB/octave roll-off, and segmented into sweeps based on an epoch from 100 ms preonset to 500 ms postonset. A sweep was rejected if any point for any electrode (or HEOG and VEOG) departed from the average across the epoch by more than 50 µV. Surviving sweeps were baseline corrected to the average of the 100-ms prestimulus interval and separately averaged for each combination of standard duration, level of attention, and ear. Difference waves were generated by subtracting the relevant average standard waveform from the average deviant waveform.
Data Reduction and Statistical Analysis
For each ear, measures of sensitivity (A') and criterion (B'') were derived (Grier 1971) and, along with the reaction time (RT) data, subjected to one-way multivariate repeated-measures analysis of variance (ANOVA) with the factor of standard duration (short vs. long).
Peak MMN latencies were automatically selected as the most positive point within defined latency ranges (shorts, 150–250 ms; longs, 200–300 ms) for each of the electrodes M1, M2, A1, and A2. "Peaks" identified at the extremities of the latency range were reassessed. Those at the start of the short range or end of the long range were moved to the nearest positive extreme within the range; those at the end of the short range or start of the long range were moved up to 25 ms outside the range limit to a local positive extreme. Latencies were averaged across the 4 sites within each combination of ear, attention, and standard duration. Averages within each ear were subjected to multivariate repeated-measures ANOVA with factors of attention (attended vs. unattended) and standard duration (short vs. long). All F tests had (1,20) degrees of freedom.
Peak MMN amplitudes within each condition were calculated by averaging the difference wave amplitude at M1, M2, A1, and A2 across a 25-ms window centered on the grand average peak latency for that condition and subjected to the same analysis as the latency data.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Behavioral Results
An identical pattern of results was found for responses to left and right ear deviant stimuli: A' significantly increased from the long to the short standard trains (left: F = 4.84, P < 0.05, 
2 = 0.20; right: F = 6.54, P < 0.05, 2 = 0.25); B'' also significantly increased (left: F = 7.78, P < 0.05, 2 = 0.28; right: F = 32.81, P < 0.01, 2 = 0.62); and RT significantly decreased (left: F = 38.15, P < 0.01, 2 = 0.67; right: F = 64.45, P < 0.01, 2 = 0.76).
Deviant Waveforms
Fronto-central MMN was found in all grand average difference waveforms (see Figs. 2 and 3), although with small amplitude for the unattended right ear after long standards. Positive mastoid activity at the same latency was evident in all conditions.
|
|
Examination of the peak mastoid and earlobe electrode activity revealed that for the 150- to 250-ms short-peak selection window, automatic peak detection placed 2.8% of the peaks at the start of the window and 4.5% of the peaks at the end of the window. As described above, all these peaks were moved later. The opposite adjustment was applied to peaks placed at the extremes of the 200- to 300-ms long-peak selection window with 1.1% occurring at the start of the window and 8.5% at the end of the window. These peaks were moved earlier. All these changes reduced the likelihood of an earlier MMN after short standards than after long standards.
Nevertheless, for left ear stimuli, average MMN peaked significantly earlier following short (209.0 ms) than long standards (258.8 ms; F = 107.47, P < 0.001, 2 = 0.84). There was no main effect for attention (F = 1.80, P > 0.05). Attended and unattended latencies after long standards differed by only 4.9 ms. This difference was larger following short standards (203.7 ms for attended vs. 217.4 ms for unattended) and was sufficient to produce a significant duration by attention interaction (F = 6.32, P < 0.05, 2 = 0.24).
Similarly, for right ear stimuli, the short standards produced a significantly earlier MMN (208.7 ms) than the long standards (259.0 ms; F = 119.65, P < 0.001, 2 = 0.86). Neither the main effect for attention nor its interaction with duration was significant (both F < 1). Both had low observed powers (0.06 and 0.14, respectively).
Left ear MMN peak amplitude was larger following short (1.48 µV) than long standards (0.96 µV; F = 5.51, P < 0.05, 2 = 0.61). This amplitude was also larger for attended (1.37 µV) than for unattended deviants (1.07 µV), but the difference was not significant (F = 1.62, P > 0.05). There was also no attention by duration interaction (F < 1). Observed powers were 0.23 and 0.07, respectively. Right ear amplitudes were also larger after short (1.22 µV) than long standards (0.59 µV; F = 4.76, P < 0.05, 2 = 0.19), but additionally, they were larger when attended (1.29 µV) than unattended (0.53 µV; F = 7.39, P < 0.05, 2 = 0.27). The interaction of attention and duration was also significant (F = 8.46, P < 0.05, 2 = 0.30).
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
We presented 2 readily discriminated sets of sounds (1 of 2000-Hz stimuli to the left ear; the other of 800-Hz stimuli to the right ear) and required participants to respond to unpredictable duration changes in 1 sound set. Behavioral results showed that participants diligently attended the required sounds, which were presented rapidly, preventing covert attention to the "unattended" sounds. Within each set of sounds, we alternated between trains of short- and long-duration standards such that standard tone duration increased in 1 set as it decreased in the other. Occasional duration deviants within either sound set generated a single MMN, with latency reflecting standard duration within that set, consistent with previous research (Ritter and others 2000
) showing 2 representations underlying MMN in dichotic listening. This extended the earlier result in 2 ways. First, by showing that MMN latency for each set changed to reflect only the changes within that set, we demonstrated simultaneous but opposite changes of MMN latency between the 2 sets of sounds. Because other factors known to alter duration MMN latency were controlled, we conclude that the simultaneous but opposite latency changes reflected changes in 2, largely independent, stimulus duration traces, one reflecting attended sounds and the other reflecting unattended sounds.
Second, our MMN amplitude data showed a significant amplitude increase for attended sounds to the right ear with respect to unattended right sounds but only a tendency toward attentive enhancement for stimuli to the left ear. Perhaps, the left ear advantage for tones in our predominantly right-handed participants (Pendse 1977) facilitated production of genuinely unattended right ear stimuli and a larger attention differential (Ofek and Pratt 2004). Evidence for attentive enhancement of the duration MMN at fronto-central sites has been previously presented by Alain and Woods (1994), consistent with strong evidence for an attention effect on the intensity MMN amplitude (Woldorff and others 1991, 1998; Alain and Woods 1997). Although no previous study has reported a mastoid MMN attention effect, the current data suggest the possibility of such a finding and, consequently, that the temporal lobe MMN generators may be influenced by attention.
The independence of the 2 memory traces was estimated with a simple model. Assume that the representation for each set of sounds is a linear combination of the standards presented in both sets. For a particular sound set, the representation is then
 | (1) |
 | (2) |
 | (3) |
 | (4) |
 | (5) |
This measure of representation independence may be underestimated. If MMN is the basis for conscious change detection (Näätänen 1990), any sound attribute change reflected in the MMN needs to be based on comparable levels of integration of the relevant sound attribute data. Nevertheless, MMN onset typically occurs during the 180-ms time established for temporal integration (Scharf 1978). This is curious. An explanation for this apparent anomaly is that integration of stimuli has 2 phases, one reflecting within-stimulus integration of attributes of the current stimulus followed by a second between-stimulus integration phase when the current stimulus is added to (and possibly compared with) a general representation of recent stimuli. The first phase, roughly analogous to the sensory feature trace of Näätänen and Winkler (1999), must be completed before MMN generation may begin. That is, if duration-deviance onset occurs following a short standard, MMN onset must further await completion of the first phase of integration. With a longer duration standard, deviance onset may well occur after the completion of the first phase of integration, allowing MMN to proceed immediately, and leaving a differential lag between deviance onset and MMN onset for these 2 standard durations.
Thus, we suggest, it is possible in our data, that MMN onset was delayed more after deviance onset following the short standards than following the long standards, reducing our estimate of representation purity. The smallest of the many (and varied) estimates for the period of temporal integration for loudness (Scharf 1978) suggests that MMN onset cannot occur prior to 70 ms but may occur later. MMN onset latency reports are rare, but using frequency deviants, average latencies of 77 ms in a passive condition, and 62 ms when hypnotized, have been reported (Kallio and others 1999). Using 1-ms duration white noise deviants relative to 100-ms standards, Jaramillo and others (2000) found duration MMN onset between 90 and 100 ms poststimulus at Fz (see their Fig. 2). Finally, based on intracranial electrodes, Kropotov and others (1995) found frequency MMN onset as early as 80 ms in Brodmann's area 42. If the earliest onset for MMN is around 70 ms, a 30-ms delay occurs between offset of the short (40 ms) standard and MMN onset, reducing Dlong–Dshort in equation (3) above. Adjustment for this delay of 30 ms increases the estimated percentage of in-channel representation to near 100%.
Under conditions of strong control of attention, we have shown a high level of separation of memory traces underlying the attended and unattended stimuli. Although this is entirely consistent with the argument that attention is necessary for auditory streaming (Carlyon and others 2001), which may, in turn, be responsible for the separation of the underlying representations (Ritter and others 2000), we prefer to broaden the explanation. We suggest that the separation is a direct reflection of the actual mechanism of sustained attention and not some by-product mediated by auditory streaming. In the context of a process that integrates "all" past stimuli (our second phase of integration), separating the representations of attended stimuli from all unattended stimulus representations preserves the attended stimulus purity. Increased attended stimulus representation purity increases contrast finesse for novel attended stimuli.
Potential evidence for across-stimulus integration was first reported nearly a century ago (Hollingworth 1910) as a bias in duration judgments toward the average of the set of presented durations (sometimes referred to as Vierordt's Law). Observers overestimate durations less than the average and underestimate durations longer than the average. If MMN reflects contrast of this collective representation with any novel duration, then increased relative purity of the attended sound representation would benefit contrast of that representation relative to attended deviant sounds. The timescale for operation of this mechanism (many seconds) separates it from a process of integration thought responsible for the increased perceived intensity with increased duration of short stimuli.
On the other hand, the unattended representation, under our explanation, would reflect integration across the set of all unattended stimuli. Carlyon and others (2001) have already suggested such a process and pointed to behavioral evidence (Brochard and others 1999). This finding contradicts the notion that auditory streaming reflects low-level grouping processes. Because the MMN has been shown to occur even under conditions of strong attention withdrawal (Näätänen and others 1993), it may be possible to obtain electrophysiological evidence for a lack of streaming among unattended stimuli. We are currently seeking such evidence.
However, the evidence presented here may still fit a streaming explanation akin to that used by Ritter and others (2000). Further, even if evidence that unattended stimuli do not form independent streams can be found, the currently predominant streaming explanation could be saved by amending it to subcortically produce 2 streams of sounds—attended and unattended.
The broader significance of the proposed mechanism of attention is for a benefit to fine contrasts between attended stimuli relative to when they are unattended within the context of a "late-selection" model. Not only is the likelihood of attention capture by unattended stimuli reduced (but not prevented) but also the resolving power of the system for attended stimuli is improved. In distinction from other ideas on attention (Näätänen 1991), we have proposed that the informational basis of the MMN comparison process is improved by attention because the preceding-stimulus representation is uncontaminated by unattended distractors. This will have consequences for the latency and amplitude of attended versus unattended MMN.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Alain C and Woods DL. (1994) Signal clustering modulates auditory cortical activity in humans. Percept Psychophys 56:501–516.[ISI][Medline]
Alain C and Woods DL. (1997) Attention modulates auditory pattern memory as indexed by event-related brain potentials. Psychophysiology 34:534–546.[ISI][Medline]
Alho K, Paavilainen P, Reinikainen K, Sams M, Näätänen R. (1986) Separability of different negative components of the event-related potential associated with auditory stimulus processing. Psychophysiology 23:613–623.[ISI][Medline]
Brochard R, Drake C, Botte M-C, McAdams S. (1999) Perceptual organization of complex auditory sequences: effect of number of stimulus subsequences and frequency separation. J Exp Psychol Hum Percept Perform 25:1742–1759.[CrossRef][ISI][Medline]
Carlyon RP, Cusack R, Foxton JM, Robertson IH. (2001) Effects of attention and unilateral neglect on auditory stream segregation. J Exp Psychol Hum Percept Perform 27:115–127.[CrossRef][ISI][Medline]
Grier JB. (1971) Nonparametric indexes for sensitivity and bias: computing formulas. Psychol Sci 75:424–429.
Hollingworth HL. (1910) The central tendency of judgment. J Philos Psychol Sci Methods 7:461–469.[CrossRef]
Jaramillo M, Paavilainen P, Näätänen R. (2000) Mismatch negativity and behavioural discrimination in humans as a function of the magnitude of change in sound duration. Neurosci Lett 290:101–104.[CrossRef][ISI][Medline]
Kallio S, Revonsuo A, Lauerma H, Hämäläinen H, Lang H. (1999) The MMN amplitude increases in hypnosis: a case study. Neuroreport 10:3579–3582.[ISI][Medline]
Kropotov JD, Näätänen R, Sevostianov AV, Alho K, Reinikainen K, Kropotova OV. (1995) Mismatch negativity to auditory stimulus change recorded directly from the human temporal cortex. Psychophysiology 32:418–422.[ISI][Medline]
Näätänen R. (1990) The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function. Behav Brain Sci 13:201–288.
Näätänen R. (1991) Mismatch negativity outside strong attentional focus: a commentary on Woldorff et al. 1991. Psychophysiology 28:478–484.[ISI][Medline]
Näätänen R, Paavilainen P, Reinikainen K. (1989) Do event related potentials to infrequent decrements in duration of auditory stimuli demonstrate a memory trace in man? . Neurosci Lett 107:347–352.[CrossRef][ISI][Medline]
Näätänen R, Paavilainen P, Tiitinen H, Jiang D, Alho K. (1993) Attention and mismatch negativity. Psychophysiology 30:436–450.[ISI][Medline]
Näätänen R, Simpson M, Loveless NE. (1982) Stimulus deviance and evoked potentials. Biol Psychol 14:53–98.[CrossRef][ISI][Medline]
Näätänen R, Syssoeva O, Takegata R. (2004) Automatic time perception in the human brain for intervals ranging from milliseconds to seconds. Psychophysiology 41:660–663.[CrossRef][ISI][Medline]
Näätänen R and Winkler I. (1999) The concept of auditory stimulus representation in cognitive neuroscience. Psychol Bull 125:826–859.[CrossRef][ISI][Medline]
Ofek E and Pratt H. (2004) Ear advantage and attention: an ERP study of auditory cued attention. Hear Res 189:107–118.[CrossRef][ISI][Medline]
Pendse SG. (1977) Hemispheric asymmetry in making absolute judgments of monaurally presented pure tones. Percept Mot Skills 44:923–928.[ISI][Medline]
Ritter W, Sussman E, Molholm S. (2000) Evidence that the mismatch negativity system works on the basis of objects. Neuroreport 11:61–63.[ISI][Medline]
Scharf B. (1978) Loudness. In Carterette EC and Friedman NP (Eds.). Handbook of perception(Academic Press, New York) pp. 189–242.
Winkler I, Karmos G, Näätänen R. (1996) Adaptive modeling of unattended acoustic environment reflected in the mismatch negativity event-related potential. Brain Res 742:239–252.[CrossRef][ISI][Medline]
Woldorff MG, Hackley SA, Hillyard SA. (1991) The effects of channel-selective attention on the mismatch negativity wave elicited by deviant tones. Psychophysiology 28:30–42.[ISI][Medline]
Woldorff MG, Hillyard SA, Gallen CC, Hampson SR, Bloom FE. (1998) Magnetoencephalographic recordings demonstrate attentional modulation of mismatch-related neural activity in human auditory cortex. Psychophysiology 35:283–292.[CrossRef][ISI][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||