Cerebral Cortex Advance Access first published online on January 11, 2007
This version published online on January 19, 2007
Cerebral Cortex, doi:10.1093/cercor/bhl147
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Hearing Lips and Seeing Voices: How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception
1 Departments of Neurology, 2 Psychology, and the Brain Research Imaging Center, The University of Chicago, Chicago, IL 60637, USA, 3 Division of Biology, California Institute of Technology, Pasadena, CA, USA
Address correspondence to Jeremy I. Skipper, Department of Neurology, The University of Chicago, MC 2030, 5841 South Maryland Ave., Chicago, IL 60637, USA. Email: skipper{at}uchicago.edu.
Observing a speaker's mouth profoundly influences speech perception. For example, listeners perceive an "illusory" "ta" when the video of a face producing /ka/ is dubbed onto an audio /pa/. Here, we show how cortical areas supporting speech production mediate this illusory percept and audiovisual (AV) speech perception more generally. Specifically, cortical activity during AV speech perception occurs in many of the same areas that are active during speech production. We find that different perceptions of the same syllable and the perception of different syllables are associated with different distributions of activity in frontal motor areas involved in speech production. Activity patterns in these frontal motor areas resulting from the illusory "ta" percept are more similar to the activity patterns evoked by AV/ta/ than they are to patterns evoked by AV/pa/ or AV/ka/. In contrast to the activity in frontal motor areas, stimulus-evoked activity for the illusory "ta" in auditory and somatosensory areas and visual areas initially resembles activity evoked by AV/pa/ and AV/ka/, respectively. Ultimately, though, activity in these regions comes to resemble activity evoked by AV/ta/. Together, these results suggest that AV speech elicits in the listener a motor plan for the production of the phoneme that the speaker might have been attempting to produce, and that feedback in the form of efference copy from the motor system ultimately influences the phonetic interpretation.
Key Words: audiovisual speech perception • efference copy • McGurk effect • mirror system • motor system • prediction
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M.-W. Suh, H.-J. Lee, J. S. Kim, C. K. Chung, and S.-H. Oh Speech experience shapes the speechreading network and subsequent deafness facilitates it Brain, October 1, 2009; 132(10): 2761 - 2771. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Peeters, L. Simone, K. Nelissen, M. Fabbri-Destro, W. Vanduffel, G. Rizzolatti, and G. A. Orban The Representation of Tool Use in Humans and Monkeys: Common and Uniquely Human Features J. Neurosci., September 16, 2009; 29(37): 11523 - 11539. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Hocking and C. J. Price The Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing Cereb Cortex, October 1, 2008; 18(10): 2439 - 2449. [Abstract] [Full Text] [PDF]
|
|