Cerebral Cortex Advance Access published online on July 21, 2007
Cerebral Cortex, doi:10.1093/cercor/bhm121
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Visual Scenes Trigger Immediate Syntactic Reanalysis: Evidence from ERPs during Situated Spoken Comprehension
1 Center for Research in Language, University of California San Diego, USA, 2 Department of Neuropsychology, Otto-von-Guericke University Magdeburg, Germany, 3 Department of Computational Linguistics, Saarland University, Germany
Address correspondence to Dr Pia Knoeferle, Center for Research in Language, 9500 Gilman Drive, Department 0526, Room CSB 263, La Jolla, CA, 92093-0526, USA. Email: pknoeferle{at}ucsd.edu.
A central topic in sentence comprehension research is the kinds of information and mechanisms involved in resolving temporary ambiguity regarding the syntactic structure of a sentence. Gaze patterns in scenes during spoken sentence comprehension have provided strong evidence that visual scenes trigger rapid syntactic reanalysis. However, they have also been interpreted as reflecting nonlinguistic, visual processes. Furthermore, little is known as to whether similar processes of syntactic revision are triggered by linguistic versus scene cues. To better understand how scenes influence comprehension and its time course, we recorded event-related potentials (ERPs) during the comprehension of spoken sentences that relate to depicted events. Prior electrophysiological research has observed a P600 when structural disambiguation toward a noncanonical structure occurred during reading and in the absence of scenes. We observed an ERP component with a similar latency, polarity, and distribution when depicted events disambiguated toward a noncanonical structure. The distributional similarities further suggest that scenes are on a par with linguistic contexts in triggering syntactic revision. Our findings confirm the interpretation of previous eye movement studies and highlight the benefits of combining ERP and eye-tracking measures to ascertain the neuronal processes enabled by, and the locus of attention in, visual contexts.
Key Words: cross-methodological approach depicted events event-related potentials situated language comprehension
Received for publication September 15, 2006. Revision received May 31, 2007. Accepted for publication June 6, 2007.