Cerebral Cortex, Vol. 11, No. 12, 1110-1123,
December 2001
© 2001 Oxford University Press
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford OX3 9DU, UK
Gemma A. Calvert, Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), John Radcliffe Hospital, Oxford OX3 9DU, UK. Email: gemma{at}fmrib.ox.ac.uk.
Modern brain imaging techniques have now made it possible to study the neural sites and mechanisms underlying crossmodal processing in the human brain. This paper reviews positron emission tomography, functional magnetic resonance imaging (fMRI), event-related potential and magnetoencephalographic studies of crossmodal matching, the crossmodal integration of content and spatial information, and crossmodal learning. These investigations are beginning to produce some consistent findings regarding the neuronal networks involved in these distinct crossmodal operations. Increasingly, specific roles are being defined for the superior temporal sulcus, the inferior parietal sulcus, regions of frontal cortex, the insula cortex and claustrum. The precise network of brain areas implicated in any one study, however, seems to be heavily dependent on the experimental paradigms used, the nature of the information being combined and the particular combination of modalities under investigation. The different analytic strategies adopted by different groups may also be a significant factor contributing to the variability in findings. In this paper, we demonstrate the impact of computing intersections, conjunctions and interaction effects on the identification of audiovisual integration sites using existing fMRI data from our own laboratory. This exercise highlights the potential value of using statistical interaction effects to model electrophysiological responses to crossmodal stimuli in order to identify possible sites of multisensory integration in the human brain.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. Falchier, C. E. Schroeder, T. A. Hackett, P. Lakatos, S. Nascimento-Silva, I. Ulbert, G. Karmos, and J. F. Smiley Projection from Visual Areas V2 and Prostriata to Caudal Auditory Cortex in the Monkey Cereb Cortex, October 29, 2009; (2009) bhp213v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. D. Dahl, N. K. Logothetis, and C. Kayser Spatial Organization of Multisensory Responses in Temporal Association Cortex J. Neurosci., September 23, 2009; 29(38): 11924 - 11932. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. A. Renier, I. Anurova, A. G. De Volder, S. Carlson, J. VanMeter, and J. P. Rauschecker Multisensory Integration of Sounds and Vibrotactile Stimuli in Processing Streams for "What" and "Where" J. Neurosci., September 2, 2009; 29(35): 10950 - 10960. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. van Wassenhove Minding time in an amodal representational space Phil Trans R Soc B, July 12, 2009; 364(1525): 1815 - 1830. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. T. Hill and L. M. Miller Auditory Attentional Control and Selection during Cocktail Party Listening Cereb Cortex, July 2, 2009; (2009) bhp124v1. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R.-i. Hashimoto, K. Lee, A. Preus, R. W. McCarley, and C. G. Wible An fMRI Study of Functional Abnormalities in the Verbal Working Memory System and the Relationship to Clinical Symptoms in Chronic Schizophrenia Cereb Cortex, April 24, 2009; (2009) bhp079v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. I. Taylor, E. A. Stamatakis, and L. K. Tyler Crossmodal integration of object features: Voxel-based correlations in brain-damaged patients Brain, March 1, 2009; 132(3): 671 - 683. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Chandrasekaran and A. A. Ghazanfar Different Neural Frequency Bands Integrate Faces and Voices Differently in the Superior Temporal Sulcus J Neurophysiol, February 1, 2009; 101(2): 773 - 788. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Hocking and C. J. Price The Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing Cereb Cortex, October 1, 2008; 18(10): 2439 - 2449. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Kayser, C. I. Petkov, and N. K. Logothetis Visual Modulation of Neurons in Auditory Cortex Cereb Cortex, July 1, 2008; 18(7): 1560 - 1574. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Plaza, P. Gatignol, H. Cohen, B. Berger, and H. Duffau A Discrete Area within the Left Dorsolateral Prefrontal Cortex Involved in Visual-Verbal Incongruence Judgment Cereb Cortex, June 1, 2008; 18(6): 1253 - 1259. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. A. Ghazanfar, C. Chandrasekaran, and N. K. Logothetis Interactions between the Superior Temporal Sulcus and Auditory Cortex Mediate Dynamic Face/Voice Integration in Rhesus Monkeys J. Neurosci., April 23, 2008; 28(17): 4457 - 4469. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 I. G Kulahci, A. Dornhaus, and D. R Papaj Multimodal signals enhance decision making in foraging bumble-bees Proc R Soc B, April 7, 2008; 275(1636): 797 - 802. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Poeppel, W. J Idsardi, and V. van Wassenhove Speech perception at the interface of neurobiology and linguistics Phil Trans R Soc B, March 12, 2008; 363(1493): 1071 - 1086. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
U. Noppeney, O. Josephs, J. Hocking, C. J. Price, and K. J. Friston The Effect of Prior Visual Information on Recognition of Speech and Sounds Cereb Cortex, March 1, 2008; 18(3): 598 - 609. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. W. Seeley, B. R. Matthews, R. K. Crawford, M. L. Gorno-Tempini, D. Foti, I. R. Mackenzie, and B. L. Miller Unravelling Bolero: progressive aphasia, transmodal creativity and the right posterior neocortex Brain, January 1, 2008; 131(1): 39 - 49. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Maurage, S. Campanella, P. Philippot, T. H. Pham, and F. Joassin The crossmodal facilitation effect is disrupted in alcoholism: A study with emotional stimuli Alcohol Alcohol., November 1, 2007; 42(6): 552 - 559. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Pressman, L. J. Welty, A. Karniel, and F. A. Mussa-Ivaldi Perception of Delayed Stiffness The International Journal of Robotics Research, November 1, 2007; 26(11-12): 1191 - 1203. [Abstract] [PDF]
|
|
|
|
 |
|
 G. Musacchia, M. Sams, E. Skoe, and N. Kraus Musicians have enhanced subcortical auditory and audiovisual processing of speech and music PNAS, October 2, 2007; 104(40): 15894 - 15898. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. M. Willems, A. Ozyurek, and P. Hagoort When Language Meets Action: The Neural Integration of Gesture and Speech Cereb Cortex, October 1, 2007; 17(10): 2322 - 2333. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. M. Romanski Representation and Integration of Auditory and Visual Stimuli in the Primate Ventral Lateral Prefrontal Cortex Cereb Cortex, September 1, 2007; 17(suppl_1): i61 - i69. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Hein, O. Doehrmann, N. G. Muller, J. Kaiser, L. Muckli, and M. J. Naumer Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas J. Neurosci., July 25, 2007; 27(30): 7881 - 7887. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Martuzzi, M. M. Murray, C. M. Michel, J.-P. Thiran, P. P. Maeder, S. Clarke, and R. A. Meuli Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics Cereb Cortex, July 1, 2007; 17(7): 1672 - 1679. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Mishra, A. Martinez, T. J. Sejnowski, and S. A. Hillyard Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion J. Neurosci., April 11, 2007; 27(15): 4120 - 4131. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. M. van Atteveldt, E. Formisano, L. Blomert, and R. Goebel The Effect of Temporal Asynchrony on the Multisensory Integration of Letters and Speech Sounds Cereb Cortex, April 1, 2007; 17(4): 962 - 974. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. C. Wilson, J. R. Melcher, C. Micheyl, A. Gutschalk, and A. J. Oxenham Cortical fMRI Activation to Sequences of Tones Alternating in Frequency: Relationship to Perceived Rate and Streaming J Neurophysiol, March 1, 2007; 97(3): 2230 - 2238. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Kayser, C. I. Petkov, M. Augath, and N. K. Logothetis Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex J. Neurosci., February 21, 2007; 27(8): 1824 - 1835. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Merabet, J. D. Swisher, S. A. McMains, M. A. Halko, A. Amedi, A. Pascual-Leone, and D. C. Somers Combined Activation and Deactivation of Visual Cortex During Tactile Sensory Processing J Neurophysiol, February 1, 2007; 97(2): 1633 - 1641. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Senkowski, S. Molholm, M. Gomez-Ramirez, and J. J. Foxe Oscillatory Beta Activity Predicts Response Speed during a Multisensory Audiovisual Reaction Time Task: A High-Density Electrical Mapping Study Cereb Cortex, November 1, 2006; 16(11): 1556 - 1565. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Sugihara, M. D. Diltz, B. B. Averbeck, and L. M. Romanski Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex J. Neurosci., October 25, 2006; 26(43): 11138 - 11147. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Dazzan, K. D. Morgan, X. Chitnis, J. Suckling, C. Morgan, P. Fearon, P. K. McGuire, P. B. Jones, J. Leff, and R. M. Murray The Structural Brain Correlates of Neurological Soft Signs in Healthy Individuals Cereb Cortex, August 1, 2006; 16(8): 1225 - 1231. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. I. Taylor, H. E. Moss, E. A. Stamatakis, and L. K. Tyler Binding crossmodal object features in perirhinal cortex PNAS, May 23, 2006; 103(21): 8239 - 8244. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Brancazio, C. T. Best, and C. A. Fowler Visual Influences on Perception of Speech and Nonspeech Vocal-Tract Events Language and Speech, March 1, 2006; 49(1): 21 - 53. [Abstract] [PDF]
|
|
|
|
|
|
J. A. Johnson and R. J. Zatorre Attention to Simultaneous Unrelated Auditory and Visual Events: Behavioral and Neural Correlates Cereb Cortex, October 1, 2005; 15(10): 1609 - 1620. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. C. Pfeiffer, T. A. Hollowood, J. Hort, and A. J. Taylor Temporal Synchrony and Integration of Sub-threshold Taste and Smell Signals Chem Senses, September 1, 2005; 30(7): 539 - 545. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Guo and A. Guo Crossmodal Interactions Between Olfactory and Visual Learning in Drosophila Science, July 8, 2005; 309(5732): 307 - 310. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. C. Tanabe, M. Honda, and N. Sadato Functionally Segregated Neural Substrates for Arbitrary Audiovisual Paired-Association Learning J. Neurosci., July 6, 2005; 25(27): 6409 - 6418. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. M. Murray, S. Molholm, C. M. Michel, D. J. Heslenfeld, W. Ritter, D. C. Javitt, C. E. Schroeder, and J. J. Foxe Grabbing Your Ear: Rapid Auditory-Somatosensory Multisensory Interactions in Low-level Sensory Cortices Are Not Constrained by Stimulus Alignment Cereb Cortex, July 1, 2005; 15(7): 963 - 974. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. C Crick and C. Koch What is the function of the claustrum? Phil Trans R Soc B, June 29, 2005; 360(1458): 1271 - 1279. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. M. Miller and M. D'Esposito Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech J. Neurosci., June 22, 2005; 25(25): 5884 - 5893. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. A. Osterbauer, P. M. Matthews, M. Jenkinson, C. F. Beckmann, P. C. Hansen, and G. A. Calvert Color of Scents: Chromatic Stimuli Modulate Odor Responses in the Human Brain J Neurophysiol, June 1, 2005; 93(6): 3434 - 3441. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. A. Ghazanfar, J. X. Maier, K. L. Hoffman, and N. K. Logothetis Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex J. Neurosci., May 18, 2005; 25(20): 5004 - 5012. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. A. Huettel, A. W. Song, and G. McCarthy Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices J. Neurosci., March 30, 2005; 25(13): 3304 - 3311. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Small, J. Voss, Y. E. Mak, K. B. Simmons, T. Parrish, and D. Gitelman Experience-Dependent Neural Integration of Taste and Smell in the Human Brain J Neurophysiol, September 1, 2004; 92(3): 1892 - 1903. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Gogtay, J. N. Giedd, L. Lusk, K. M. Hayashi, D. Greenstein, A. C. Vaituzis, T. F. Nugent III, D. H. Herman, L. S. Clasen, A. W. Toga, et al. From the Cover: Dynamic mapping of human cortical development during childhood through early adulthood PNAS, May 25, 2004; 101(21): 8174 - 8179. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. R. Dehner, L. P. Keniston, H. R. Clemo, and M. A. Meredith Cross-modal Circuitry Between Auditory and Somatosensory Areas of the Cat Anterior Ectosylvian Sulcal Cortex: A 'New' Inhibitory Form of Multisensory Convergence Cereb Cortex, April 1, 2004; 14(4): 387 - 403. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Molholm, W. Ritter, D. C. Javitt, and J. J. Foxe Multisensory Visual-Auditory Object Recognition in Humans: a High-density Electrical Mapping Study Cereb Cortex, April 1, 2004; 14(4): 452 - 465. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Dazzan, K. D. Morgan, K. G. Orr, G. Hutchinson, X. Chitnis, J. Suckling, P. Fearon, J. Salvo, P. K. McGuire, R. M. Mallett, et al. The structural brain correlates of neurological soft signs in AeSOP first-episode psychoses study Brain, January 1, 2004; 127(1): 143 - 153. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. M. Wright, K. A. Pelphrey, T. Allison, M. J. McKeown, and G. McCarthy Polysensory Interactions along Lateral Temporal Regions Evoked by Audiovisual Speech Cereb Cortex, October 1, 2003; 13(10): 1034 - 1043. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. N. Elston and K. S. Rockland The Pyramidal Cell of the Sensorimotor Cortex of the Macaque Monkey: Phenotypic Variation Cereb Cortex, October 1, 2002; 12(10): 1071 - 1078. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Lewald Rapid Adaptation to Auditory-Visual Spatial Disparity Learn. Mem., September 1, 2002; 9(5): 268 - 278. [Abstract] [Full Text] [PDF]
|
|