|
Posted On: May 27, 2008
During my audiology career at the University of Arkansas at Little Rock, I regularly taught a course in the audiologic rehabilitation of adults with hearing loss. Speechreading, also known as lipreading, was always covered as a component in the rehab process, but I didn't actually "teach" audiologists how to conduct formal lipreading lessons as I had been taught in my graduate coursework in communication disorders way back in the l960s.
Back then, speechreading was considered a crucial skill for improving communication function, and we spent many sessions debating the differences between the "analytic" and "synthetic" schools of thought. The names Nitchie, Kelly, Jena, and Morkovin-Moore, among others, were bandied about as pioneers in the many approaches to "learning" how to speechread. Students were divided into groups, with each group researching a specific approach and presenting detailed lessons based on available materials. For weeks, we drilled each other with silently presented vowels, consonants, words, sentences, and paragraphs designed to increase our proficiency.
At the end of the course, I could only admit to one thing: I remained a poor speechreader even after our exhaustive training regimens. Others had been excellent lipreaders all along, able to spy with ease on what others were saying at a far-distant table in a noisy restaurant. I was unconvinced back then of the validity of speechreading lessons, and doubts remained with me throughout my teaching and clinical career.
I have no doubts, however, of the value of combining auditory and visual cues for speech understanding, especially in noisy conditions. By watching the speaker's face and body while listening, we can increase speech recognition considerably over auditory-alone conditions. It's not unusual for speech intelligibility to increase from 20 percent when listening in noise without vision, to 80 percent or more when the speaker is seen as well as heard. This four-fold performance improvement is based upon the principle of "bisensory integration," in which combining two senses produce more information together than would be predicted from merely adding the performance of the two senses separately.
Recent studies, which tie hearing and vision together even more closely at the neural level, convince me that audiologists don't have to provide formal lessons on speechreading to improve performance; rather, we should encourage patients to be more attentive to and watchful of the visual as well as auditory aspects of communication while wearing their hearing aids.
First, a little background: In the traditional view of brain functioning, individual senses-vision, audition, touch, smell, and taste-had separate areas of the brain dedicated to them for initial processing, with information from each integrated only later in the higher-order association areas. However, in a 2006 article entitled "Is neocortex essentially multisensory?" in Trends in Cognitive Sciences, Princeton University researchers reviewed numerous studies related to multisensory processing in the brain and brainstem and concluded, "Although sensory perception and neurobiology are traditionally investigated one modality at a time, real world behaviour and perception are driven by the integration of information from multiple sensory sources." 1
These studies show that multisensory processing of auditory and visual stimuli occur in many areas of the brain, not only in the higher-order association cortex but also in the primary and secondary areas of the auditory cortex, the visual cortex, and other cortical and subcortical areas. The fact that neural cells in the brain and brainstem share sensory information early on suggests that the process of auditory-visual speech integration may happen much faster than previously thought.
In 2006, Finnish researchers reported in Human Brain Mapping on the effect of attention on visual speech-induced auditory cortex activity in 10 healthy volunteers.2 They used functional magnetic resonance imaging (fMRI) during simultaneous presentation of visual speech gestures and moving geometrical forms, with the instruction to either focus on or ignore the seen articulations.
During the attentive condition, visual speech gestures enhanced activity in the left planum temporale, a cortical area in the temporal lobe just posterior to the auditory cortex and responsible for language processing. Such findings suggest that, "attention to visually perceived speech gestures modulates auditory cortex function."
In a 2008 study in Cerebral Cortex, German researchers again demonstrated that visual stimuli can modulate the firing of neurons in the auditory cortex and concluded that, "These neurons thus meet the criteria for sensory integration and provide the auditory modality with multisensory contextual information about co-occurring environmental events."3
A recent study reported in ScienceDaily implies that the integration of the different senses may begin even at sub-cortical levels.4 Here, Duke University researchers found that within the inferior colliculus, the principal midbrain nucleus of the auditory pathway, about 64 percent of the neurons can carry visual as well as auditory signals. "This means that visual and auditory information gets combined quite early," said neurobiologist Jennifer Groh, PhD, "and before the 'thinking part' of the brain can make sense of it."
This is why ventriloquism seems to work, she said. The association between the voice and the moving mouth of the dummy is made before the viewer consciously thinks about it.
Even the dramatic McGurk Effect, 6 in which an incongruous sound and visual speech gesture produce the persistent illusion of a third sound, may have a sub-cortical component. Stimuli depicting the illusion were presented to a 12-year-old child with a circumscribed unilateral lesion of the right inferior colliculus. The illusion was significantly reduced when presented in the contralesional hemifield when compared with the ipsilesional hemifield and a group of age-matched controls. These data suggest. "a functional role for the inferior colliculus in the audio-visual integration of speech."5
The early and strong effect of seeing as well as hearing the speaker on the ultimate perception of speech should again encourage audiologists to use the WATCH protocol, presented in a 2007 article in ADVANCE for Audiologists online.6 The five steps for patients using hearing aids include the following: W - Wear glasses when necessary to see the speaker's face clearly. Even listeners with normal hearing often comment that they "hear" better in noisy environments when they have their glasses on.
A - Attend closely to the face and body of the speaker. Again, visual cues complement and reinforce the enhanced auditory cues provided by hearing aids or assistive listening devices.
T - Tell speakers to keep their faces visible and use clear speech, which make both visual cues and auditory cues even more accessible.
C - Change positions for best viewing of the speaker. Avoid sitting in poor lighting or where the glare of a window obscures a speaker's face. In a conversational setting, use seating where all faces can be seen easily, such as at a round table rather than a square one.
H - Hats off to McGurk and McDonald, who humorously pointed out in l976 the importance of "hearing lips and seeing voices."
Jess Dancer, EdD, is professor emeritus of audiology at the University of Arkansas at Little Rock. Contact him at jedancer@ualr.edu with your experiences with bisensory integration in the audiologic rehabilitation process.
References
1. Ghazanfar, A, Schrdoeder, C. (2006). Is neocortex essentially multisensory? Trends in Cognitive Sciences, 10(6):278-86.
2. Pekkola, J, Ojanen, V, Autti, T, Jaaskelainen, I, Mottonen, R, Sams, M. (2006). Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale. Human Brain Mapping, 27(6): 471-7.
3. Kayser, C, Petkov, CT, Logothetis, NK. (2008, January 6). Visual modulation of neurons in auditory cortex. Cerebral Cortex [Epub ahead of print].
4. Duke University Medical Center (2007, November 2). How ventriloquists trick the brain: sight, sound processed together and earlier than previously thought. ScienceDaily. Accessed online at www.sciencedaily.com/releases/2007/10/071029172919.htm
5. Champoux, F. Tremblay, C, Mercier, C, Lassonde, M, Lepore, F, Gagne, JP, Theoret, H. (2006). A role for the inferior colliculus in multisensory speech integration, Neuroreport, 17(15): 1607-10.
6. Dancer, J. (2007). Hearing lips and seeing voices: The McGurk effect 30 years later. ADVANCE for Audiologists online. Accessed online at http://audiology.advanceweb.com/editorial/content/editorial.aspx?cc=90118
|