Foreign Subtitles Help and Native Language Subtitles Harm

Foreign Subtitles Help but Native-Language Subtitles Harm Foreign Speech Perception

Holger Mitterer1*, James M. McQueen1,2

1 Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands, 2 Behavioural Science Institute and Donders Institute for Brain, Cognition & Behaviour, Centre for Cognition, Radboud University Nijmegen, Nijmegen, The Netherlands

Abstract

Understanding foreign speech is difficult, in part because of unusual mappings between sounds and words. It is known that listeners in their native language can use lexical knowledge (about how words ought to sound) to learn how to interpret unusual speech-sounds. We therefore investigated whether subtitles, which provide lexical information, support perceptual learning about foreign speech. Dutch participants, unfamiliar with Scottish and Australian regional accents of English, watched Scottish or Australian English videos with Dutch, English or no subtitles, and then repeated audio fragments of both accents. Repetition of novel fragments was worse after Dutch-subtitle exposure but better after English-subtitle exposure. Native-language subtitles appear to create lexical interference, but foreign-language subtitles assist speech learning by indicating which words (and hence sounds) are being spoken.

Excerpt:

'Imagine an American listener, fluent in Mexican Spanish, watching El Laberinto del fauno [Pan's Labyrinth]. She may have considerable difficulty understanding the European Spanish if she is unfamiliar with that language variety. How might she be able to cope better?'

'We argue here that subtitles can help. Critically, the subtitles should be in Spanish, not English. This is because subtitles in the language of the film indicate which words are being spoken, and so can boost speech learning about foreign speech sounds.'

"Perceptual learning studies show that speech processing in the listener's native language can be retuned by lexical knowledge."

"listeners were using lexical knowledge to retune phonetic perception."

"Native-language subtitles help recognition of previously heard words but harm recognition of new words; (3) Foreign-language subtitles improve repetition of previously heard and new words, the latter demonstrating lexically-guided retuning of perception."


"We asked two questions. First, we tested whether audiovisual exposure allows listeners to adapt to an unfamiliar foreign accent. Second, we asked whether subtitles can influence this process. Our results show that this kind of adaptation is possible, and that subtitles which match the foreign spoken language help adaptation while subtitles in the listener's native language hinder adaptation.

The differences between the experimental and the control conditions speak to our first question. They show that listeners can adapt to an unfamiliar regional accent in a second language after only brief audiovisual exposure."

"Two points follow from the conclusion that the subtitle effects reflect lexically-guided retuning of perceptual categories. First, it would appear that lexically-guided learning can occur with real speech in a naturalistic setting. The phenomenon seems not to be restricted to the psycholinguistic laboratory. Second, this conclusion is consistent with the claim that lexically-guided retuning contributes to the way native listeners adapt to foreign-accented speech [9]. Importantly, however, the present findings show for the first time that this kind of perceptual learning is not restricted to native listening: It also occurs in second-language listening.

Our demonstration of perceptual learning about speech sounds in a second language has implications for both theory and practice in second-language acquisition. It has been suggested that certain aspects of language acquisition are fundamentally different in a second as opposed to a first language [28], [29]. With respect to speech recognition, however, the same perceptual-learning mechanism appears to apply in first- and second-language processing."

"As we used real subtitles, our results also have practical implications. Although the use of real subtitles meant that the listeners did not get a word-by-word transcription of the dialogue, it allows us to generalize our results to visual media exposure outside the laboratory. It appears that the largest benefit from this kind of real-world exposure, in the recognition of regional accents in a second language, comes from the use of subtitles in that language. But foreign-language subtitles are not what television viewers and filmgoers are familiar with. In many European countries (e.g., Germany) there is considerable public concern about international comparisons of scholarly achievements [e.g., 32]. Yet viewers are denied access to foreign-language speech, even on publicly-financed television programs. Instead, foreign languages are dubbed. In countries which use subtitles instead of dubbing (e.g., the Netherlands), only native-language subtitles are available, so again listeners are denied potential benefits in speech learning. Native-language subtitles are obviously essential for listeners who do not already speak a second language, and may thus be the only practical solution in cinemas. With the advent of digital television broadcasting, however, it is now possible to broadcast multiple audio channels and multiple types of subtitles. We suggest that it is now time to exploit these possibilities. Individuals can already take matters into their own hands, however. It is often possible to select foreign subtitles on commercial DVDs. So if, for example, an American speaker of Mexican Spanish wants to improve her understanding of European Spanish, we suggest that she should watch some DVDs of European Spanish films with Spanish subtitles."