CRL Speaker Series Abstracts - Spring 2009
Giving Speech a Hand: Neural processing of co-speech gesture in native English speakers and Japanese-English bilinguals as well as typically-developing children and children with autism
Amy L. Hubbard
Successful social communication involves the integration of simultaneous input from multiple sensory modalities. Co-speech gesture plays a key role in multimodal communication, its effects on speech perception having been demonstrated on the behavioral and neural level (cf. McNeill, 2005; Willems et al., 2007). We used an ecologically valid fMRI paradigm to investigate neural responses to spontaneously produced beat gesture and speech. In our first study, we found that adult native English speakers show increased activity in superior temporal gyrus and sulcus (STG/S) while viewing beat gesture in the context of speech (versus viewing a still body or nonsense movements in the context of speech). In our second study, we again observed increases in the BOLD signal in STG/S while Japanese ESL speakers viewed beat gesture in the context of speech (as compared to viewing a still body or gesture tempo in the context of speech). These data suggest that co-speech gesture is processed (and/or integrated) in areas known to underlie speech perception, and meaningfulness of co-speech gesture is linked to its embodiment. In our third study, we examined co-speech gesture processing in children with Autism Spectrum Disorder (ASD; a developmental disorder characterized by excessive deficits in social communication) and typically developing children. Similar to our adult subjects, our typically developing matched controls showed increased activity in STG/S for viewing co-speech gesture (versus a still body with speech). However, children with ASD showed no increases in STG/S for this same contrast. These findings suggest that speech and gesture contribute jointly to communication during social interactions and that neural processes underlying co-speech gesture processing are disrupted in a clinical disorder well-known for its deficits in social communication.
A new model of local coherences as resulting from Bayesian belief update
Klinton Bicknell
Joint work with Roger Levy (UC San Diego) & Vera Demberg (University of Edinburgh)
Most models of incremental sentence processing assume that the processor does not consider ungrammatical structures. However, Tabor, Galantucci, and Richardson (2004) showed evidence of cases in which a syntactic structure that is ungrammatical given the preceding input nevertheless affects the difficulty of a word, termed local coherence effects. Our work fills two gaps in the literature on local coherences. First, it demonstrates from two experiments with an eye-tracking corpus that local coherence effects are evident in the reading of naturalistic text, not just rare sentence types like Tabor et al.'s. Second, it specifies a new computational model of local coherence effects under rational comprehension, proposing that local coherences arise as a result of updating bottom-up prior beliefs about the structures for a given string to posterior beliefs about the likelihoods of those structures in context. The critical intuition embodied in the model is that larger updates in probability distributions should be more processing-intensive; hence, the farther the context-conditioned posterior is from the unconditioned prior, the more radical the update required and the greater the processing load. We show that an implementation of our model using a stochastic context-free grammar (SCFG) correctly predicts the pattern of results in Tabor et al.
Resolving Conflicting Information from First-Mention Biases and Discourse Event Structure in Ambiguous Pronoun Interpretation in a Short Story Paradigm
Nathalie Bélanger
A small proportion of profoundly deaf individuals attain expert reading skills and it is important to understand why they become skilled readers and other deaf people do not. Despite the hypothesis that good phonological processing skills during reading are associated with good reading skills in deaf readers (Perfetti & Sandak, 2000), research has not yet provided clear answers as to whether it is the case or not. We investigated skilled and less skilled severely to profoundly deaf adult readers’ use of phonological codes during French word recognition and recall. A group of skilled hearing readers was also included as a means of comparison to existing literature. Given the close mapping of orthographic and phonological information in alphabetical writing systems, the unique contribution of orthographic codes was also investigated. Bearing in mind the particular focus on phonological processing in deaf (and hearing) readers and the potential implications on reading education for deaf children, it appears crucial to ensure that effects of orthographic and phonological information during word processing and recall are disentangled in this population. Results from a masked primed lexical decision task where orthographic and phonological overlap between primes and targets was manipulated show no difference between skilled hearing, skilled deaf and less skilled deaf readers in the way they activate orthographic and phonological information during early word recognition. The same groups of participants also performed a serial recall task where words were orthographically and phonologically similar (pierre, lierre, erre, etc), orthographically dissimilar and phonologically similar (chair, clerc, bière, etc), or orthographically and phonologically unrelated (ventre, manchot, oreille, etc). Skilled hearing readers showed a robust phonological similarity effect, but neither group of deaf readers (skilled or less skilled readers) did. All participants showed an advantage in recalling words that were orthographically and phonologically similar over the words that were orthographically dissimilar and phonologically similar suggesting that orthographic codes are also used to maintain words in short-term memory. The results of these two studies will be discussed and contrasted and will be presented in the context of reading instruction for deaf children.
How the conceptual system gets started and why it might interest image-schema theorists
Jean Mandler
A good case can be made that the foundations of the conceptual system rest on a small number of spatial primitives. Object concepts (animal, vehicle), relational concepts (in, out), and abstract concepts (cause, goal) all begin on a purely spatial basis and can easily be represented by spatial image-schemas. Only later in development do concepts accrue bodily associations, such as feelings of force and motor information. Bodily feelings enrich concepts but their representation remains crude and less structured than spatial representation. I suggest that simulations used to understand events rely primarily on spatial image-schemas and do not necessarily, include bodily feelings.
Enemies and friends in the neighborhood: cascaded activation of word meaning and the role of phonology
Diane Pecher (with René Zeelenberg)
Erasmus University Rotterdam
Many models of word recognition predict that orthographic neighbors (e.g., broom) of target words (e.g., bloom) will be activated during word processing. Cascaded models predict that semantic features of neighbors get activated before the target has been uniquely identified. This prediction is supported by the semantic congruency effect, the finding that neighbors that require the same response (e.g., living thing) facilitate semantic decisions whereas neighbors that require the opposite response (e.g., non-living thing) interfere with semantic decisions. In a recent study we investigated the role of phonology by manipulating whether orthographic neighbors had consistent (broom) or inconsistent phonology (blood). Congruency effects in animacy decision were larger when consistent neighbors had been primed than when inconsistent neighbors had been primed. In addition, semantic congruency effects were larger for targets with phonologically consistent neighbors than to targets with phonologically inconsistent neighbors. These results in line with models that assume an important role for phonology even in written word recognition (e.g., Van Orden, 1987).
Do lexical-syntactic selection mechanisms have rhythm?
Yet another "that" experiment, by Vic Ferreira (with Katie Doyle and Tom Christensen)
Speech tends to be rhythmic, alternating strong and weak syllables. To promote alternation, speakers (of English, at least) change *how* they say things ("thirTEEN," but "THIRteen MEN"), but will they change *what* they say? Perhaps not. Words and structures may be selected only to convey speakers' messages. And, phonological information may become available too late to influence lexical and syntactic selection. In two experiments, speakers produced sentences like, "NATE mainTAINED (that) ERin DAmaged EVery CAR in SIGHT" or "NATE mainTAINED (that) irENE deSTROYED the BUMper ON the TRUCK." The optional "that," a weak syllable, would promote stress alternation if mentioned in the first sentence and omitted in the second. Speakers in Experiment 1 produced sentences from memory, and said "that" about 6% more in the first type of sentence than the second. But, memory involves comprehension and production, and evidence suggests that comprehension more than production prefers alternating stress. So speakers in Experiment 2 produced sentences by combining simple sentences into complex ones; now, no difference is observed. This suggests that in extemporaneous production, speakers do not choose words and structures to promote alternating stress.
"The hand that rocks the cradle rules the brain"
Tom Bever (University of Arizona)
Fifty years of behavioral and clinical research supports the hypothesis that right handers with familial left handedness (RHFLH) have a distinct patterns of language behavior, which may reflect differences in neurological organization of the lexicon. RHFLH people organize their language processing with relative emphasis on individual words, while RHFRH people are more reliant on syntactic patterns. Recent fMRI studies support the idea that RHFLH people may access words more easily than RHFRH people because their lexicon is more bilaterally represented: syntactic tasks elicit left hemisphere activation in relevant areas for all subjects; corresponding lexical/semantic tasks elicit left hemisphere activation in RHFRH people, but bilateral representation in RHFLH people. This suggests that, while syntactic representation is normally represented in the left hemisphere, lexical information and access can be more widespread in the brain. This result has implications for clinical work and interpretation of many clinical and neurolinguistics studies that fail to differentiate subjects’ familial handedness. It also is suggestive about the language-specific neurological basis for syntax, amidst a more general basis for the lexicon.
Are the Literacy Challenges of Spanish-English Bilinguals Limited to Reading?
Darin Woolpert
Spanish-English bilinguals (SEBs) represent 9% of students in U.S. schools. In California alone, we have 1.3 million SEB students - more than a third of that total. These children have well-established academic struggles, with literacy being a particular concern (Grigg, Donahue, & Dion, 2005; Lee, Grigg, & Donahue, 2007; Restrepo & Gray, 2007). These reading problems persist throughout their academic careers, with SEB children lagging behind their monolingual English (ME) peers in pre-literacy skills such as phonological awareness (FACES 2000, 2003), and those that graduate high school do so reading, on average, at the 8th grade level (Donahue, Voekl, Campbell, & Mazzeo, 1999). A great deal of research has focused on early emerging literacy skills (e.g., Dickinson, McCabe, Clark-Chiarelli, & Wolf, 2004; Rolla San Francisco, Mo, Carlo, August, & Snow, 2006), such as phonological decoding (word reading) and encoding (spelling), as this is a crucial first step towards literacy acquisition for ME children (Bialystok, Luk, & Kwan, 2005; Gough & Tunmer, 1986). Recent research, however, has suggested that later-emerging skills, such as morphosyntactic awareness and reading comprehension, are the most problematic for SEB children (August & Shanahan, 2006), leaving questions about the origins of these deficits and the best way to address them. Children with a first language of Spanish may struggle to learn to decode in English due to typological differences such as the opacity of English orthography (seen in Bernard Shaw’s suggestion of "ghoti" as an alternate spelling for "fish"). Alternatively, SEB children may be struggling to build their literacy skills on a shaky foundation of spoken English, leading to problems as they get older.
To evaluate these competing claims, we gave standardized tests of spoken (sentence repetition and vocabulary) and written language (spelling and reading) to 53 SEB students from kindergarten to second grade, as well as a spoken and written narrative task. The children performed at age level in regards to spelling and reading (i.e., early-emerging literacy). The children tested below the normal range on the sentence repetition and vocabulary tasks, however. On the narrative task, the children struggled with verb morphology in both the spoken and written domains, with no significant differences in terms of error rate between the first and second graders.
These findings support those reported by August and Shanahan, and suggest that SEB children do not have problems with word decoding, but rather struggle to acquire literacy due to a lack of proficiency with English overall. This has implications for interventions developed for ME children with reading problems, and for the issue of properly diagnosing language impairment in ME children given the language profile of SEB children (e.g., Paradis, Rice, Crago, & Marquis, 2008). Directions for future research will be discussed.
"Point to where the frog is pilking the rabbit”: Investigating how children learn the meaning of sentences
Caroline Rowland (University of Liverpool, UK)
A unique but universal quality of language is the fact that the structure (or form) of a sentence affects its meaning. To master a language, learners must discover how sentence structure conveys meaning - the form-function mapping problem. This task is complicated by the fact that different languages require speakers to encode different aspects of the event; for example, in Spanish and German (but not in English) a speaker can change the order of the words without necessarily changing the meaning of the sentence, in German (but not English or Spanish), nouns must be marked for case, and in Spanish (but not English or German), speakers must use a grammatical patient marker if the object affected is animate.
Despite the apparent complexity of the task, recent research suggests that certain aspects of form-function mapping are learned very early on. For example, even before two years of age, English children can use word order to identify who is doing what to whom, detecting that transitives with novel verbs such as "the rabbit is glorping the duck" must refer to a cartoon showing a rabbit acting on a duck, not one in which a duck acts on a rabbit. However, it is unclear whether early ability is limited to frequently heard, simple structures like the transitive, or extends to other, more complex ones. This has implications for the amount of knowledge we attribute to young children and how we characterise the acquisition process. In addition, previous work often focuses only on showing that young children can understand form-function mappings, without investigating what it is that may underlie their performance (e.g. what might be the nature of any innate biases, what cues to meaning are most salient in the language children hear).
In this talk, I will present a number of studies using a new forced-choice pointing paradigm to investigate 3- and 4- year-old English and Welsh children’s comprehension of two structures that are less frequent and more complex than the transitive – the prepositional and double object dative. The results demonstrate that English and Welsh children have some verb-general knowledge of how dative syntax encodes meaning soon after their third birthday but that this is not always enough for successful comprehension. Cross- and within-language differences suggest that the correct interpretation of datives relies on the presence of a number of surface cues, and that the children’s ability to use a cue depends on its frequency and salience in child directed speech. Implications for theories of grammar and verb learning are discussed.
|
