Learning Rediscovered

Elizabeth Bates, Jeffrey Elman

The authors [HN2], [HN3] are in the Departments of Cognitive Science and Psychology, University of California, San Diego, CA 92093, USA. E-mail: bates@cr1.ucsd.edu

First, the nature of this learning is surprising: a purely inductive, statistically driven process, based on only 2 min of incidental input, with no reward or punishment other than the pleasure of listening to a disembodied human voice. Second, it contradicts the widespread belief that humans cannot and do not use generalized statistical procedures to acquire language (3, 4, 5, 6, 7). Noam Chomsky, [HN1] the founder of generative linguistics, has argued for 40 years that language is unlearnable; he and his followers have generalized this belief to other cognitive domains, denying the existence of learning as a meaningful scientific construct:

"We may usefully think of the language faculty, the number faculty, and others, as 'mental organs' [that] develop in specific ways, each in accordance with the genetic program...multipurpose learning strategies are no more likely to exist than general principles of 'growth of organs' that account for the shape, structure and growth of the kidney" (3, pp. 138-139).

"I, for one, see no advantage in the preservation of the term 'learning'...we would gain in clarity if the scientific use of the term were simply discontinued" (7, p. 2).

"It is possible that the notion 'learning' may go the way of the rising and setting of the sun" (3, p. 245).

This belief is based on the famous "poverty-of-the-stimulus" argument: linguistic knowledge is "perfect," and it is impossible in principle to extract perfect knowledge from the imperfect data of everyday language use. A formal proof by Gold (5) appeared to support this assumption, although Gold's theorem is relevant only if we make assumptions about the nature of the learning device that are wildly unlike the conditions that hold in any known nervous system (8). There are, in fact, a number of ways to get around the poverty-of-the-stimulus argument.

PHOTO: MICHAEL AND JEANETTE TWA

First, we could relax our definition of knowledge, defining successful learning to include behavior that is asymptotically correct but somewhere short of perfect ("close enough for government work"). Although there is plenty of evidence that humans use language creatively (saying and understanding things that have never been said before) and well (with very low error rates), there is very little evidence for the claim that "perfect" knowledge underlies our (occasionally) imperfect behavior.

Second, we could base our estimates of learnability on a more robust learning device than the one assumed by Chomsky. There is now a large body of evidence showing that artificial neural networks can induce regular patterns from imperfect, but quasi-regular input, and generalize those patterns to novel instances (8, 9, 10). Within the language domain, examples include the extraction of phonetic and phonological structures from raw speech (9), the discovery of word boundaries from connected speech (8), and the extraction of grammatical regularities from unlabeled strings of words generated by an artificial grammar with many of the properties of natural language (10).

Third, we now know that real speech contains a host of statistical regularities that are sufficient to support the kind of robust learning observed in neural networks (11). This knowledge has emerged from the analysis of huge computerized corpora of written and spoken language, revealing regularities that are not visible to the naked eye (or audible to the naked ear). Chomsky's belief in the impoverished nature of linguistic input holds only if we look "locally" at relatively short segments of speech. Such imperfections wash out with a large enough sample.

This brings us to the central contribution of the Saffran et al. report. Although we now know that linguistic regularities are learnable by neural networks with an imperfect but very large database, it has been argued that human infants do not learn in this way, and even if they did, their memory and attention span are insufficient to support the kind of statistical learning required to get language off the ground. This conclusion was premature: The new work (1) has shown that infants are capable of extracting statistical regularities from only 2 min of spoken input with little effort. To be sure, this experiment is not the first demonstration of early learning. For example, studies show that newborns prefer to listen to passages of speech from their native language, which means that some unspecified form of auditory learning has taken place in utero (12). Saffran et al. take us several steps further, with careful controls that make it absolutely clear what was learned, when, and how. Learning is much more powerful than previously believed, and arguments about the innateness of language and other forms of cognition need to take that undeniable fact into account.

The authors of the new work are quick to point out that their discovery does not justify a return to the tabula rasa. Learning is powerful, but it is not everything. In fact, relatively small variations in the initial architecture of a neural network can make the difference between "learnability" and "unlearnability" in the language domain (8, 10). Even if we assume that a brain (real or artificial) contains no innate knowledge at all, we have to make crucial assumptions about the structure of the learning device, its rate and style of learning, and the kinds of input that it "prefers" to receive. The emergence of language in the hominid line must have involved a certain amount of tinkering with the primate brain, leading ultimately to a brain that was capable of learning language.

References

1. J. R. Saffran, R. N. Aslin, E. L. Newport, Science 274, 1926 (1996).
2. L. Fenson et al., Monogr. Soc. Res. Child Dev. 59 (1994).
3. N. Chomsky, Rules and Representations (Columbia Univ. Press, New York, 1980).
4. S. Crain, Behav. Brain Sci. 14, 597 (1991).
5. E. Gold, Inf. Control 16, 447 (1967).
6. D. Lightfoot, The Language Lottery: Towards a Biology of Grammars (MIT Press, Cambridge, MA, 1980); F. Newmeyer, Linguistic Theory in America (Academic Press, New York, 1980).
7. M. Piatelli-Palmarini, Cognition 31, 1 (1989).
8. J. Elman et al., Rethinking Innateness: A Connectionist Perspective on Development (MIT Press and Bradford Book, Cambridge, MA, 1996).
9. J. Elman and D. Ziupser, J. Acoust. Soc. Am. 83, 1615 (1988).
10. J. Elman, Cognition 48, 71 (1993).
11. K. Lund and C. Burgess, Behav. Res. Methods Instrum. Comput. 28, 203 (1996).
12. P. Jusczyk, A. Friederici, J. Wessels, V. Svenkerud, A. Jusczyk, J. Mem. Lang. 32, 402 (1993); J. Mehler, G. Lambertz, P. Jusczyk, C. Amiel-Tison, C. R. Acad. Sci. Paris 303, 637 (1986).

HyperNotes
Related Resources on the World Wide Web

Linguistics Materials on the Web , maintained by the Department of Linguistics at the University of Rochester, provides links to databases and other resources for linguists.

1. Noam Chomsky's Web page includes a brief bibliography.

2. Elizabeth Bates' Web page includes a bibliography of her recent works and links to the full text of some of her publications.

3. Jeffrey L. Elman's Web page lists his recent publications and provides links to the full text of some of his writings. It also provides ordering information for his book, Rethinking Innateness: A Connectionist Perspective on Development.

4. Department of Cognitive Science, University of California, San Diego.

5. Department of Psychology, University of California, San Diego.

6. University of California, San Diego.