Language and the Primate Brain

Martin I. Sereno

Cognitive Science Department, UCSD

      [Diagrams for the following paper have been omitted 
      here.  Explanations of the diagrams, however, follow the 
      bibliography.  Hard copies, which include the diagrams, 
      can be obtained from:  Comments that 
      pertain to the content of this paper should be directed 
      to the author at:]


                      Martin I. Sereno
                  Cognitive Science D-015
            University of California, San Diego
                    La Jolla, CA  92093


  Psychologists, neuropsychologists, and primate neurobiolo-
  gists  have  studied  human language comprehension and its
  relation to the primate brain in almost complete isolation
  from  each other.  Recent developments provide grounds for
  a new attempt  at  drawing  some  preliminary  connections
  across the levels of organization spanned by these fields.
  New data on the large number of modality-specific areas in
  the post-central cortex of several non-human primates, and
  recent anatomical and  functional  studies  of  the  human
  brain  suggest  that very little of the cortex consists of
  poly-modal 'association' areas.   These  observations  are
  used  to  reinterpret psychological and neuropsychological
  data on language comprehension in normal and brain-damaged
  humans.   I  argue  that language comprehension in sighted
  people might best be thought of as a kind of code-directed
  scene  comprehension  that draws heavily upon specifically
  visual, and probably largely prelinguistic processing con-
  straints.   The  key processes of word-recognition and the
  assembly of visual word meaning patterns into  interacting
  chains,  however,  may  be  mediated  in  part by species-
  specific activity patterns in  secondary  auditory  cortex
  similar  to  those generated by uninterpreted speech-sound

     One obvious reason to study non-human primate brains is 
that they resemble the human primate brain in many ways.  Yet
humans exhibit behaviors--especially  the  comprehension  of
linguistic  discourse--that are qualitatively very different
from behaviors of primates and other  animals.   Because  of
this,  some  have  concluded  that animal brains may be poor
models for the human brain.  There are presently quite  sub-
stantial  rifts  between  psychological, neuropsychological,
and neurobiological approaches to language. Recent  develop-
ments  in studying human and animal brains, however, provide
a strong impetus to re-open discourse  among  these  discip-

     The neocortex of all mammals is now  known  to  consist
primarily  of  a  mosaic of visual, auditory, somatosensory,
motor, and limbic areas.  Primitive  mammals  have  a  small
number  of  areas  in  each  of these modalities, while car-
nivores and primates have many.  In monkeys, for example,  a
mosaic  of  25  visual  areas occupies more than half of the
entire neocortex (Merzenich  and  Kaas,  1980;  Kaas,  1987;
Sereno,  1988; Felleman and Van Essen, 1990; Sereno and All-
man,  1990).   The   traditional   site   for   higher-level
functions--"polymodal  association cortex"--has been reduced
to a few diminutive strips in between large expanses of uni-
modal visual, auditory, and somatosensory areas.  The poten-
tial significance of this reparcellation of cortex  for  the
study  of  language  and the brain has hardly been explored.
The aim of this paper is to re-introduce a  thoroughly  com-
parative  perspective  into  the evolutionary acquisition of
the capacity for language, but one that does not  back  away
from  the  obvious  cognitive differences between humans and
other animals.  The anatomical and  physiological  organiza-
tion of cortical areas in primates, including recent work on
human cortex, is reviewed first.  The implications  of  this
work  for  theories of human language comprehension are then

Cortical Sensory Areas in Primates

Definition of a Visual Area.   Cortical  sensory  areas  are
best  defined  by  multiple  converging criteria (Van Essen,
1985; Sereno and Allman, 1990).  I begin  here  with  visual
areas, since they constitute the largest of the primary sub-
divisions of the cortex. Criteria for the  definition  of  a
visual  area presently include architectonic features (e.g.,
degree of myelination, cell size, cell morphology, and  cell
packing density in cortical layers, histochemical features),
connection patterns (e.g., input and output  areas,  laminar
origins and targets of connections), visuotopic organization
(e.g., mirror-image or non-mirror-image  map  of  hemifield,
bounding  areas,  pattern  of map discontinuities, degree of
retinotopy), and physiological properties (e.g.,  excitatory
receptive  field  size,  direction  selectivity,  attention-
related modulation).  Areas differ in the  degree  to  which
these  criteria have been explored.  V1 (primary visual cor-
tex) and MT  (middle  temporal  area)  are  distinct,  well-
studied  areas  in primates that are convergently identified
by many of these criteria.  Other areas--e.g., in inferotem-
poral cortex--are less well studied. There is no evidence to
suggest that they are any less distinct.

Visual Areas in Prosimians and Monkeys.  The first  primates
were  probably  nocturnal,  judging  from  the large size of
their orbits (Szalay and Delson, 1979).  The primates living
today  most closely related to these early primates are also
nocturnal or crepuscular.  The bush baby or galago, the only
prosimian  primate studied in detail (Allman and McGuinness,
1983; Sereno et al., unpublished studies), has on the  order
of  16  distinct  visual  areas.  Almost all visual areas in
galagos exhibit a substantial degree of retinotopic  organi-
zation,  including  areas  in the inferotemporal cortex.  In
these studies,  the  entire  extent  of  visual  cortex  was
physiologically  mapped  in detail for the first time.  In a
passive animal, visual areas only respond to visual stimuli,
auditory  areas  only to auditory stimuli, and somatosensory
areas only to somatosensory stimuli.  Visual cortical  areas
border  almost  directly upon somatosensory areas (dorsally)
and auditory  areas  (ventrally).   The  transitional  strip
between,  for  example,  auditory and visual areas (in which
neurons have both a visual and an auditory receptive  field)
is less than one millimeter wide.

     Monkeys (anthropoids) are thought to have diverged from
the  ancestors  of  galagos  at  least  40 million years ago
(Szalay and Delson, 1979).  All but one of  the  anthropoids
are  diurnal  (day-living),    suggesting strongly that day-
living habits evolved early in the monkey lineage.  The  one
nocturnal monkey, the New World owl monkey, lacks a tapetum,
suggesting that  its  ancestors  had  diurnal  habits.   The
organization  of visual cortex has been studied in detail in
two different monkeys--the owl monkey and the  macaque  mon-
key.   Figure  1A (hard copy only) shows a flattened summary
map of visual areas in the  owl  monkey  (Allman  and  Kaas,
1976; Weller and Kaas, 1985; 1987; Sereno and Allman, 1990).
As in galagos, V1 is  the  largest  area,  followed  by  V2.
There   appear  to  be  at  least  three  somewhat  separate
'streams' of information passing through V1 and V2--the mag-
nocellular,  parvocellular interblob, and parvocellular blob
streams (named after their relay structures  in  the  dorsal
lateral  geniculate  nucleus and area V1)--that remain some-
what separated as one moves on to higher areas  (Ungerleider
and  Mishkin,  1982;  Livingstone and Hubel, 1984; DeYoe and
Van Essen, 1988; Zeki and Shipp, 1988).  These pathways pro-
cess  different  aspects  of the visual signal in parallel--
roughly, motion, location, and depth  in  the  magnocellular
pathway,  and color, shape, and shading in the parvocellular
pathways.  The pathways pass through  layer  4B,  layer  2-3
interblobs,  and  layer  2-3  blobs  in  V1,  and  the thick
stripes, interstripes, and thin stripes in V2, respectively.
There  is  a  broad  subdivision  of the more rostral visual
areas into parietal (e.g., TP, ST--receiving primarily  mag-
nocellular  stream  input)  and  inferotemporal (e.g., ITcd,
ITr-- receiving primarily parvocellular interblob and parvo-
cellular  blob  input).  Retinotopy is only lost in the most
anterior members of these two streams.   One  can  define  a
hierarchy  of  visual  areas based on the laminar targets of
corticocortical projections; feedforward projections synapse
mainly in layer 4 of the target area, while feedback projec-
tions avoid layer 4 (Rockland and Pandya, 1979; Maunsell and
Van  Essen, 1983; Felleman and Van Essen, 1990).  The border
between different modalities appears to be as  sharp  as  in
galagos; detailed mapping experiments at the anterior border
of visual cortex reveal that the transitional strip  between
visual and somatosensory areas in parietal cortex as well as
the strip between visual  and  auditory  areas  in  temporal
cortex  is less than one millimeter wide (Sereno and Allman,
1990; unpublished studies).

     Figure 1B (hard copy only) shows a similar summary  map
for  the  macaque monkey (an Old World monkey) (based on Van
Essen, 1985; Desimone and  Ungerleider,  1986;  Felleman  et
al.,  1986;  1987; and personal communication; Colby et al.,
1988).  Although many of the areal names are not  the  same,
and  though  the relative sizes of similar areas differ, the
overall configuration of the map, the retinotopic and  func-
tional  organization of individual areas, and the interareal
connection pattern is remarkably similar to our  results  in
the  owl monkey.  New and Old World monkeys diverged over 30
million years ago.  The main difference between the maps  is
the  reduced size of the areas between V2 and MT in owl mon-
keys, the shape of V3 (owl monkey DM, its  probably  homolo-
gue,  is much less elongated than the macaque area), and the
somewhat larger size of several inferotemporal areas.   Most
of  these  differences  reflect  the reduced emphasis on the
center of gaze in the retina of  the  secondarily  nocturnal
owl  monkey.   An  important  point  is  that there does not
appear to be any substantial increase in the area of overlap
between  modalities.   The  zone  in  the dorsal bank of the
superior temporal sulcus that  responds  to  more  than  one
modality  is  several  millimeters wide (Seltzer and Pandya,
1989); this is in line with the greater overall area of  the
primary  cortical  areas  in the macaque compared to the owl

Auditory and Somatosensory Areas in Monkeys.   Auditory  and
somatosensory  areas  have  been  studied  in  parallel with
visual areas.  The main differences are the basis for topog-
raphy  (tonotopy  and  somatotopy  vs. retinotopy), the one-
dimensional  nature  of  tonotopy  (in  contrast   to   two-
dimensional  retinotopy and somatotopy), the smaller overall
size of auditory and somatosensory cortex, and  the  greater
diversity of types of information collected by somatosensory
receptor types (light touch, pain  and  temperature,  muscle
length  changes, force on tendons, joint position).  In both
New and Old World monkeys, there are about 9 auditory corti-
cal  areas (Merzenich and Brugge, 1973; Pandya and Yeterian,
1985) and about 9 somatosensory cortical areas (Merzenich et
al.,  1978; Burton, 1986; Cusick et al, 1989).  As in visual
cortex, one can define a hierarchy of  areas  based  on  the
laminar  targets  of  between-area  projections,  and, as in
vision, there is a successive loss  of  receptotopy  as  one
progresses to higher levels in the two systems.  Most of the
somatosensory maps  are  based  on  responses  to  cutaneous
stimulation  (it is difficult to stimulate muscle and tendon
receptors without also stimulating the skin).

     These  maps  (and  more  fragmentary  data  from  other
species) suggest that the parcellation of most of the cortex
has not changed radically during the evolution of  the  pri-
mate  order  (Sereno and Allman, 1990).  Notably, there does
not seem to be any significant increase in the  areas  where
several  modalities overlap; rather, modality-specific areas
have increased in size, and quite moderately in number;  the
number of cortical areas has probably not changed in New and
Old World monkeys, which have evolved independently for over
30 million years.

Visual Areas in Apes and Humans.  The  organization  of  the
cortex  in a variety of mammals including humans was studied
extensively by Brodmann and others at the beginning  of  the
century  using  stains for cell bodies and myelin (Brodmann,
1909).  Since then,  anatomical  and  physiological  studies
have  revised many of Brodmann's conclusions with respect to
non-human primate brains (e.g., Brodmann's area  18  in  Old
World  monkeys  is  twice  as  wide  as it should have been;
Brodmann's area 19 actually contains many distinct  cortical
areas).   But it is only very recently that human cortex has
been approached  from  a  modern  perspective.   Preliminary
results  suggest that human visual cortical areas are organ-
ized quite similarly to those of other primates.

     The human visual area whose borders are best  known  is
V1--by  far  the  most distinct visual area on architectonic
grounds.  Fixed- tissue injections of membrane-intercalating
dyes suggest that local circuit connections within, and long
range connections between, human areas V1 and  V2  are  very
similar to those of other primates (Burkhalter and Bernardo,
1989).  There is a densely myelinated, ellipsoidal area in a
dorsolateral  occipital  sulcus that may correspond to human
visual area MT, an area found in  all  primates  (Sereno  et
al., 1988; Sereno and Allman, 1990) (see Figure 2).  Studies
using PET to monitor blood flow and a stimulus  designed  to
selectively  activate  MT  (based  on  animal  studies) have
uncovered an active locus near the densely myelinated region
(Miezin et al., 1987).

     Now clearly, there is  a  great  deal  of  'additional'
non-primary  cortex  in  humans.  Despite the fact that mon-
keys, apes and humans all have  about  the  same  number  of
cells  in the retina, the dorsal lateral geniculate nucleus,
and in V1 (Frahm et al., 1984; Tolhurst and Ling, 1988),  V1
comes  to  occupy  a  smaller  and smaller proportion of the
total neocortex--about 10-12% or the neocortex  in  monkeys,
about  6%  of  the neocortex in apes, but only about 2.5% of
the total neocortex in  humans.    The  preliminary  studies
cited  above  suggest  a  new  answer to the problem of this
'extra' cortex in  humans--it  may  be  occupied  mostly  by
larger  versions of areas already familiar from work in mon-
keys (as opposed, for example, to an  evolutionarily  unpre-
cedented  'language  organ').  V2 in humans, for example, is
much wider than  would  be  expected  when  normalized  with
respect  to  the  area of V1.  Similarly, there is much more
area between V1 and the putative  human  MT  than  would  be
expected  (this  region is mostly occupied by area V4 in Old
World monkeys). Finally, the area of the putative  human  MT
is  about  3  to  4  times as big as would be predicted on a
macaque model.  If the other 25 or so extrastriate areas  in
human  visual  cortex  increased in size (relative to V1) as
much as this preliminary data suggests that V2, V4,  and  MT
have,  we  could  almost  completely account for the 'extra'
non-primary cortex in humans  relative  to  monkeys.   These
observations,  combined  with  the  lack of any trend toward
increased polymodal cortex in neocortical evolution, suggest
a radical revision of current neuropsychological theories of
human cognitive processing.

Language Processing in the Context of the Primate Brain.

Modularity and Levels of Explanation.  The question of  what
language processing looks like in the brain is a contentious
one, especially given the preliminary state of  our  current
knowledge  in  this  area.  A certain tradition in cognitive
science and neuropsychology seems to have taken as its goal,
the  isolation  of  higher  levels of explanation from their
lower level implementation. Such  a  so-called  'functional'
approach  is  quite  curious  from a biological perspective.
Surely, biologists are interested  in  function  (e.g.,  the
heart serves as a pump for blood).  But the goal there is to
try to explain how it is that the  structure  of  the  heart
gives rise to its function--not to ignore that structure and
build an  independently  motivated  theory  in  a  different
language  (a language of 'heart'?!).  The fact that the same
program can run on somewhat differently designed von Neumann
machines  (e.g., Fodor and Pylyshyn, 1988) seems an insuffi-
cient  reason  to  abandon  a  biological  and  evolutionary
approach to the functional organization of the human brain.

     This tendency to ignore the structure of the  brain  is
quite  unfortunate  in  light of the recent progress made in
primate neurobiology.  Most current texts  of  physiological
psychology,   neuropsychology,  and  cognitive  neuroscience
(e.g., Damasio and Geschwind, 1984; Caplan, 1987; Ellis  and
Young,  1988) still implicitly employ a model of the organi-
zation and evolution of the cortex that dates to the associ-
ationists  of  the  late nineteenth century.  In this way of
thinking, 'primitive' mammals like rats start out with  pri-
mary visual, auditory, and somatosensory areas almost touch-
ing.  Next up the rung of  an  essentially  pre-evolutionary
scala  natura  come  animals  like  cats, which have a small
amount of 'uncommitted' space in between.  Finally,  at  the
top,  are  primates  and  especially humans, where we find a
great deal of  uncommitted  'association'  cortex,  properly
situated  to  integrate  and associate the modality-specific
information  presented  to  it  by  visual,   auditory   and
somatosensory  cortices  (see  e.g.,  Fodor, 1983; Ellis and
Young, 1988, on the 'semantic  system'  postulated  in  most
models of word processing; Damasio, 1989).

     Fine-grained mapping experiments in hedgehogs, rodents,
cats,  and  primates, during the past decade have shown this
picture of the evolution of  the  cortex  to  be  incorrect.
Cats and primates do have more cortex in between the primary
sensory areas; but that cortex consists  not  of  poly-modal
association  areas,  but  rather  larger  and  more numerous
modality-specific (i.e., visual,  auditory,  and  somatosen-
sory) areas.  The studies discussed above provide no indica-
tion that humans are any different in this regard. The prob-
lem  is,  then, in the spirit of biological studies of func-
tional organization, to try to describe how the  basic  ana-
tomical  modules of primate cortex--namely visual, auditory,
somatosensory, motor, and limbic areas--support a new, pecu-
liarly human function.

Language as Code-directed Scene Perception.  Vision is  very
important  to  primates;  in fact, over 50% of the cortex in
primates,  probably  including  humans,  consists  of  areas
devoted  to  specifically visual processing.  This is not to
deny that information about an object perceived via  another
modality--say  the  somatosensory  system--might  be able to
enter visual areas in the form of a visual copy of the soma-
tosensory  areas' activity pattern (see e.g., experiments by
Haenny et al. (1988) in  macaque  visual  area  V4  using  a
somatosensory-visual  matching  task).   But it does suggest
that we carefully distinguish a visual copy of a  somatosen-
sory  stimulus  (in  a visual area with a visual map) from a
somatosensory copy of a visual stimulus (in a  somatosensory
area with a somatosensory map).

     Some linguists have independently suggested that visual
representations  may  be  very important in the semantics of
natural language (Jackendoff, 1983; 1987; Fauconnier,  1985;
Lakoff,  1987;  Langacker, 1987).  An idea common to several
different approaches is that more concrete  visual  meanings
may  have been extended by analogical processes to deal with
more abstract objects and relations.  The  present  proposal
goes  further  in suggesting a particularly direct relation-
ship  between  the  mechanisms  of   scene   and   discourse

     The integration of successive glances in the comprehen-
sion  of  a  visual scene requires a kind of serial assembly
operation similar in some respects  to  the  integration  of
word  meanings  in  discourse  comprehension.  Primates (but
also many other animals) make long series  of  fixations  at
the  rate  of  several  new  views  per  second during scene
comprehension.  Each fixation brings the  retina  to  a  new
part  of  the visual scene and generates a burst of activity
in V1, which largely replaces the burst caused by the previ-
ous fixation.  Higher visual areas with less precise retino-
topy somehow integrate information from  these  disconnected
activity sequences to generate an internal representation of
the location and identity of the  relevant  objects  in  the
current  scene (e.g., predators, food items, particular con-
specifics, escape routes,  suitable  sleeping  trees,  etc.)
that  can serve as a basis for action.  Many aspects of this
process are redolent of  linguistic  integration--e.g.,  the
underspecified,  context-free  information  in  an  isolated
glance is sharpened and focused by context  (cf.  polysemy);
information  from  temporally  distant  glances must be tied
together (cf. anaphora). None of  this  implies  that  scene
representations  (or their presumed linguistic fellows) need
look anything like pictures; the patterns in question  would
be  distributed across many areas, some of which show little

     One  main  difference  between  scene   and   discourse
comprehension  is,  of  course,  that scene comprehension is
tied closely to the current scene.  Discourse  comprehension
might  best  be thought of as a kind of fictive visual scene
comprehension directed,  in  the  case  of  spoken  language
comprehension,  by  sequences  of phoneme representations in
secondary auditory  cortex.   The  advantage  of  linguistic
discourse comprehension is that we are no longer tied to the
current scene.  However, once the  appropriate  visual  word
meaning patterns have been called up and bound together, the
nature and interactions of the composite pattern may be con-
ditioned mainly by the prelinguistic rules of interaction of
scene representations in primate visual areas networks.   In
this  sense, a large part of what has been called linguistic
syntax and semantics might not be modular  with  respect  to
the neurobiology of vision.

     There is in fact substantial evidence that visual areas
in humans are involved in specifically linguistic functions.
There is a kind of aphasia confusingly called 'transcortical
sensory'  aphasia  (i.e.,  'across-from-the-language-cortex'
aphasia!) that is  generated  by  a  lesion  in  left  human
inferotemporal  cortex  (Rubens and Kertesz, 1983).  Many of
these lesions are so posterior and  ventral  that  they  are
associated  with  overt  visual field defects. Transcortical
sensory aphasics have poor, "Wernicke's-like" comprehension,
yet  paradoxically  (at  least in the context of traditional
models of language comprehension), can repeat words  effort-
lessly.   Far  from being 'across from the language cortex',
the visual areas in posterior inferotemporal cortex  damaged
in  these  patients may be the primary site of semantic pro-
cessing in sighted humans.  Transcortical  sensory  aphasics
recover more quickly than patients with more dorsal lesions;
this may only be an indication that the functions  performed
by   visual   cortex  in  language  comprehension  are  less
lateralized than those performed by auditory  cortex.   This
is  consistent with what we know about primate visual areas;
permanent deficits in visual pattern recognition in  monkeys
require  bilateral  inferotemporal  cortex  lesions  (Gross,
1973).  There is no need to assume  that  all  the  cortical
areas   involved   in  language  comprehension  are  equally
lateralized; for example, the  functions  performed  by  the
superior  temporal gyrus (see below) may be more lateralized
than the functions performed by the inferotemporal cortex.

     Psycholinguistic experiments  using  pictures  inserted
into sentences and picture-word priming (Potter et al, 1986;
Vanderwart, 1984) suggest that it is surprisingly  easy  for
visually  represented concepts to be integrated into ongoing
linguistic discourse comprehension.   This  may  be  another
indicator  of  the  closeness of visual category representa-
tions to linguistic meanings.

Some PET Experiments.  Recently, it  was  suggested  on  the
basis  of  PET  experiments  that semantic processing may be
localized instead in the frontal  lobe,  just  in  front  of
"Broca's area" (Petersen et al., 1988; Posner et al., 1988).
In the key  experiment,  subjects  performed  two  tasks--1)
repeating visually presented nouns, and 2) generating "uses"
(related verbs) upon viewing an otherwise comparable  series
nouns.   Upon subtracting these two conditions, an activated
locus was uncovered in frontal cortex, just anterior to  the
representation  of  face, tongue, and throat muscles in pri-
mary motor cortex.  Given the ease  with  which  preparation
for  movement  elicits  strong  activation in premotor areas
(see e.g., Roland et al., 1980), however,  it  seems  likely
that  the  activity  uncovered  in  this experiment actually
represents the different motor programming  demands  of  the
two  tasks.  In the first case, a motor pattern is called up
directly via overlearned  connections  between  visual  word
shape  and  articulatory  movements.  In the second case, by
contrast, the subject must make a new motor plan  to  say  a
word that is different from that which was viewed.  In fact,
the subject must also suppress an output that would normally
be generated by looking at the first word (in the context of
reading words aloud).  Frontal cortex lesions in monkeys and
man  are  known  to  especially  impair  the ability to make
delayed responses.  Given that posterior inferotemporal cor-
tex has rarely if ever been selectively activated in a blood
flow experiment, and that  the  PET  technique  has  limited
resolution,  the  activation  underlying semantic processing
may not yet have been seen.  A posterior locus for semantics
is  more  in  line  with  the observation made long ago (and
hardly overturned by more recent studies) that patients with
large  posterior lesions are generally much more impaired in
extracting meaning  from  linguistic  discourse--and  surely
seem  to  have  a  much  more  severe derangement of thought
processes--than patients with large anterior lesions.

What's in Wernicke's Area?   Wernicke's  area  has  occupied
several  different  gyri  over  the  years.  Sometimes it is
placed on the angular gyrus; sometimes it sits more  anteri-
orly  on  the  superior  temporal gyrus; and often it sneaks
across the superior temporal sulcus  (the  boundary  between
auditory  cortical  areas dorsally and visual cortical areas
ventrally in primates) to sit partly in inferotemporal  cor-
tex.   The  left-right  asymmetry originally demonstrated by
Geschwind and Levitsky (1968) was in yet a different  place-
-on  the  planum  temporale  (not  even clearly visible in a
lateral view).  Several architectonic studies (Braak,  1978;
Galaburda and Sanides, 1980) have identified a distinct area
that shows a considerable left-right asymmetry (Braak's tem-
poral  magnopyramidal zone; Galaburda and Sanides' area Tpt)
confined entirely to the posterior part of the lateral supe-
rior  temporal  gyrus.   By  comparison with other primates,
this area is very likely to be a unimodal, secondary or ter-
tiary  auditory  cortical  area. Merzenich and Brugge (1973)
recorded diffuse auditory responses  from  a  geographically
similar area in macaques.

     If Wernicke's area proper (e.g., of Braak) is in fact a
secondary  or tertiary auditory area, we are left with some-
thing of a conundrum.  Why should a lesion  in  an  auditory
area  cause deficits in the assembly of the meaningful units
of language?  The deficits exhibited by many patients with a
lesion in this area seem to extend beyond mere problems with
auditory  representations  of  words--their  thoughts   seem
disarranged;  often they are unable to manipulate even words
with concrete visual meanings.  The  traditional  conclusion
has thus been that Wernicke's area must be an evolutionarily
new 'language  organ'  not  tied  to  one  modality.  A  new
interpretation  more  in line with the animal literature, is
that the internal representations of speech sound  sequences
that  a  primate  neurobiologist  would  expect  to  find in
Wernicke's area proper must have some other function besides
merely  serving  as  internal  copies  of the speech stream;
these uninterpreted speech sound representations  must  also
be  involved  in word recognition and assembly of (primarily
visual) meanings into  coherent  discourse  structures.   By
this  account,  what  distinguishes humans is the ability to
use a sequence of symbol patterns from another  modality  to
cause  the  assembly  of meaning patterns in tertiary visual
cortex.  But the product of that assembly may be very  simi-
lar to patterns assembled from direct visual inputs arriving
via V1 during scene comprehension.  The implication is  that
the  trick  of  language  was not to have invented the basic
meaningful  units  but  to  have  found  a  way  of   making
standardized  connections  between  them  (see Sereno, 1986;
1990a; 1990b, for an extended discussion).

     In monkeys, the  superior  temporal  sulcus  forms,  as
noted,  the  border  between  auditory  and visual cortices.
Since clinically defined Wernicke's-like aphasics often have
lesions  that  extend  into the inferotemporal region on the
middle and inferior temporal  gyri,  a  typical  'Wernicke's
aphasia'  may  require  damage  to  both the auditory cortex
meaning assemblers  and  the  visual  cortex  meanings  they

New Routes Between  Modalities.   In  monkeys,  one  pathway
responsible  for  cross-modal  matching performance has been
well-defined.  Performance on somatosensory-visual  matching
tasks is catastrophically impaired by lesions to the basola-
teral amygdala (Murray and Mishkin, 1985).  This part of the
amygdala  receives  projections  from secondary and tertiary
visual, somatosensory, and auditory areas, and projects back
to  them.   There is also a small polymodal strip on part of
the upper  bank  of  the  superior  temporal  sulcus  (e.g.,
Seltzer  and Pandya, 1989).  But this strip cannot by itself
support cross-modal matching in monkeys.

     The situation in humans must be somewhat different,  at
least with regard to the relative importance of the amygdala
in one particular kind of cross-modal mapping  that  charac-
terizes  human  language--the  mapping between speech sounds
and visual word meanings.  The  patient  H.M.  who  had  his
amygdala  removed  bilaterally is quite unimpaired in recog-
nizing visual objects named for him  (or  in  naming  visual
objects  himself).   This  suggests  that humans must have a
more robust connection between areas on either side  of  the
superior  temporal  sulcus  than  monkeys  do.   Cross-modal
matching experiments of the kind that amygdala-lesioned mon-
keys  fail to perform have not yet been tried with H.M., and
so the cross-modal pathway through the amygdala  could  very
well still be important for some tasks in humans.


Language is recently derived; based on the evidence of stone
tools  and other more spectacular artifacts like cave paint-
ings, it seems likely that peculiarly  human  cognition  and
presumably language use originated rather suddenly less than
50,000 to 100,000 years ago. In view of our knowledge of the
strong  similarities between the brains of various non-human
primates, it seems unlikely that the cortex could have  been
completely  reorganized in so short a time. Surely, there is
no  positive  evidence  for  such  a  major  reorganization.
Recent  evidence  instead  suggests that human and non-human
primate brains are organized quite similarly.  We need  more
attempts   to  explain  the  large  qualitative  differences
between animal cognition and human language-based  cognition
as  the  result of relatively minor modifications and re-use
of pre-existing primate neural circuitry (cf. Bates et  al.,

     This paper suggests that it might be profitable to view
language  comprehension in sighted people as a kind of code-
directed scene comprehension taking place primarily in  uni-
modal  visual  areas  in posterior inferotemporal cortex.  A
second suggestion is that internal representation of  speech
sound   chains   in   secondary   auditory   cortical  areas
(Wernicke's area proper)  may have other  functions  besides
merely  serving as internal copies of the speech code chain;
they may be intimately involved in word recognition and  the
binding  together  of visual cortex meaning patterns.  Code-
directed pattern binding is  clearly  a  specifically  human
faculty;  but  many  of  the  constraints  on  the resulting
bound-together  patterns  may  reflect  prelinguistic  (non-
modular)  constraints  on interactions between activity pat-
terns in tertiary visual areas. Studies of  the  connections
of  superior  temporal  sulcus  region  in  humans--just now
becoming possible--may throw more  light  on  the  presently
obscure neural substrate of language and human thought.


Allman, J.M. and J.H.  Kaas  (1976)  Representation  of  the
  visual field on the medial wall of occipital-parietal cor-
  tex in the owl monkey.  Science 191:572-575.
Allman, J.M. and E. McGuinness (1983)  The  organization  of
  visual  areas  in  a  strepsirrhine  primate, Galago sene-
  galensis.  Society for Neuroscience, Abstracts 9:957.
Bates, E., D. Thal, and V. Marchman (1989) Symbols and  syn-
  tax:  Darwinian  approach  to language development.  Tech.
  Rep. 89-04, Project in Cognitive and  Neural  Development,
  Univeristy of California, San Diego.
Braak, H. (1978) On magnopyramidal temporal  fields  in  the
  human   brain--probable   morphological   counterparts  of
  Wernicke's sensory speech region.  Anatomy and  Embryology
Brodmann, K.  (1909)  Vergleichende  Lokalisationslehre  der
  Grosshirnrinde.  Leipzig: J.A. Barth.
Burkhalter, A. and K.L. Bernardo (1989) Organization of cor-
  ticocortical  connections in human visual cortex. Proceed-
  ings of the National Academy of Sciences 86:1071-1075.
Burton, H. (1986) Second somatosensory  cortex  and  related
  areas.  In  E.G. Jones and A. Peters (eds.), Cerebral Cor-
  tex, Volume 5.  Plenum Press, pp. 31-98.
Caplan, D. (1987) Neurolinguistics and linguistic  aphasiol-
  ogy.  Cambridge University Press.
Cusick, C.G., J.T.  Wall,  Felleman,  D.J.,  and  J.H.  Kaas
  (1989)  Somatotopic  organization of the lateral sulcus of
  owl monkeys: area 3b, S-II, and  a  ventral  somatosensory
  area.  Journal of Comparative Neurology 282:169-190.
Damasio, A.R. and N. Geschwind (1984) The  neural  basis  of
  language. Annual Review of Neuroscience 7:127-147.
Damasio, A.R. (1989) The brain binds entities and events  by
  multiregional  activation  from convergence zones.  Neural
  Computation 1:123-132.
Desimone, R. and L.G.  Ungerleider  (1986)  Multiple  visual
  areas  in  the  caudal  superior  temporal  sulcus  of the
  macaque.  Journal of Comparative Neurology 248:164-189.
DeYoe, E.A. and D.C. Van Essen (1988) Concurrent  processing
  streams  in monkey visual cortex.  Trends in Neurosciences
Fauconnier, G. (1985) Mental Spaces.  MIT Press.
Felleman, D.J., J.J. Knierim, and D.C. Van Essen (1986) Mul-
  tiple  topographic and non-topographic subdivisions of the
  temporal lobe  revealed  by  connections  of  area  V4  in
  macaques.  Society for Neuroscience, Abstracts 12:1182.
Felleman, D.J., A. Burkhalter, and  D.C.  Van  Essen  (1986)
  Visual area PIP: an extrastriate cortical area in the pos-
  terior  intraparietal  sulcus  of  the   macaque   monkey.
  Society for Neuroscience, Abstracts 13:626.
Felleman,  D.J.  and  D.C.  Van  Essen  (1990)   Distributed
  hierarchical processing in primate visual cortex. (submit-
  ted to Cerebral Cortex).
Fodor, J.A. (1983) The Modularity of Mind.  MIT Press.
Fodor, J.A. and Z.W. Pylyshyn (1988) Connectionism and  cog-
  nitive   architecture:  a  critical  analysis.   Cognition
Frahm, H.D., H. Stephan, and G. Baron (1984)  Comparison  of
  brain  structure  volumes in Insectivora and Primates.  V.
  area striata (AS).  Journal fur Hirnforschung 25:537-557.
Galaburda, A.M.  and  F.  Sanides  (1980)  Cytoarchitectonic
  organization  of  the  human  auditory cortex.  Journal of
  Comparative Neurology 190:597-610.
Garfield,  J.L.  (ed.)  (1987)   Modularity   in   Knowledge
  Representation  and  Natural-Language  Understanding.  MIT
Gross, C.G. (1973) Inferotemporal cortex and  vision.   Pro-
  gress in Psychobiology and Physiological Psych. 5:77-123.
Haenny, P.E., J.H. Maunsell, and P.H. Schiller (1988)  State
  dependent  activity  in monkey visual cortex.  II. retinal
  and  extraretinal  factors  in  V4.   Experimental   Brain
  Research 69:245-259.
Jackendoff, R. (1983) Semantics and Cognition.  MIT Press.
Jackendoff, R. (1987) Consciousness  and  the  Computational
  Mind.  MIT Press.
Kaas, J.H. (1987) The origin and evolution of neocortex.  In
  S. Wise (ed.) Higher Brain Functions.  Wiley, pp. 347-378.
Lakoff, G. (1987) Women, Fire, and Dangerous Things. Univer-
  sity of Chicago Press.
Langacker, R. (1987) Foundations of Cognitive Grammar. Stan-
  ford University Press.
Livingstone, M.S. and D.H. Hubel (1984) Anatomy and physiol-
  ogy of a color system in the primate visual cortex.  Jour-
  nal of Neuroscience 4:309-356.
Maunsell, J.H.R. and D.C. Van Essen (1983)  The  connections
  of  the  middle temporal areas (MT) and their relationship
  to a cortical hierarchy in  macaque  monkey.   Journal  of
  Neuroscience 3:2563-2586.
Merzenich, M.M. and J.F. Brugge (1973) Representation of the
  cochlear  partition  on the superior temporal plane of the
  macaque monkey.  Brain Research 50:275-296.
Merzenich, M.M., J.H. Kaas, M. Sur,  and  C.-S.  Lin  (1978)
  Double  representation  of the body surface within cytoar-
  chitectonic areas 3b and 1 in "S-I"  in  the  owl  monkey.
  Journal of Comparative Neurology 181:41-74.
Merzenich, M.M. and J.H. Kaas (1980) Principles of organiza-
  tion of sensory-perceptual systems in mammals.  rogress in
  Psychobiology and Physiological Psychology 9:1-42.
Miezin, F.M., P.T. Fox, M.E. Raichle, and J.M. Allman (1987)
  Localized responses to low contrast moving random dot pat-
  terns in human visual cortex monitored with positron emis-
  sion tomography. Soc. Neurosci., Abstr. 13:631.
Murray, E.A. and M.  Mishkin  (1985)  Amygdalectomy  impairs
  crossmodal association in monkeys.  Science 28:604-606.
Pandya, D.N. and E.H. Yeterian (1985) Architecture and  con-
  nections  of cortical association areas.  In A. Peters and
  E.G. Jones  (eds.), Cerebral  Cortex,  Volume  4.   Plenum
  Press, pp. 3-61.
Petersen, S.E., P.T. Fox, M.I. Posner, M. Mintun,  and  M.E.
  Raichle  (1988)  Positron  emission tomographic studies of
  the cortical anatomy of  single-word  processing.   Nature
  331:585- 589.
Posner, M.I., S.E. Petersen,  P.T.  Fox,  and  M.E.  Raichle
  (1988)  Localization  of cognitive operations in the human
  brain. Science 240:1627-1631.
Potter, M.C., J.F. Kroll, B. Yachzel, E. Carpenter,  and  J.
  Sherman   (1986)   Pictures  in  sentences:  understanding
  without words. Journal of Experimental Psychology: General
  115:281- 294.
Rockland, K.S. and D.N. Pandya (1979)  Laminar  origins  and
  terminations of cortical connections of the occipital lobe
  in the rhesus monkey.  Brain Research 179:3-20.
Roland, P.E., B. Larson, N.A. Lassen, and E. Skinhoj  (1980)
  Supplementary  motor  area  and  other  cortical  areas in
  organization of voluntary movements in  man.   Journal  of
  Neurophysiology 43:118-136.
Rubens, A.B. and  A.  Kertesz  (1983)  The  localization  of
  lesions  in  transcortical aphasias.  In A. Kertesz (ed.),
  Localization in Neuropsychology.  Academic Press, pp. 245-
Seltzer, B, and D.N. Pandya (1989) Intrinsic connections and
  architectonics  of  the  superior  temporal  sulcus in the
  rhesus monkey.  Journal of Comparative Neurology  290:451-
Sereno, M.I. (1986) A program for the neurobiology of  mind.
  Inquiry 29:217-240.
Sereno, M.I. (1988) The visual system.   In  I.W.v.  Seelen,
  U.M.  Leinhos,  and G. Shaw (eds.), Organization of Neural
  Networks. Weinheim: VCH Verlagsgesellschaft, pp. 167-184.
Sereno, M.I., C.T. McDonald, and J.M. Allman (1988) Myeloar-
  chitecture  of flat-mounted human occipital lobe: Possible
  location of visual area  MT.   Society  for  Neuroscience,
  Abstracts 14:1123.
Sereno, M.I. and J.M. Allman (1990) Cortical visual areas in
  mammals.   In  A.G.  Leventhal  (ed.), The Neural Basis of
  Visual Function.  London: Macmillan, pp. 160-172.
Sereno, M.I. (1990a) Four analogies between  biological  and
  cultural/linguistic  evolution.   Journal  of  Theoretical
  Biology (in press).
Sereno, M.I. (1990b) DNA and Language.  (under  contract  to
  be published by MIT Press).
Szalay, F. and E. Delson (1979) Evolutionary History of  the
  Primates.  Academic Press.
Tolhurst, D.J. and L. Ling (1988) Magnification  factor  and
  the  organization of the human striate cortex.  Human Neu-
  robiology 6:247-254.
Ungerleider, L.G. and M. Mishkin (1982) Two cortical  visual
  systems.   In  D.J.  Ingle, M.A. Goodale, and R.J.W. Mans-
  field (eds.), Analysis of Visual Behavior, MIT Press,  pp.
Van Essen (1985) Functional organization of  primate  visual
  cortex.  In E.G. Jones and A. Peters (eds.), Cerebral Cor-
  tex, Volume 3.  Plenum Press, pp. 259-329.
Vanderwart, M. (1984) Priming by pictures in  lexical  deci-
  sion.  Journal of Verbal Learning and Verbal Behav. 23:67-
Weller, R.E. and J.H. Kaas (1985)  Cortical  projections  of
  the  dorsolateral visual area in owl monkeys: the prestri-
  ate relay to inferior temporal cortex.   Journal  of  Com-
  parative Neurology 234:35-59.
Weller, R.E. and J.H. Kaas (1987) Subdivisions  and  connec-
  tions of inferior temporal cortex in owl monkeys.  Journal
  of Comparative Neurology 256:137-172.
Zeki, S. and S. Shipp (1988)  The functional logic of corti-
  cal connections.  Nature 335:311-317.

             Captions for Figures in hard copy

     Figure 1.  Cortical visual areas in the owl monkey  (A)
and the macaque monkey (B). A cut was made in V1 medially to
allow the cortex to lie flat.   The  insets  illustrate  the
location  these  areas  in occipital, parietal, and temporal
cortex (after Sereno and Allman, 1990).  All areas shown are
visual  except  for  area  PM  (owl  monkey)  and  area  STP
(macaque),  which  border  on  somatosensory  and   auditory
cortices (not shown).

     Figure 2.  Cortical visual areas in the human  (prelim-
inary).  A  left  occipital  lobe (reversed here to aid com-
parison with previous  figures)  was  physically  flattened,
sectioned,  and  stained  for myelin.  The exposed crowns of
the gyri are colored black.  A cut was made in  V1  medially
to  allow the cortex to lie flat.  The insets illustrate the
location these areas in the intact brain (after  Sereno  and
Allman, 1990).  Note that the scale is now in centimeters.

[CRL Newsletter Home Page]
[CRL Home Page]
Center for Research in Language
CRL Newsletter
Article 4-4