Learning Language from the Input

“Learning Language from the Input: Why Innate Constraints Can’t Explain Noun Compounding” is now ‘trending’ on SciVerse.  The paper, which was published in the February edition of Cognitive Psychology, was one of the journal’s top 5 most downloaded articles in the last three months.  The 7-experiment paper gives a detailed account of why children’s learning reflects the statistical patterns seen in the input and not, as has been frequently claimed, the working of a native rule-based constraint.  If you are interested in our corpus-based research, you may wish to skip to Exp. 7 and our subsequent analyses (pp. 28-35).

One of our favorite quotes from the paper?

“Thought-experiments, by their very nature, run into serious problems when it comes to making hypothesis blind observations, and because of this, it seems reasonable to suggest that their results should be afforded less credence in considering the phenomena themselves.” (p. 35)

The world is your lobster

This morning, Mark Liberman over at Language Log wrote an intriguing post about how people use the word “snuck” in conversation, but then sneakily use the word “sneaked” in writing.  At the end of his post, Mark recommends everyone read Donald Davidson’s brilliant article A Nice Derangement of Epitaphs on the puzzles posed by malapropisms, and ‘passing theories’ of language.

We were also very much inspired by Davidson’s paper, which motivated two recent research efforts in our lab:

1. Princesses, peas and probabilities: readers’ sensitivity to the “surface” statistics of literary and non-literary English.

On how readers of fiction and non-fiction differ in their sensitivity to distributions of words in literary and non-literary texts.

2. Taking language for granite: On the comprehensibility of malapropisms.

On how listeners parse (and remember) malapropisms in speech, depending on the predictability of the preceding context.

*Earlier readers may have noticed that we managed to mangle the spelling of “Liberman” in the original post. We offer our sincere apologies to Mark, whose name — all things considered — shouldn’t be that difficult to spell. All we can say by way of excuse is – we’re not the first?

April CNS Meeting

In April, Profs Michael Ramscar and Sam McClure will be presenting at the 18th Annual Cognitive Neuroscience meeting in San Francisco. They will report the first neurobiological evidence for Feature-Label-Order Effects in learning.

Manipulating information structure as a method of localizing information processing in the brain

When formalized in terms of prediction and cue competition, symbolic learning takes two forms: learning to predict labels from the features of objects and events (Feature-to-Label learning), or learning to predict features from labels (Label-to-Feature learning). When the information available in training is structured in one or another of these formats, qualitative differences in symbolic learning occur. Discrimination learning is facilitated when objects precede labels (FL), because the structure of information promotes cue competition between individual features. However, this competition is inhibited when labels predict objects (LF; Ramscar et al, 2010). We report an fMRI investigation of these Feature-Label-Ordering effects in learning. Participants were trained and tested on a category-learning task while the frequency of confusable categories was manipulated so that successful discrimination was essential to successful categorization. Participants trained to predict labels from features (FL) showed higher levels of dorsal striatal activity (caudate and putamen), which correlated with overall performance at test. The opposite pattern was observed with ventrolateral prefrontal cortex (VLPFC) activation, which was greater in participants trained to predict features from labels (LF), and which correlated negatively with performance on the difficult to discriminate low frequency items. The increased striatal activity we observed in the FL-trained participants is consistent with evidence linking this area to discrimination learning, while the correlation between VLPFC activity and poorer discrimination in the LF-trained participants supports the idea that the structure of information in training forced participants to rely on working memory, fixating on cues that were frequent, salient, and yet ultimately uninformative.

Open Science Starts Here

In January, lab researcher Melody Dye helped chair a panel on “What’s Keeping Us From Open Science?” at ScienceOnline 2011. (A video of the session is now available at Wired.com.) In the spirit of opening science here at the lab, we have decided to to upload preprints of 12 articles that have already been accepted for review at various journals (including Nature, Science, Psych Review, Psych Science, Cognitive Science, Cognition, Child Development and Developmental Science). These can now be downloaded from the “Access Articles” page.

Because these articles will almost certainly undergo revision before publication, we ask that if you are interested in citing any of them, that you contact us for final copies and/or publication information.  Please feel free to email with comments, suggestions or questions. If you are interested in reading about a model of Open Science our lab endorses, please see Axel Boldt’s essay “Extending Arxiv.org to Achieve Open Peer Review and Publishing.”

Save the Date

The year’s calendar is filling up fast!

• This month, Michael will be giving a talk at UCSD for the Quantitative Measures in Morphology and Morphological Development Workshop on findings with Richard Futrell and Dan Jurafsky on the evolution of grammatical gender.  That same weekend, Melody will be holding down the fort at Science Online 2011 and advocating for open science.
• In February, Cognitive Psychology will publish a seven-experiment paper on how children learn language from the input (Ramscar & Dye, 2011).
• In April, Michael will join forces with Sam McClure and present on the neurobiological underpinnings of FLO-Effects in language learning at the annual Cognitive Neuroscience Society meeting in San Francisco.
• May will find Michael traveling to Colorado, as an honorary B.F. Skinner lecturer at the ABAI’s annual conference.  He will be lecturing on Language as Prediction.  Also look out this month for a feature story in Scientific American Mind on how children learn their colors!

Lab Research to be published in Scientific American

Last June, lab researcher Melody Dye wrote a piece for Scientific American’s Mind Matters column on the lab’s research into color learning.  That piece will now feature in an upcoming issue of the magazine’s print edition!  The article has also been selected to feature in this year’s Open Lab Anthology. If you’re curious to learn more about the research, last year’s Cognitive Science article (see pp. 932-7) and conference proceedings are a good place to start.

In short, our research team discovered that a simple principle of learning provided crucial insight into how best to structure color learning for young children, making color words dramatically easier for them to learn. More recently, the team has extended these principles to number learning and discovered that the same intervention significantly benefits early number categorization.  In that experiment, the improvements seen over a 15-minute training period were similar to those seen across a six-month longitudinal study. This finding was replicated across 3 samples of 2.5 – 3.5 year olds, and could have significant implications for early mathematics education.

Letter in the APS Observer

Michael wrote a brief letter on citation practices, which appears in this month’s APS Observer.

The letter is reproduced below:

I read Roddy Roediger’s recent column on the measurement of journal quality with great interest. As Roddy noted, while “psychologists usually like to think that the behaviors generated by human beings will arrange themselves along a normal curve,” the distribution of citations has a strongly positive skew: most papers are very rarely cited, while a small few are cited often (i.e., if citations items are rank-ordered by frequency, the distribution represents an inverse power function).

One psychological scientist who probably wouldn’t have been surprised by this is George Kingsley Zipf, who discovered that the distribution Roddy noted in citations is ubiquitous in language — so that if you look at the pattern of overall word use in any document or language, it exhibits the same right-skewed pattern of distribution.

One way of thinking about the pattern of behavior Roddy observed is that it is just as we might expect if scientists’ citation behavior obeyed the same principles as other linguistic behaviors.  That is, given what we have read the literature, if we need to cite an article after making point X, the distribution in the literature we have already read means that article Y is the most likely candidate to spring to mind, and so by default we cite Y; which means that the citation simply reproduces and reinforces the existing distribution.

Tellingly, Zipf described his law as the “the principle of least effort” in communication.

Although the relationship between Zipf’s law and the distribution of citations has long been noticed by information scientists (e.g., Liming and Lihua, 1993; Perc, 2010; see also de Solla Price, 1965) the idea of framing the question of what makes papers get cited in terms of the mechanisms underlying the behavior of the people who actually make those citations ought to be of particular interest to psychologists.  While most of us might like to suppose that we behave rationally when it comes to choosing our citations, the statistics imply that a less reflective approach to citations may well be the norm. Assuming that there is actually any substance to these speculations, this in turn raises an interesting question: how are we to weigh the impact of these underlying psychological factors in determining the actual impact of a given journal or article?

— Michael Ramscar
San Francisco, CA

BloggingHeadsTV ‘Science Saturdays’

Melody and Michael’s chat on learning theory, capacity limitations and color learning, goes live today over at BloggingHeadsTV ‘Science Saturdays.’ Unfortunately, as may be obvious to viewers, there were a number of technical difficulties the two experienced over the course of the chat — the connection was broken and choppy at several points, and there was construction in the background, making it difficult to hear. This leads to some epic failures of “language as prediction” as the two try to navigate some of the topics across a (literally) noisy channel. However, whether you’re watching for content — the occasional befuddlement of the hosts — or simply to add to your understanding of how predictive processing works in the wild — it may add a couple of interesting minutes to your Saturday.

If you’re interested in some of the research discussed in the chat, we recommend “Computing Machinery and Understanding” and “The Effects of Feature-Label-Order and their Implications for Symbolic Learning,” available in our articles stockpile.

CSDL / ESLP Conference

This weekend, Prof Michael Ramscar and researcher Melody Dye will be attending the joint meeting of the Conceptual Structure, Discourse and Language Conference with the The Embodied and Situated Language Processing Workshop in San Diego.  They will be presenting a number of posters on cutting-edge research, and Melody will be giving a talk on Sunday afternoon.

Lab co-authors and contributors to these presentations include Hanna Popick, Joseph Klein, Edward Suh, Justine Kao, Robert Ryan, and Fiona O’Donnell-McCarthy. Running Down the Clock, which was published as a journal article just this past year, was co-authored with UC Merced Prof Teenie Matlock, a long time lab collaborator.

You can access the conference schedule here.

• Directional Effects and the Distributional Hypothesis
• Running Down the Clock : The Role of Expectation in our Understanding of Time & Motion (poster)(journal article)
• Ordering Effects in the Acquisition of Number Words (poster)
• A Steep Price to Pay? On the Costs and Benefits of Learning Relative Pitch (poster)
• Readers’ Sensitivities to the Surface Statistics of Literary and Non-Literary Writing (proceedings)
• What can Blocking Effects Tell Us About Mutual Exclusivity? (poster)

August Papers

The Effects of Feature-Label-Order and their Implications for Symbolic Learning” has just been published in Cognitive Science, along with “Computing Machinery and Understanding.”  These articles suggest that language can best be understood and modeled as a predictive process.  We recommend reading the Reply first, before delving into the FLO-Effects paper.  You might also be interested to read the popular send-ups of this research : BigThink cited “The Mind is a Search Engine” as an Idea of the Day, and Scientific American Mind ran an article explaining “Why Johnny Can’t Name His Colors.”

Keywords: language, prediction, processing, cognitive modeling, concepts, categories, time, reference.