Embrace the Data
By Alex Danvers
What words can classify a movie review as positive? What words classify it as negative?
In the symposium Big Data: Vast Opportunities for Psychological Insight from Mining Enormous Datasets at the SPSP Annual Convention, Harvard economist Sendil Mullainathan threw up some obvious candidates, like “dazzling” or “gripping”—words that researchers brainstormed would do a good job. Using these “theory-grounded” words, a team of computer scientists was able to classify reviews with 60% accuracy—not much of an improvement over 50/50 guessing.
But when the computer scientists let the model empirically determine what was most predictive, some surprising candidates jumped out. For example, the word “still”—as in, “I didn’t like the acting, but still I felt compelled to watch the cinematography”—was highly predictive of a positive review. Using the empirically selected words, a machine learning algorithm was able to classify movie reviews with 95% accuracy.
These contrasting models, according to Mullainathan, represent a shift in ways to approach studying intelligence. Early pioneers in psychology, like Herb Simon, began trying to create artificial intelligence through introspection—reasoning through what processes they followed in order to solve a problem. Once they figured that out, they assumed it would be a simple matter to train up a computer to mimic human processes.
But human intelligence and machine intelligence turn out to be very different. We lack self-knowledge, and there are many tasks—like statistical inference—that humans are known to perform poorly at. What the era of machine learning and big data can offer us is a way of flipping the problem of intelligence from one of introspection to one of empiricism. Ignore intuition. Embrace the data.
Mullainathan was one of four speakers exploring this bottom-up approach, finding surprising results extracted from a huge stack of observations.
In the first talk, Emily Oster found that a reliable change could be found in household food consumption after one member of the household was diagnosed with diabetes. Using only “scanner data”—a record of barcodes from household purchases over several years and over 100,000 people—she was able to detect a statistically significant decrease in consumption of “bad” or unhealthy food after an individual began purchasing diabetes-related products. There was no increase in “good” foods.
In the second talk, Michal Kosinski was able to use Facebook profile pictures to predict Big Five personality traits with significant accuracy: just over 20% for extraversion, agreeableness, and neuroticism, and over 10% for openness and conscientiousness.
Although the deep learning algorithm he used to make these predictions is in some ways opaque, one finding that emerged across different genders and ethnic groups was an association between a broader face and introversion. Earlier research on people’s intuitive judgments of faces had suggested just the opposite: we tend to believe wide faces mean extraversion.
Johannes Eichsteadt used word frequencies from people’s tweets to predict personality, finding that his algorithms matched the accuracy ratings of a close friend—and exceeded it significantly in predicting openness.
Some entertaining findings: introversion is predicted by use of the words “manga,” “anime,” and “pokemon;” extraversion is related to “party” and “!!!”
Eichsteadt also used tweets to categorize counties according to their prevalence of heart disease. He compared his predictions to actual CDC incidence ratings, and found that Twitter alone was a better predictor than all of the most common demographic risk factors combined.
Tweets indicating hostility, aggression, and boredom seemed to indicate a county where heart disease would be high. Tweets indicating skilled occupations, positive experiences, and optimism indicated lower incidence of heart disease.
Finally, Mullainathan used a machine learning algorithm to classify which criminal offenders should be left in jail while awaiting crime. He found that, if we are comfortable with the current 18.7% crime rate, we could be releasing 78% of people—as compared to the 61% that judges currently release. Using these algorithms to determine risk of flight or further crime would significantly decrease burden on U.S. jails and could lead to huge savings.
When we leverage the particular strengths of machine intelligence, we can find surprising and effective new ways of solving problems. This does not mean ceding ground to computers; instead we should use our uniquely human ability to extend our cognitive capacities through tools to explore behavior and mind in ways we never have before.
Alex Danvers is a PhD student in social psychology studying emotions in social interactions. He uses dynamical systems and evolutionary perspectives, and is interested in new methods for exploring psychological phenomena.