The Voice of Confidence – How Listeners Decode a “Feeling of Knowing”

By Xiaoming Jiang

As social animals, human beings possess a unique ability to communicate social intentions—such as a feeling of knowing—in their tone of voice. Vocal confidence can be used as a signal of one’s persuasiveness, expertise, and trustworthiness. However, scientific investigations of vocal confidence are relatively scarce. As a postdoctoral researcher in the Neuropragmatics and Emotion Lab led by Dr. Marc Pell at McGill University, I am working on an ambitious project investigating how the human voice can encode a speaker’s feeling of knowing, and how the human brain has evolved the ability to decode this complex mental state.

Perceptual and vocal attributes of voice of confidence

In a recent communication on social prosody (Jiang & Pell, 2014), we reported on a newly validated vocal database of over 3,600 expressions produced in North American English. We invited 6 professional actors or public speakers to record statements intended to convey that they were very confident, close to confident, or non-confident (in addition to a neutral comparison). These expressions were led by verbal probability phrases such as I’m confident, Mostly likely, or Perhaps which were congruent with the tone of voice in the expression. These cues were used to help speakers produce the appropriate confidence level, but the recordings were edited so that listeners could hear versions of the statements both with and without the explicit linguistic cue. A group of 60 native speakers ratings of the intended levels of confidence tracked the speakers’ intentions with high accuracy. The average confidence rating was lowest for non-confident voices (2.3), followed by close-to-confident (3.5), and neutral (4.0), and was highest for confident voices (4.5). We found that the confident voice, as compared with the non-confident one, was characterized by specific acoustic signals: a lower pitch, reduced loudness, more flat intonation, faster speech rate, and more restricted change in loudness with the unfolding of the vocal expression.

How fast can we decode the voice of confidence?

Our lab followed up on these findings by investigating how these differences in perceiving confidence relate to neural signals. We recorded electroencephalograms (EEG) while listeners decoded vocal signals of varying levels of confidence. When only vocal cues were available (the linguistic cues like “perhaps” were not included), there were neural differences in the confident and non-confident voices after only 200ms. These neural signals appear to reflect increased attentional processing for the confident voice. The close-to-confident voice received more continuous attention over the course of the statement, as reflected by a larger positive wave evoked at 370ms when processing this voice compared to the more definitive confident or non-confident voices. Neutral-intending voices tended to be rated as reasonably confident, and neutrally they were best distinguished from other voices in a late positive wave occurring at around 900ms after the speech act. This may reflect a mismatch between the task goal of rating confidence and the speaker intention—not to include information about intention (Jiang & Pell, 2015).

We were also interested in how vocal and verbal speech signals interact. We found that, when vocal cues are combined with verbal cues, the differentiation of confident and non-confident voices occurred earlier—at 100ms. Compared with vocal-cue only expressions, neural responses to expressions with combined verbal and vocal cues were greatly reduced. Verbal information thus appears to facilitate the processing of vocal information for inferring speaker confidence, leading to less heightened neural activity but also to more rapid neural differentiation between varying levels of confidence (Jiang & Pell, 2016a).

How can we resolve conflicting message in speech signaling feeling of knowing?

We examined how lexical and vocal cues can facilitate each other, but we were also interested in how people decode mixed messages—those where verbal and vocal cues conflict in conveying feelings of (un)knowing. We reported two different cases, where a listener’s brain appears to decode with distinct strategies (Jiang & Pell, 2016b). Hearing I’m confident followed by unconfident voice places a heavy burden on the integration and update of vocal information into the verbal context, using brain structures dedicated to conflict resolution. Hearing Maybe followed by confident voice induces delayed inferential mechanisms, using the brain structure focusing on perception of other’s hidden intention. Thus resolving conflicting messages regarding confidence depends on what type of conflict is being resolved, with distinct brain regions playing different roles in processing depending on whether the confident signal is preceded or followed by the unconfident signal.

Females are better at confidence decoding?

We also found a sex difference in decoding feelings of knowing from the voice. Female listeners in our study had stronger sensitivity to vocal cues of confidence, rating confident-intending expressions as more confident and non-confident-intending expressions as less confident than male listeners. We believe that these sex differences are mediated by individual differences in the neural responses to vocal expressions or by differences in personality, such as trait anxiety and trait empathy. It appears that female listeners engaged in very early acoustic analysis and made delayed social inferences for complex messages. In contrast, male listeners detected the relevant social information in the voice and immediately changed their levels of attention. A statistical mediation analysis we performed also found that females tended to have higher levels of trait anxiety, and that this, in turn, influenced early stage neural processing.

Overall, our research project has found that people appear to be good at evaluating confidence in another person’s statements, and this evaluation involves the interaction of both vocal cues—such as speed and intonation of speech—and linguistic cues—such as preceding a statement with the probability phrase “I’m confident” or “perhaps.” These cues also appear to be processed rapidly at the neural level—in some cases being differentiated as rapidly as 1/10th of a second into a statement. However, the way that mixed signals of confidence generate brain activity point to different underlying integrative processes: some related to conflict resolution and others related to detecting concealed intentions. These reports have also raised intriguing new questions currently under investigation regarding how personality traits can be inferred from how “feelings of knowing” affect the voice. We hope this research project will continue to shed light on how we make inferences in mental states of speakers—a common process used to evaluate teachers, business negotiators, politicians, and many other influential figures.


Xiaoming Jiang is a postdoctoral researcher in the Neuropragmatics and Emotion Lab led by Dr. Marc Pell at McGill University. Jiang investigates how the brain makes social inference through vocal cues, more specifically on how inferential processes takes place in a cross-cultural communication setting.

Reference

Jiang, X. & Pell, D. M. (2016b). Feeling of another knowing: how “mixed messages” in speech are reconciled. Journal of Experimental Psychology: Human Perception and Performance. In Press.

Jiang, X. & Pell, D. M. (2016a). Neural responses towards a speaker’s feeling of (un)knowing. Neuropsychologia, 81, 79-93.

Jiang, X. & Pell, D. M. (2015). On how the brain decodes speaker’s confidence. Cortex, 66, 9-34. 

Jiang, X., & Pell, M. D. (2014). Encoding and decoding confidence information in speech. Proceedings of the 7th International Conference in Speech Prosody (Social and Linguistic Speech Prosody), 576–579.

 

Michael Inzlicht

Michael Inzlicht is a Research Excellence Faculty Scholar at the University of Toronto. Michael’s research sits at the boundaries of social psychology, cognitive science, and neuroscience. His most recent research focuses on the underlying nature of self-control, including how it is driven by motivation. Michael received his B.Sc. from McGill University, and his Ph.D. in Experimental Psychology from Brown University.

What led you to choose a career in personality and social psychology?

I did not always want to be a social psychologist. In fact, for much of my early life I had a different aspiration: I wanted to be a dentist! Thus, I took all the hard sciences in high school and college, and majored in anatomical sciences at McGill University (involving lots of classes in physiology, biology, microbiology, organic chemistry, etc).

I took a few classes in psychology because I thought they were fun and interesting, and after a while I had enough credits to complete a minor in psychology. In my senior year at McGill, I finally woke up and realized that aside from the money, dentistry had no real appeal to me. I didn’t know what to do, though I wondered if I could make a career out of my genuine interest in psychology.

I graduated and worked briefly in the real world and then traveled a bunch, after which I returned to McGill to take more psychology classes. During that time I took an introductory social psychology class taught by Dr. Donald Taylor, who was legendary at McGill, and everything changed for me. It was during that class that I discovered what I truly loved and started wondering how I could make a career out of social psychology.

Briefly summarize your current research, and any future research interests you plan to pursue. 

My interests are eclectic. In the earlier part of my career, I was primarily interested in the psychology of stigma, asking how stereotyping and prejudice affects the very groups and people who are targeted by stereotypes and prejudice. In the latter part of my career, I have become fascinated by the topic of self-control.

My work has challenged the notion that self-control is some sort of finite resource that becomes depleted after use, instead suggesting that self-control is a product of desires and priorities that sometimes shift over time. More recently, I have come to believe that self-control (defined as self-restraint or self-stopping) is mostly unimportant for meeting long term goals. In its place, my work indicates that motives and desires are far weightier.

On a meta-scientific level, I am deeply influenced by the scientific reform movement and have committed to improving my scientific hygiene so that my work is rigorous and replicable. It’s impossible for me to know what I’ll become interested in the future, but right now I am obsessed by the psychology of effort, and believe that a deeper understanding of effort can help us understand things like empathy, moralization, and blame.

Why did you join SPSP?

Well, it’s THE place for social psychology in the world. So the better question, in my mind, is why would you not join SPSP? If you’re a social and personality psychologist, there is no better society out there.

SPSP does so much: incredible convention, top-notch journal, awards of all sorts including early career researchers, small grants, representing our interests to policy makers and key people in government, and increasingly embracing scientific reform to improve our discipline.

What is your most memorable SPSP Annual Convention experience?

There are SO many good memories of the SPSP convention, only a handful of which are suitable for this newsletter! I’ve been to almost every conference, including the first one in Nashville. I guess my most memorable moment comes from that first-ever SPSP convention.

I was a graduate student at the time and was presenting some of my research as a poster when Dr. Brenda Major, one of my all-time social psychology heroes, came to talk to me about my research. I was so excited to meet Brenda face-to-face and to hear her say nice thing about my research. That small moment sustained me; it made my conference, if not my year!

How has being a member of SPSP helped to advance your career?

In countless ways, but probably the most obvious is that I’ve had many opportunities to speak at the SPSP convention, thus allowing my work to reach a broader audience.

Do you have any advice for individuals who wish to pursue a career in personality and social psychology?

I have two bits of advice, one generic, the other less so. First, follow your passion. Life in academia can be hard and can involve many annoyances and concessions. However, it can also be immensely rewarding; but only if you love what you study.

I have now met a number of early career researchers who are simply going through the motions, conducting the research their advisors want them to pursue or that they started in graduate school. These students (but sometimes assistant professors) don’t love what they study and they are unhappy as a result. So, study what you love and not what you think others will love. You and your work will benefit.

Second, pay attention to the conversation the field is now having about scientific reform and best-practices. Our field is changing a lot, with a lot more still to come. Some of what we now consider as solid and true (e.g., the stuff we read in our textbooks) will turn out to be neither.

You can improve the field by learning the right way to do research from the get-go. So pay attention to the discussion (often happening on social media), adopt practices that might be new for you or your lab, and embrace uncertainty.

Outside of psychology, how do you spend your free time?

I have young children, so I’m not sure I have lots of free time! That said, I truly enjoy drinking beer (the democratic libation) with friends on a patio. I also try to travel as often as I can with my wife and life-partner, often for long periods. Oh yeah, I like The Big Lebowski. A lot.