Want to understand people's emotions?

Tone of voice—not facial expressions—may be the best way to figure out what someone is feeling.

Speedy internet connections and cheap video calling have made face-to-face interaction easier than ever. But, a new study suggests that audio-only conversations may offer the clearest communication.

Body language and facial expressions have been extensively studied for the emotions they convey in conversations. But that’s precisely why they can be more deceptive, says Michael Kraus, assistant professor of organizational behavior at the Yale University School of Management.

“Misleading people through vocal expressions is more unlikely because controlling vocals is much harder to do.”

“There’s now a lot of discussion about how to look more confident, or how to hide certain less desirable emotion states by using non-verbal communication.

“There is a chance that people might mislead listeners with their nonverbal communication. Misleading people through vocal expressions is more unlikely because controlling vocals is much harder to do.”

Previous studies showed that people are better at reading emotions when presented both audio and facial expressions than when they’re asked to observe facial expressions alone. But how voice-only communication ranked was unclear.

For the study in American Psychologist, researchers recruited participants online and presented them with short videos of a group of friends talking and teasing each other over a nickname. Participants were given one of three versions: one group watched and listened to the video, a second only heard the interaction, and a third group only saw the video but did not hear the voices.

They were then asked to estimate what emotions they thought the friends were experiencing, by rating feelings such as amusement, embarrassment, or happiness on a scale of 0 to 8. People who only heard the interaction—but did not watch the video—made more accurate estimates of what the friends were feeling.

In a subsequent experiment, researchers recruited undergraduate students to come to the lab and chat with each other about their preferences for movies or TV shows, and what food and drinks they liked. The students also had similarly themed conversations in a darkened room. Then, they were asked to rate their own and their partners’ emotions during both exchanges. Participants who couldn’t see each other in the darkened room fared better at reading their partners’ emotions.

Finally, the researchers presented online participants with a digital voice reciting the friends’ teasing interaction from the prior study. If people were gauging emotional content based on the kinds of words being used, they would glean the same information with the digital voice.

But the artificial voice was the worst.

“The difference between emotional information in voice-only communication by a computer versus a human voice was the largest across all studies,” Kraus says. “It’s really how you speak—not just what you say—that matters for conveying emotion.”

One reason the voice is so effective at conveying emotion may be that speakers are less likely to be able to alter their tone to disguise their feelings. Another possible explanation stems from our cognitive capabilities. When communicating across multiple modes, a listener must focus on many kinds of information at once: facial expressions, words, body language, and the speaker’s tone.

“It’s difficult because you might be switching attention across those channels in order to perceive emotion,” Kraus says. “Whereas if you focus on any one that has the necessary information you’d be most accurate. Our research points to the voice as the most viable channel for emotion perception.”

The results underscore the importance of listening, a skill that’s going to be increasingly important as workplaces grow more global and more diverse. For managers, listening effectively can help them understand when an employee is unhappy or anticipate the needs of a business partner sooner.

“There’s an opportunity here to boost your listening skills to work more effectively across cultures and demographic characteristics,” Kraus says. “Understanding other people’s intentions is foundational to success in the global and diverse business environment that characterizes both the present and the future.”