CHAMPAIGN, Ill. — Scientists used animated humanoid avatars to study how nonverbal cues influence people’s behavior. Reported in the Journal of Cognitive Neuroscience, the research offers insight into the brain mechanisms that drive social and economic decision-making.
The study revealed that participants were more willing to cooperate with animated avatars than with static figures representing their negotiation partners. It also found – somewhat surprisingly – that people were more consistently willing to accept unfair offers from unfriendly avatars than from friendly ones.
“This work is an extension of previous studies exploring how nonverbal cues influence people’s perceptions of one another,” said Matthew Moore, who led the research at the University of Illinois Urbana-Champaign with psychology professors Florin Dolcos and Sanda Dolcos. The new research was conducted at the U. of I.’s Beckman Institute for Advanced Science and Technology, where Moore was a postdoctoral fellow.
“Nonverbal interactions represent a huge part of human communication,” Sanda Dolcos said. “We might not be aware of this, but much of the information that we take in is through these nonverbal channels.”
Previous studies often used still photos or other static representations of people engaged in social interactions to study how people form opinions or make decisions, Florin Dolcos said.
“By animating the avatars, we’re capturing interactions that are much closer to what happens in real-life situations,” he said.
To understand how social impressions guide decision-making, the researchers asked participants to play a modified “ultimatum game.”
“In the typical game, a ‘proposer’ player offers to split $10 with you,” Moore said. “If you accept the proposal, then you each get a portion of that $10 as it was proposed. But if you reject the offer, then neither of you get anything.”
The goal of the game is to collect as much money as possible, Moore said. But people are often influenced by other people’s behavior, and don’t always choose to accept unfair offers, even though it would maximize their own gains in the game.
For the new study, participants watched a “responder” avatar engaging with an animated “proposer” avatar. Sometimes the proposer behaved in a friendly manner, smiling, moving closer and grasping the responder’s shoulder. In other trials, the proposer appeared unfriendly, frowning and moving away, with arms folded across her chest. And in control trials, the proposer was represented by an immobile posterboard, with a stance that was neither friendly nor unfriendly.
“Remember, these are not the participants themselves engaged in these interactions,” Florin Dolcos said. “We designed these experiments in a way that will facilitate the participants identifying with the responder avatars.”
The scientists used EEG to track brain activity while participants watched the avatar interactions, then looked for any correspondence between participants’ brain activity and their subsequent acceptance – or rejection – of an avatar’s proposal.
They found that watching the animated avatars interact increased the likelihood that an observer would agree to a proposed split of the $10 – even if it meant accepting an unfair offer. Interactions with static avatars led to more rejections of unfair proposals.
The researchers saw that participants were slightly more consistent in accepting an unfair offer when the proposer avatars behaved in an unfriendly manner, suggesting that participants were more comfortable when the avatars’ social behavior corresponded more closely with what they were proposing.
The team also observed that the pattern of a participant’s brain activity often predicted whether they would accept or reject an offer.
“The EEG patterns, known as event-related spectral perturbations, track changes in spectral power at each frequency in ongoing electrical activity across the brain, and can capture brain responses within hundreds of milliseconds after a person observes a stimulus,” Moore said. “Brain responses associated with processing movement and cognitive control were activated most in trials where – shortly after observing an interaction – participants chose to accept an unfair offer.”
The research is an important step toward understanding how different brain responses contribute to social perception and decision-making, Moore said.
“If we better understand the mechanisms involved, then we can better understand things like how to intervene,” he said. “So, for example, if we have a goal of increasing cooperation or helping people make adaptive decisions, then we have clearer targets for our interventions.”
Moore now is an associated health fellow at the Palo Alto Veterans Affairs Medical Center in Palo Alto, California.