Professor of psychology Paula Niedenthal would be among the first to tell you that a smile is never just a smile.
While many of us assume that when a person smiles, they’re simply signaling happiness and positive emotions, scientists like Niedenthal, who’ve studied the nuances of human smiles for decades, know it’s far more complicated than that.
Over the years, Niedenthal’s lab has identified three types of smiles: Reward smiles (the kind that typically signal positive emotions), smiles of affiliation (the polite kind that signal a social situation is safe) and smiles of dominance (think smug smiles and smirks). Those smile designations help us understand and navigate all kinds of social situations—except, of course, when all the signals are hidden behind a face mask, as they have been for much of the last two-plus years.
During that time, Niedenthal’s lab has been studying just how much social signaling is lost or garbled when a smile is masked. We recently talked to her about what she and her colleagues have learned.
How did you pursue this question of lost social signaling due to masks?
PN: We needed to recognize that different facial expressions have different amounts of their signal on the top part of the face and on the bottom part. Surprise and anger involve a lot of the upper part of the face—when you're surprised, your eyes are open and your eyebrows are high and there's a lot going on there. If you're angry, there's a lot happening on the forehead.
With a smile, a lot is going on in the bottom part of the face. We associate crow's feet with big smiles because when you really lift your cheeks you make crow's feet, but a whole lot of the symmetry of the smile, which is an important part of the smile, is underneath the mask. So, if you think about where the real action is from the face during the expression of a particular emotion or signal, social signal expressions vary in whether the mask is preventing a lot or a little of the signal getting through.
Our first studies suggested that masks impair the intensity that you get from facial expression, and particularly the ones where a lot of the information is on the bottom part of the face. We studied surprise, anger, disgust, and the reward smile in some of our first studies, and we found that disgust and smiling were most impaired using a mask and that surprise and anger were well conveyed.
What we also saw with those four expressions is that people saw other emotions in the face that weren't there, because the face was partially covered. If it was masked, people saw more anger or even more surprise, because there was so much more ambiguity. One of the things that's important is that a mask not only prevents you from seeing the signal, but it allows you to add signals that you're expecting. Maybe you think the person's angry, and they're disgusted by something, or you think that they're angry, and they're actually surprised. We can have distortions based on our own expectations or biases if the face is masked.
Did you see any specific examples where that was the case?
Service people mentioned to me informally throughout the pandemic that people didn't know when they were joking. People didn't know when they were being warm, and that they had to do a lot of things to convey positivity to customers. One of the ways that they did it was by crinkling their eyes on purpose, so it gave the appearance of lifting their cheeks. People also have used their voices to modulate and show more happiness, for example, because their smiles are covered. One of the things we will be studying in the future is the use of other emotion-conveying signals when you're wearing a mask, and how much they can or cannot compensate for the loss of visual information.
How did the mask affect our perception of the three types of smiles?
The smile was clearly pretty messed up because people can't distinguish between the three smiles with a mask on. They saw less dominance and less affiliation, and they saw less of the signal when a mask was involved. These are the kinds of interactions that can go badly wrong with masks because they involve really very different signals.
How can we compensate for that deficit?
I think we might alter what we're doing when we're wearing a mask to become more communicative on the non-masked part of the face. I also think people will naturally accommodate with their voices. I have noticed that people are doing thumbs up and thumbs down gestures and stuff like that even more than before.
I worry a lot about eye contact and facial sourcing. We're used to looking at people for information. We look at people's faces for identity information, but also emotion and intentions. I'm afraid that we're just not going to do it as much, which I think is a loss of social information in our life.
What about relying on context? Can we expect any help there?
Some of the studies that we're doing right now look at the use of context when a facial expression is masked. For example, imagine that a smile is masked, and you can kind of tell the person is smiling. But you're told that the person just got an F on their exam. The question is, how much value is there in knowing that the F was just acquired? How much is that going to influence the interpretation of the facial expression? We now have data suggesting that when a facial expression is masked, the context, whatever it is, influences the interpretation of the face more than if it was not masked.
What happens when—or if—we’re finally able to put the masks away?
I think we all wonder if that is what is going to happen post-pandemic: Are we going to have lost sort of certain kinds of habits and are we going to retake them? Faces are enormously complex. They convey tons of information and blocking information from the face, in general, is not a good thing.