Tag Archives: reading facial expressions

Taking the Reading the Mind in the Eyes test

December 18, 2012 musingsofanaspie 40 Comments

The “Reading the Mind in the Eyes” is meant to test Theory of Mind (ToM) or the ability to recognize and understand another person’s mental state. It’s supposed to be a more advanced test than “Fear, Anger, Joy“, which tests simple emotion recognition.

The original 1997 version of Reading the Mind in the Eyes consisted of a set of 25 photos showing the area around the eyes and a choice of two possible mental states for each photo.

However, the limited number of items on the test combined with the choice of only two responses resulted in a test that wasn’t very reliable. Parents of autistic children were scoring as far below the controls as the AS/HFA group was. Additionally, the original version of the test included some expressions for basic emotions (happiness, sadness) which were considered too easily recognizable and not a true test of ToM.

The revised version of Reading the Mind in the Eyes contains 36 items with 4 answer choices for each item, increasing the possible range of scores along with the difficulty level. It also contains a balance of male and female photos, a choice between more closely related mental states (i.e. not a choice between opposites like sympathetic/unsympathetic), and is composed entirely of photos representing complex mental states.

In the original study to validate the test, the AS/HFA group scored a mean of 21.9 while the control had a mean of 26.2. However, the AS/HFA group had only 15 participants versus 239 controls. A sample size of 15 is small, especially for study in which participants only have to complete two questionnaires (the AQ and Reading the Mind the Mind in the Eyes. I’m curious why the researchers didn’t make an effort to obtain a larger AS/HFA sample when they had the resources to administer the test to so many controls.

Pros and Cons of the Reading the Mind in the Eyes test

Pros:

Tests recognition of complex mental states
Balanced presentation of male and female expressions
Offers subtly similar answer options to increase difficulty level
Self-scoring
Provides a list of items that were answered incorrectly (with the correct answers)

Cons:

Validation study relied on a small sample size
Sets up artificial constraints not present in real life (limited choice of options, time to study “frozen” expressions)
Allows for unlimited time to answer each item

Taking the Test

You can take the test here. It’s all on one page. Just look at each set of eyes and then choose which of the four options best describes the state of mind that the pair of eyes conveys. Ideally, you should make your choice as quickly as possible.

It took me a little over 5 minutes to complete the test. I feel like I spent too much time on a few of the photos. For an idea of how unintuitive my process is when I take this kind of test, at one point I found myself unable to decide if a particular expression was content or defiant. These are very different mental states, but I ended up guessing (correctly!) because I couldn’t conclusively pick one over the other.

Once you’ve selected an answer for all 36 items, click the “get score” button and your score will be displayed at the top of the page.

Scoring the Test

Your score is a measure of how many out of the 36 items you answer correctly. You’ll also get a list of which answers you missed and a short summary of where your score fits in the distribution (below average, average or above average).

Here is my scoring information:

Your score: 31

A typical score is in the range 22-30. If you scored over 30, you are very accurate at decoding a person’s facial expressions around their eyes. A score under 22 indicates you find this quite difficult.

The correct answers for the ones you missed are: [I added in my answers in brackets so you can laugh at how wildly off some of them are]

17: doubtful [I chose affectionate – this could be a serious gaffe in a social situation!]
18: decisive [I chose bored]
19: tentative [I chose grateful]
28: interested [I chose affectionate – not that far off]
35: nervous [I chose contemplative]

Like the “Fear, Anger, Joy” test, I scored slightly above average. I’m starting to question how much these tests say about a person’s ability to read facial expressions in “live” social contexts.

When I’m taking a test like this, there are two artificial constraints:

I’m forcing myself to intensely focus on and study each facial expression.
I’m given limited options to choose from.

Based on the availability of 4 choices, random guessing alone would result in, on average, 13 correct answers. If you look at the options for each expression, at least one and often two are obviously incorrect (to me, and that may just be me). One of my primary test taking strategies is process of elimination and my approach to this test was no different. If I can eliminate one or two options, my odds of guessing correctly go up significantly.

The artificial nature of the test seems to reduce its value in identifying problems with ToM. When I’m interacting with another person, I’m usually too preoccupied with trying to follow the conversation to spend much time “studying” the other person’s constantly changing expressions.

Often when I’m concentrating on a conversation, I’ll look away from the other person’s face because I find it easier to process information that way. You can’t gather a lot of facial expression data when you’re staring out the window. And, most importantly, there are no prompts. The other person’s expression could be saying literally anything and I have no helpful cue words to narrow that down for me.

Can you guess what these pair of eyes are saying without any cue words to help you? Give it a shot in the comments if you like. — Can you guess what this pair of eyes is saying without any cue words to help you? Give it a shot in the comments if you like.

Then there is the fact that recognizing an expression is one thing; attributing causation is another thing entirely. Facial expressions are supposed to provide the clues that allow us to understand what another person is experiencing (the content of their mental state). Recognizing an expression of anticipation is the first step; deducing what the other person is anticipating should logically follow. Together these make up the concept of Theory of Mind.

To say that the Reading the Mind in the Eyes test is a measure of Theory of Mind is only partially true, especially for those of us on the spectrum. The second step of the process–understanding the content of the other person’s mental state–is where I often go wrong.

The Bottom Line

This is an interesting test of static facial expression reading. It’s value as a test of Theory of Mind is less certain.

Asperger's Tests

Taking the “Fear, Anger and Joy” Test

December 4, 2012 musingsofanaspie 33 Comments

This week for Take-a-Test Tuesday, I took the “Fear, Anger, Joy” test. It’s not an Asperger’s test, but it does measure something that aspies often struggle with: reading emotions through facial expressions. Also, since the last two tests have been question-oriented, I thought a visually-oriented test might be a nice break.

testmybrain.org is a citizen science project run by Harvard University. The website is used to conduct large-scale behavioral research studies with the goal of developing neurocognitive tests. By taking one (or more) of the tests on the website, you contribute to ongoing research and you also get a nice little write-up about your results, what they mean and why the research is being done.

I like the interactivity of the tests and the novel approach that Test My Brain takes. For example, the test that I’m taking today, “Fear, Anger and Joy,” is a test of the ability to identify emotions in facial expressions. Unlike some of the other facial expression recognition tests out there that allow you to study an expression for an unlimited time before identifying it, this test only allows you to see the facial expressions for a couple of seconds. I feel like this is closer to what happens in social situations, where facial expressions are fleeting.

Pros and Cons of Fear, Anger, Joy

Pros:

Automated and self-scoring
Better simulates social situations by limiting viewing duration for each set of faces
Detailed explanation of scores
Results include average scores and percentile ranking so you know how you measure against others who’ve taken the test
Explanation of why the research is being done with a link to more info
Taking the test contributes to ongoing research

Cons:

Test is part of active research so no conclusive results
No direct link to AS/autism/autistic traits
Forced choice between 2 options equals a 50% chance of being correct when guessing

Taking the Test

This test takes 15 to 20 minutes so be sure you have the time to complete it before you begin. To take the test, go to testmybrain.org and click on the Go! button next to the “Fear, Anger, Joy” test (you may have to scroll down to find it–the order of the tests changes).

You’ll be asked to make your browser window large. I kept mine the size I normally use and it worked fine. The next screen is a simplified informed consent form. You’ll be told what the research is being used for and asked to consent (agree). Once you do, you’re officially a research subject! The next screen collects some demographic information. It’s all anonymous–you won’t be asked any identifying information to get your results or anything annoying like that.

Once you’ve entered your demographic info, you’ll begin the test. There are a couple of practice questions before the first set of faces appears. You’ll see two faces side by side and be asked to identify which is happier, sadder or angrier. There are 56 sets of faces for each emotion. That sounds like a lot until you realize that each set is visible for only a couple of seconds.

After you’ve completed all three sets of faces, you’ll be asked if you encountered any technical problems or if you cheated in any way. Researchers think of everything. Once you confirm that your results aren’t tainted, you’ll be taken to the scoring page.

Scoring the Test

The aim of the test is to evaluate how well people can detect subtle differences in the level of emotion conveyed by an expression. I found the expressions presented at the beginning of each to be obviously different. One face was clearly much happier, sadder or angrier.

As each section progressed, the differences in the expressions became much less obvious and I found myself guessing more frequently. I also realized that scared and happy can look similar to me. The same is true for scared and angry.

Here are my scores (out of a possible 56 on each section):

Which face is angrier?: Your score was 45. The average score is 45.18.You scored higher than three out of every ten people who took this test.

My rankings on the anger section — I scored slightly below average on the anger section.

Which face is happier?: Your score was 54. The average score is 49.49.You scored higher than seven out of every ten people who took this test.

Nearly a perfect score on the happy faces!

Which face is more afraid?: Your score was 47. The average score is 43.18.You scored higher than six out of every ten people who took this test.

And a little bit above average on the afraid faces. — And a bit above average on the afraid faces.

I did better than I expected to. I’m not surprised that I score the highest on the happy faces because happiness is (for me) most clearly expressed around the mouth (smiling) and that’s where I most often focus my gaze.

Anger forced me to look at the area around the eyes more and I had to purposely do that. For fear, I quickly found a pattern that helped me: furrowed brows or foreheads. I spent the rest of the “afraid” section looking first (and sometimes only) at the foreheads of both people to see which looked more scrunched up. Maybe I should have confessed to cheating?

Apparently I have trouble taking in a facial expression as a unified whole. I seem to look at the various parts for clues and then try to sum them up.

I also found my perseverative nature kick in when I had to change from finding the angrier face to finding the happier face. On the first of the happy practice sets, I chose the angrier face because that’s what I’d been doing for the previous 58 trials. Oops.

The Bottom Line

This test was fun to take and I felt like I learned something about how I process facial expressions.

How did you do? Were you surprised by the results?

Communication

You Scare Me

October 10, 2012 musingsofanaspie 60 Comments

Last summer, my husband and I had some new friends over for lunch. They brought along their two young boys. Toward the end of the meal, the 5-year-old, who was sitting next to me, looked at me and said, “You scare me.”

This was pre-Asperger’s, so like everyone else at the table, I laughed it off as one of those inappropriate things that kids sometimes say.

Still, his comment stayed with me. I couldn’t figure out what I’d done to scare him. He was a friendly, talkative little boy. I’d showed him how to get my dog to do a couple of simple tricks and had given him some bits of hot dog to use as treats. I’d asked him about his swimming lessons and whether he wanted a dog of his own. I’d cooked him a cheeseburger so he wouldn’t have to eat the fancy grown-up food. He’d even chosen the seat next to me at lunch. I thought we were getting along great!

And then, out of nowhere, he told me that I was scary. I was more puzzled than offended, but there was something about his comment that really stuck in my head. Sometimes when I’d catch people staring at me in a restaurant or a store, his words would come back.

You scare me.

Why?

A Hard Truth

Months later, I was sorting through boxes of photos and it hit me. There it was–there I was–staring back at myself from photo after photo with the dreaded flat affect. Since they say a picture is worth a thousand words, here’s how a flat affect looks:

And to prove that wasn’t just me getting caught at a bad moment, here’s one I actually posed for:

These pictures are hard to share. I don’t like looking at them. In fact, I almost never like looking at photos of myself. If I don’t have a blank expression, I tend to look like I’m faking a smile or making an uncomfortable, when-is-this-going-to-be-over expression.

As I was sorting through twenty-plus years worth of photos before we moved, I found dozens or maybe hundreds of pictures of myself with some variation of a blank, checked-out expression. I don’t know why I hadn’t seen it before but there it was. Standing in front of the Christmas tree. Attending a wedding. On vacation.

One after another, I tossed them in the trash bag on the floor beside me, tired of looking at this woman who was starting to scare me.

Flat Affect

From my reading about Asperger’s I was aware of the difficulty aspies have in reading facial expressions, but it hadn’t occurred to me that I don’t project appropriate facial expressions–or sometimes any expression at all.

The technical term for this is flat affect, which means that a person displays reduced emotional expressiveness. It takes a five-year-old to put it in plain English though: you scare me.

Looking at those photos, hundreds of them in a row, for hours on end that afternoon, I finally answered the question why? Flat affect is unsettling to others–it makes me look bored, angry, sad or spaced out at inappropriate times.

To a five-year-old, who is probably relying more heavily on nonverbal than verbal communication to judge adults, my inappropriate or absent expressions were creating mixed messages. Though I was saying and doing “nice” things, the nonverbal expressions I was projecting weren’t the typical “kind, caring adult” cues he was expecting to go with my words and actions.

Maybe I Can Learn to Fake It?

The disconnect between my expressions and thoughts is frustrating. Not only do I have trouble verbalizing my emotions but my face keeps wandering off on its own and freelancing.

More than once I’ve had a professor pause during a lecture to ask me if I had a question. One day, curious about why this happened so often, I finally said, “No, why?”

“Because you’re frowning,” the professor replied.

Surprised at his reply, I blurted out, “I’m not frowning. This is my concentrating face.”

The rest of the class laughed, but the question was right up there with you scare me in how deeply it unsettled me.

Obviously I was projecting something different from what I was experiencing internally. There I was sitting in calculus class day after day, looking confused, but never asking any questions. This made my professor so uncomfortable that he stopped in the middle of his lecture to ask me what my problem was. I wonder if he even believed me when I told him I wasn’t confused.

I wonder how often people think I’m being deceitful because my verbal and nonverbal communication doesn’t match.

This is a problem that feels too pervasive to fix. I’m literally projecting an expression of some sort during my every waking moment. There’s no way I could–or would even want to–pay attention to what that expression is all the time.

There are also plenty of times when my expression does agree with my disposition, especially when I’m genuinely happy.

Here’s a photo taken around the same time as the above two shots, except in this one I was truly happy and look it:

Since that exchange with my calculus professor, I’ve occasionally tried projecting a specific expression. In class, if I noticed a professor glancing in my direction too often, I assumed that I was doing the confused face and tried put on my “interested but neutral” face. I also made sure to nod a lot, a reassuring sign to NTs.

It seemed to help–it at least reduced the number of concerned looks in my direction–but I’m not very motivated to do this on a regular basis. I’ve seen other aspies talk about how acting lessons or practicing in a mirror helped them overcome flat affect. I admire their commitment to doing this–it sounds like it would take a lot of time and practice to get right.

Then again, if I had a job that required a lot of contact with the public, I might have the motivation to put more effort into improving the type of nonverbal cues I project. Maybe somewhere down the road it will be something I’ll decide to try but for now, I’ll just go on scaring small children and bewildering acquaintances.

Communication

Eye Contact: The Conversation within the Conversation

September 26, 2012 musingsofanaspie 51 Comments

Making eye contact–or more precisely, not making eye contact–is a big issue for people with Asperger’s. Neurotypical people seem to be especially frustrated and confused by this aspect of aspie behavior.

Even the professionals, who can offer up all sorts of theories as to why we don’t make eye contact, don’t seem to get the fundamental issue. In one study, the author pointed out that people with AS don’t make use of expressive information in the facial region of the eyes, “even when it’s available.”

That last part made me laugh. That’s like giving a Russian/Japanese dictionary to a person that can’t read either language and asking them why they aren’t using it. Just like that dictionary, the part of the facial expression around the eyes is a foreign language to an aspie.

Reading Expressions: The Eyes Test

I’m moderately good at reading facial expressions. I get the biggies like happiness, sadness, surprise, anger, and confusion. More subtle expressions–those that rely most heavily on the use of the eyes and the area around the eyes–are much harder for me. For example, on this list of 30 facial expressions, I would not have correctly identified the ashamed, concerned and contempt faces.

It’s important to note that “reading the mind in the eyes” (as the professionals call it) involves more than just the eyeballs themselves. It requires understanding how the subtle changes in the muscles around the eyes convey emotion. This becomes more obvious when you take the creatively named “Eyes Test” which requires you to use just the portion of the face between the nose and forehead. If you’re curious, you can find and take the test here:

Instructions (scroll all the way to the bottom of this file for the record sheet and scoring key)
Eyes Test Part 1
Eyes Test Part 2

(You can also find an interactive version called Reading the Mind in the Eyes at Test My Brain but you need to enter some demographic information to take it.)

I did surprisingly well on the test and here’s why: I cheated.

I suspect that most NTs taking this test would look at an expression and spontaneously have an idea of what the expression was. This is probably why the instructions say to choose an answer as quickly as possible.

How did I cheat? Before even looking at the expressions, I found myself reading all four possible answers. Then I looked at the expression and began the process of elimination. Here’s one of the expressions from the test with the four possible answers:

eyes — A sample question from the Eyes Test

My process of elimination: I know right away that it’s not dominant or horrified. It looks more like friendly than guilty, so the answer is friendly. If I hadn’t been given four choices for each expression, there is no way I would have spontaneously provided some of the more subtle answers like insisting or uneasy.

Obviously, in a real-time social interaction, no one is going to have four possible emotional options floating around their head to help me guess at what they’re feeling. Like most aspies, I’ve learned to use other information to try to fill in the blanks when it comes to facial expressions. I often focus on a person’s mouth, which I find conveys emotions more explicitly to me than the eyes.

I can gather some additional data from a person’s voice, especially when I know the person well. When my daughter calls me, I can tell from the way she says “hi” what kind of mood she’s in and what the conversation will likely be about. Unfortunately this only works with people I know well.

Over time I build up a databank of voice qualities for a person. With enough conversations in the databank, I need only a few words to recognize the happy voice, the lying voice or the you’re-not-gonna-like-this voice.

Why Learning to Read the Eyes is Challenging for Aspies

If we can learn all of these work-arounds and hacks for reading emotion in others, why can’t adult aspies just learn to read the eyes?

There are programs to teach children how to do interpret facial expressions. But for adult aspies, it’s a bit more complicated. We’ve grown up and spent decades of our lives not making eye contact.

Are we unable to read the eyes because we don’t make eye contact and therefore don’t have a rich enough data bank to draw on?

Or . . .

Do we not make eye contact because we discovered early on that the eye region doesn’t hold any useful data for us and our limited data gathering abilities are better “spent” on areas like (in my case) the mouth or voice?

It sounds like a classic “chicken and egg” scenario, doesn’t it?

Add to that level of discomfort that many aspies feel when it comes to making eye contact and it’s easy to see why learning to use and read the eyes in social interaction can feel like such an overwhelming prospect.

Is Making Eye Contact Multi-Tasking?

More than once I’ve read the suggestion that autistic people don’t make eye contact because we have trouble doing two things at the same time. This strikes me as ridiculous. Is making eye contact and talking really the equivalent of doing two things at once? If that’s the case, what about walking and talking? Running and talking? Driving and talking? I can do all of these things with no trouble. Sometimes I even talk while walking and chewing gum. How’s that for multitasking?

Moving on . . .

When Eye Contact Becomes Too Much

The level of discomfort that people on the spectrum experience when it comes to eye contact varies from mild to unbearable. The discomfort also varies from situation to situation. I’m more comfortable making eye contact with people I know very well or hardly at all. People who I know somewhat (professors, fellow students, business acquaintances) are the ones who make me most uncomfortable when it comes to eye contact.

The discomfort goes beyond simply feeling weird. It’s a physical sensation. I physically feel like I can’t continue to look into the person’s eyes a second longer. If I do sustain eye contact beyond that point, I get a strong sensation of needing to flee the room.

The classic “fight or flight” symptoms suggest that my brain is perceiving sustained eye contact with certain people as a threat. Why is this? And why does it happen most strongly with people who I consider acquaintances but not strangers or intimates?

Perhaps it’s because I’m self-conscious about the uneven balance of social power in these situations? The other person is clearly gathering social data from my eyes but I’m not able to do the same. Ironically, given my small range of often inappropriate expressions, what they’re gathering is probably as useless as the fragments of nonverbal communication that I’m picking up from them.

Eye contact with strangers tends to be fleeting and meaningless. The exchange of information feels limited and safe. Eye contact with my husband or daughter, on the other hand, carries none of the social balance of power issues that I feel with acquaintances. I don’t need to worry about what they might be discovering about me or whether I’m missing some key nonverbal cue (because if I am, they’ll tell me).

Acquaintances, however, are still (or sometimes perpetually) in the “getting to know you” stage. We’re feeling each other out, gathering data that will determine the course of future interactions and cement our perceptions of each other.

For NTs, eye contact is a rich and layered language. It’s the conversation within the conversation. As aspies, we’re largely deaf to this language. It’s no wonder it makes us so uncomfortable when others try to “speak” to us with their eyes.