Cameras and software
couldhelp machines interpret human emotions
July 01 - 07, 2002
When you encounter strangers, you can probably
sense their emotional moods without a word being uttered. An eyes-down
frown probably means she's having a bad day. Bright, wide-open eyes
and upturned lips probably mean he's happy and feeling good.
These visual clues are usually easy for a human to
see and interpret. And soon, computers might be capable of making the
same judgment calls.
"We're teaching the computers to be more like
human beings," said Dave Schrader, an engineer with Teradata, a
division of automatic teller machine manufacturer NCR. In an attempt
to give consumers a better banking experience, Schrader is teaching
ATMs to discern emotions.
"The value of the tech is we're taking the ATM
one stage closer to behaving like a good, perceptive teller might so
that interactive dialogue can start beginning," Schrader said.
"The ATM can adapt itself to you instead of you adapting yourself
to the technology."
ATMs are the perfect match for this new technology
being developed by NCR and the University of Southern California's
Integrated Media Systems Center. The machines already have cameras,
powerful computers and fast Internet connections.
Engineers simply take an image from the camera, use
software to pinpoint facial features (such as the measurement from the
corner of your eye to your lip), and create a map of your probable
emotional state. Those points are then compared to Teradata's massive
catalog of emotions in an attempt to decipher your emotional state.
"Then," Schrader said, the computers
"direct that ATM to do something smart."
Forgot your glasses and can't see the numbers? The
computer can detect this and automatically increase the font size or
change the contrast of the display. Hate annoying advertisements?
Emotion-sensing ATMs could help ease your irritation.
"When you interact with an ATM it's all about
convenience," Schrader said. "It's also all about not having
a lot of clutter advertising coming your way or one-size-fits-all kind
of advertising. The bank can be much smarter and treat you
The machine can track which ads you frown at and
change those pitches. Maybe a cute, humorous ad pops up that causes
you to chuckle. The database remembers your reaction and you'll likely
see the advertisement again.
MAYBE ALL THOSE ANNOYING ATM FEES WILL GO DOWN.
"The real reason [for implementing the new
technology] is cost reduction," Schrader said. "If you can
interface with an ATM instead of standing in line for the bank teller,
it's good for you, it's faster, more convenient for you, and it's
cheaper for the bank, less labor cost."
BEYOND ATM COMMERCIALS
But Schrader said implementation in ATMs is just
the beginning for this emotion-mapping technology. It could be used at
information kiosks, displaying information about city events or
advertisements for a new restaurant or automobile. It could then track
people's emotional responses to the spots to see what type of person
appears interested and entertained by the spots.
"Technology is always in front of the actual
uses, but we're beginning to see a lot of potential uses of how facial
reactions occur, like in the medical area — maybe even in the
terrorism area," Schrader said.
How many times have you answered security questions
at an airport and wondered, "Why in the world would a terrorist
answer them truthfully?" Schrader said the emotion-scanning
technology could act like an instant lie detector, tracking whether
people become nervous or afraid as they answer the questions.
"Identifying people's reactions when they're
asked questions — their nonverbal clues that skilled detectives
might pick up on — we can train the computer to pick up on those,
too," Schrader said.
A THERAPIST'S FRIEND?
A more likely scenario is the technology's use by
psychologists and counselors.
Dr. Skip Rizzo, a clinical psychologist from the
University of Southern California, said he thinks the cameras and
software could be a tool for therapists.
"We still have a long way to go with this, but
we believe by tracking facial expressions it gives us added
information that a therapist can use to get a better insight into a
patient," Rizzo said.
"Humans are fallible," he said, "and
there are times when I might not be — paying attention to the facial
expressions, and I might miss something, so the computer can
supplement the perceptual skills of the therapist."
This could be especially critical to detect a
phenomenon known to psychologists as "suicide face," a blank
expression that has been documented in patients the day before they
kill themselves. "A person may have a certain characteristic
facial expression that indicates they have a sense of hopelessness
besides depression that will put them at risk for suicide, so perhaps
those are the kinds of things that we can start to look at,"
With such powerful information being available in
an instant, there are obviously privacy concerns.
"Businesses would need to keep this
information private just like they would with any other kinds of
records," Schrader said.
Sonia Arrison, of the Center for Freedom and
Technology at the Pacific Research Institute, said she wants more
restrictions. She wants to know whether people can remain anonymous if
the technology is employed.
"If they know who I am, that gives me a
different perspective — what are they really going to do with this
data?" Arrison said. "The other [concern] is, is there
consent, and do I know about it?"
She also pointed out that people might react
differently if they knew they were being scanned, undermining the
Arrison also cautioned that while the technology
could be convenient for consumers, there could be a backlash.
"A lot of folks don't like to feel like
they're being watched," Arrison said. "It's not so much a
privacy issue as it is a dignity issue or a comfort issue. Companies
have to be aware of this."
Schrader suggests that banks and other institutions
using the scanning technology could put up signs or use a red light to
indicate the system is in use. Ultimately, he said, that will be up to
the managers of the machines to decide. He is quick to point out that
when you go in to see a bank teller, or to buy a soda or a T-shirt,
the teller or the store clerk also performs emotional scanning on you,
looking at whether you're happy or displeased with your banking or
"I think the privacy expectations would be
exactly equivalent for the same reasons the tellers don't go running
around talking about the people's emotions," Schrader said.
GETTING IT RIGHT
The other concern for the scanning technology is
how accurate it can be — especially if the government relies on it
to boost airport security or therapists trust it to track patients'
"One of the interesting research problems I
gave the University of Southern California is whether they can
distinguish between confusion and frustration," Schrader said.
"Your facial reactions there are fairly close."
ANOTHER ISSUE IS DETECTING DEPRESSION.
"It's not hard to build a database with people
who are happy, frustrated, or confused, but it's hard to make a
database with people who are genuinely sad," Schrader said.
"That's maybe the weakest area of the database."
Schrader recalled one experiment in which
researchers from the USC psychology department showed people pictures
of other people being killed, and the experiment participants actually
"They were jaded by watching too many Hollywood movies,"
Schrader said. "Sadness is an emotion that will be tough for us
to come up with, but I don't expect it to frequently come up in the
day in the life of an ATM."