Human Lie Detector Paul Ekman Decodes The Faces Of Depression, Terrorism, And Joy

Since he experienced tragedy at age of 14, the real-life psychologist who inspired the show "Lie To Me" has searched for signs of hidden human emotion in faces. New applications based on his findings are getting attention from Apple, Pixar, Google, the Army, and others.

Expert humans or face-reading machines could have saved thousands of lives on 9/11 by detecting the emotional states of hijackers. They would have triggered detainments, says San Francisco-based psychologist Paul Ekman. But they weren't being used.

In the years after 9/11, Ekman, the expert who inspired the fib-hunting character played by Tim Roth in the Fox series Lie To Me, has worked with the Central Intelligence Agency (CIA), the Department of Defense (DOD), the Department of Homeland Security (DHS), and others to help develop both people and machines that read faces for emotions and help stop disastrous events on all levels. New applications are helping TSA agents to screen for potential terrorists at airports or training U.S. Army personnel to determine enemy combatant veracity or intent to kill. Ekman has helped train a whole series of people who were guards at Abu Ghraib, too, in how to extract information and truth without torture. "They used my [facial analysis] work, and it was very successful," Ekman tells Fast Company.

He's helped pioneer a field called facial emotional measurement, which shares some ties with face-recognition and neuromarketing. The CIA and Cars studio Pixar alike are users of facial emotional analysis services. Apple and Microsoft have been building their own capabilities while Google has just recruited a facial expressions expert from a company Ekman advises. But the field goes way beyond the mere branding or advertising uses and delves into a realm that would wow Philip K. Dick. Ekman has written a book, Emotional Awareness, with the Dalai Lama. Classrooms are putting his methodology to work. And others' applications are helping autistic patients be better understood or enabling people to become more attuned to their own emotions.

There are two ways to go about facial emotional measurement: a human way to analyze facial "microexpressions" and emotions; and a technological, automated method. In life or death matters, the human approach, which relies on specially trained facial observers to gauge perpetrator deception, is significantly more accurate than the automated method, according to Ekman. He is universally credited for developing the Facial Action Coding System (FACS), a comprehensive dictionary of facial expression measurements that has become the scientific underpinning for human observer and automated facial analysis around the world, across academic and commercial fields. And while he spends a lot of time these days training intelligence, law enforcement, and military officers on how to uncover a subject's deception in a split second (when there is no time for laborious video analysis or FACS catalog consultation), he also acknowledges the value in a "backup" system whereby computers automate the FACS work. Humans get tired, after all, and their levels of observation may vary.

Ekman, in fact, sits on the board of a nascent company, Machine Perception Technologies (MPT), which specializes in automating facial expressions analysis based on his FACS foundation. MPT was spun out of the DOD's innovation arm DARPA and DHS in the aftermath of 9/11. The CIA has been studying the efficacy of MPT's "Neural Network" method of automating facial microexpressions versus an alternative based on "Artificial Intelligence."

About half of MPT's work is in the "security" arena (including DHS Project Hostile Intent); the other half is in marketing, COO Stanley Kim tells Fast Company. MPT wants to advance "machine learning" and create smarter, more natural human-machine interactions and interfaces. Current MPT clients Kim would disclose include Procter & Gamble, Intel, and Sony (MPT built the Sony Smile Shutter).

Then there's Affectiva, which bills itself as an automated emotional measurement company. It was recently spun out of MIT Media Lab. Affectiva's methodology is based on FACS but updates it, links it to a variety of emotional states, and automates it with its proprietary algorithm. CTO Rana El Kaliouby tells Fast Company Affectiva's initial focus has been in health care. They have developed tools that help autistic patients communicate and consumers self-monitor their health status via webcams and skin and facial sensors. Recently Affectiva has been busy consulting to marketing and media clients on advertising and consumer engagement (for example, measuring the extent to which various ads and elements evoke facial joy, interest, humor, etc--as shown in the Affectiva dashboard below).

While critics of Ekman's own field assail its big brother intrusions, sinister mind-reading, phony junk science, low accuracy, or high expense, Kaliouby says his FACS encyclopedia on its own is incomplete, and that it does not correlate, explain, or predict the most common and relevant human emotions.

"Our system first detects the FACS units and then combines those codes in space and time to infer what emotion the person is communicating/expressing," she says. Affectiva tries to build algorithms that capture nuances of facial expressions (beyond the basic emotions) and focus on relevant emotions, such as enjoyment--or interest and confusion--which aren't part of Ekman's basic emotions but are critically important in applications such as advertising, an online tutoring system, or an online game that tracks whether users are engaged, bored, or confused, Kaliouby says.

Ekman, in turn, thinks the Affectiva methodology is less comprehensive and precise than his own or MPT's approach. "They take some shortcuts. I'm sure it works to some extent. But when the stakes are very high (in criminal or security matters) ... Over 90% accuracy (of FACS) is required. Seventy percent--which you can get published in Science--has no practical use." Unlike some of his counterparts in the field, Ekman will not apply his work to partisan politics or judicial cases (e.g. supplying witness testimony or advising on jury selection) and decries the ethics of doing so.

Affectiva is aiming to make neuroscientific and emotional measurement tools accessible and transparent to consumers (only as they opt in) and do good in the world with the broadest applications (beyond ad testing). Kaliouby is uncomfortable having the company associated with "neuromarketing" and says Affectiva will never focus on security and deception.

Ultimately Affectiva says it wants people to be more aware of their emotions and empowered to take control of their health, life, work, and fun. Cofounder Rosalind Picard says their cloud-based technology will be used to crowdsource and report facial reactions to stimuli, including political candidates and events, business presentations, movies, news stories, and, yes, ads.

Another maker of an automated emotional measurement system, London-based Realeyes, provides eye-tracking and facial emotional measurement analytics. Its CEO Mihkel Jaatma says they are less focused on the full body, biology, and health care than Affectiva. While Realeyes also does a lot of facial measurement work in marketing, the company is seeing growth in games and education. Jaatma tells Fast Company that Realeyes is now working with Kaplan Education, Fox International Channels, the University of Cambridge, and Hungarian Artificial Intelligence scientists on Project Picaro to develop video games that help teach Hungarian students aged 4-9 English based on a continual, optimizing process of analyzing and responding to their faces. Picaro's facially enabled learning products are being used in classrooms in Szeged, Hungary today.

Jaatma, like Affectiva's Kaliouby and MPT's Kim, says there's been a recent surge in inquiries from potential clients spanning the globe and a range of industries. "Facial expressions and emotional states will soon become an essential part of interfacing users with all sorts of software," Jaatma says. "Less of keyboards, less of using [a] mouse, more of users just being themselves. More devices and more applications will understand the face of the person and be able to adjust themselves according to how the user is feeling and what the next feelings are predicted to be. The expressions recognition technology will also be used beyond the face for the whole body." Full-body gesture tracking will enable more understanding of emotions and allow us to categorize behavior in places such as shopping malls, casinos, and transportation hubs for example, Jaatma says.

For Kim, "The next generation for us will be about the interface, you, and the machine in Minority Report asking, 'How are you feeling?' It's going to watch you and learn from you and vice versa."

Ekman thinks FACS is more advanced, useful, and accurate than other hot neuroscientific areas such as neuromarketing, which focuses on the brain; he says the study of the face continues to reveal more and more precise connections between known physiological and emotional states. His chief strategic objective is "preventing assassinations of political leaders and stopping terrorism in our country," he says. But not all of his work involves detecting liars and catching perpetrators. While he turns down marketing assignments almost daily, Ekman devotes a lot of time and energy to softer, emotionally "uplifting" and "fun" projects too. He continues to consult to Pixar Director Pete Docter on animated character expressions through his updated dictionary. (Toy Story was the first cinematic use of FACS). Ekman worked on Rise of the Planet of the Apes and Avatar. Disney and Industrial Light & Magic are clients. For the hit Fox TV series Lie To Me, which is based on him and his work, Ekman advised cast members on acting to facial expressions.

As different as their methods, services and views may be, all of the facial emotional measurement experts agree that new applications in the space will emerge that cannot be imagined now; several mentioned a tool to detect drunkenness, driver fatigue, disease, or depression.

For Ekman, one goal--"improving emotional life"--hasn't changed since 1948, when his mother committed suicide. Ekman was 14. Wishing he could have seen warning signs on her face before it was too late, he became driven by a determination to "help mental patients" and "save lives."

In the process, he's built a new science.

Follow Kevin Randall on Twitter: @KevinBRandall

---

Original Page: http://www.fastcompany.com/1800709/paul-ekman-facial-emotional-measurement-microexpressions?partner=rss

Inviato da iPad