UMD     This Site

Emotions Don't Lie: A Deepfake Detection Method using Audio-Visual Affective Cues

Emotions Don't Lie: A Deepfake Detection Method using Audio-Visual Affective Cues


It has been said that “seeing is believing,” but in the age of social media, viral videos, and artificial intelligence (AI) technology, can we truly believe what we see on the internet? Computer science researchers at the University of Maryland have invented a “Deepfake Detection Tool” to help answer that question.

Advances in computer vision and learning have enabled the creation of sophisticated and convincing forgeries of images and videos known as deepfakes. These falsified media are often used maliciously to spread misinformation or commit acts of fraud and cybercrime. A November 2020 report by Sensity, a software development company, estimated that approximately 85,000 harmful deepfake videos were detected, and that the number was expected to double every six months. 

Notable examples of deepfakes include a public service announcement from Barack Obama, co-created by Jordan Peele and Buzzfeed, to prove that you cannot trust everything you consume online, as well as a forged Instagram video featuring Meta CEO Mark Zuckerberg “admitting” that Facebook’s true intent is to manipulate users. Deepfakes pose a significant threat to politics, with the potential to manipulate elections, alter political narratives, weaken the public’s trust in a country’s leadership, and increase hatred among various social groups. Deepfakes can also be used to generate non-consensual pornography.

Paul Chrisman, Iribe Professor of Computer Science and Electrical and Computer Engineering, and Distinguished University Professor Dinesh Manocha, along with Ph.D. students Trisha Mittal,Uttaran Bhattacharya and Rohan Chandra, and Research Assistant Professor Aniket Bera, have invented a method that can detect such deepfake media, and in turn, help prevent the spread of misinformation and fraud. The researchers’ deepfake detection tool analyzes affective cues, such as eye dilation, raised eyebrows, speaking volume, pace, pitch and tone. This method is unlike its predecessors because it incorporates inputs from both audio and video, rather than a single input. Together, these cues provide complementary information to develop stronger inferences about the video in question. The relationship between affective cues from the audio and cues from the same video are analyzed to see if there is a strong correlation, which is used to determine if a video is “real” or “fake.”

The Deepfake Detection Tool was recognized as a finalist for an Invention of the Year Award in the Information Sciences category. The awards were held on May 3, 2022, at Innovate Maryland, a campus-wide celebration of innovation and partnerships at the University of Maryland.

More information about this invention can be found at:

The researchers’ work was supported in part by the Army Research Office.

Story reprinted from tthe University of Maryland Division of Research.

Related Articles:
Improving Fairness and Trust in AI Used for College Admissions and Language Translation
Qu Wins NIST Grant
Alumnus Forte Accepts Position at University of Florida
Six Clark School Faculty Receive 2024 DURIP Awards
Bio-Nose Technology: Conferring a Sense of Smell
Maryland Engineers Awarded Grants to Address Humanity's Grand Challenges
Big Ten Network spotlights Maryland Engineering
UMD Autonomous Navigation Research Featured in Tech Explore
Manocha Receives 2022 Verisk AI Faculty Research Award
Gemstone Honors Program Marks 25th Anniversary

May 12, 2022

«Previous Story  



Current Headlines

Ghodssi Honored With Gaede-Langmuir Award

Milchberg and Wu named Distinguished University Professors

New features on ingestible capsule will deliver targeted drugs to better treat IBD, Crohn’s disease

Forty years of MEMS research at the Hilton Head Workshop

Baturalp Buyukates (ECE Ph.D. ’21) Honored by IEEE ComSoc

ECE/ISR Ph.D. Students Win Hardware Demo Competitions in HOST Symposium  

Two Papers on  Gut-Microbiome-Brain-Axis (GMBA) Published

University of Maryland Has Strong Presence at ICRA 2024

Khaligh Honored With Linda Clement Outstanding Advisor Award

UMD Launches Institute Focused on Ethical AI Development

Back to top  
Home Clark School Home UMD Home