Boosting Emotion Recognition in Context using Non-target Subject Information

Selvarajah Thuseethan, Sutharshan Rajasegarar, John Yearwood

Research output: Chapter in Book/Report/Conference proceedingConference Paper published in Proceedingspeer-review

Abstract

Recognizing emotions in context from an image has become an emerging topic in the recent past due to its high practical demand in various domains. Performance of the existing works is limited as they predominantly utilized the entire image and the primary human subject as major cues to recognize the emotions from images. However, in addition to the primary human subject, other human subjects in a scene also play a vital role in determining the image's overall emotional state. In this work, a novel visual feature type is introduced based on the features extracted from other human subjects in order to enhance emotion recognition performance. A novel deep learning based hybrid framework is also proposed to effectively integrate the proposed feature with the visual cues from the entire image and primary human subject. The extensive experiments carried out on a subset of the Emotic dataset reveal that the newly introduced visual feature type contributes to the overall emotional state of the scene. Furthermore, the proposed framework achieved 45.11% average accuracy for discrete emotion categories, which is a significant improvement in the emotion recognition performance compared to existing techniques that use emotion information from the entire image and primary human subject.

Original languageEnglish
Title of host publicationIJCNN 2021 - International Joint Conference on Neural Networks, Proceedings
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9780738133669
DOIs
Publication statusPublished - 18 Jul 2021
Externally publishedYes
Event2021 International Joint Conference on Neural Networks, IJCNN 2021 - Virtual, Shenzhen, China
Duration: 18 Jul 202122 Jul 2021

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2021-July

Conference

Conference2021 International Joint Conference on Neural Networks, IJCNN 2021
Country/TerritoryChina
CityVirtual, Shenzhen
Period18/07/2122/07/21

Fingerprint

Dive into the research topics of 'Boosting Emotion Recognition in Context using Non-target Subject Information'. Together they form a unique fingerprint.

Cite this