Emotion Intensity Estimation from Video Frames using Deep Hybrid Convolutional Neural Networks

Selvarajah Thuseethan, Sutharshan Rajasegarar, John Yearwood

Research output: Chapter in Book/Report/Conference proceedingConference Paper published in Proceedingspeer-review

10 Downloads (Pure)


Detecting emotional states of human from videos is essential in order to automate the process of profiling human behaviour, which has applications in a variety of domains, such as social, medical and behavioural science. Considerable research has been carried out for binary classification of emotions using facial expressions. However, a challenge exists to automate the feature extraction process to recognise the various intensities or levels of emotions. The intensity information of emotions is essential for tasks such as sentiment analysis. In this work, we propose a metric-based intensity estimation mechanism for primary emotions, and a deep hybrid convolutional neural network-based approach to recognise the defined intensities of the primary emotions from spontaneous and posed sequences. Further, we extend the intensity estimation approach to detect the basic emotions. The frame level facial action coding system annotations and the intensities of action units associated with each primary emotion are considered for deriving the various intensity levels of emotions. The evaluation on benchmark datasets demonstrates that our proposed approach is capable of correctly classifying the various intensity levels of emotions as well as detecting them.

Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9781728119854
Publication statusPublished - Jul 2019
Externally publishedYes
Event2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
Duration: 14 Jul 201919 Jul 2019

Publication series

NameProceedings of the International Joint Conference on Neural Networks


Conference2019 International Joint Conference on Neural Networks, IJCNN 2019

Cite this