Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN

Marc Dominic Enriquez; Crisron Rudolf Lucas; Angelina Aquino

doi:10.1109/ISSC59246.2023.10162085

Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN

Marc Dominic Enriquez, Crisron Rudolf Lucas, Angelina Aquino

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper published in Proceedings › peer-review

1 Citation (Scopus)

Abstract

Speech Emotion Recognition (SER) focuses on understanding the human emotion in a given speech utterance using its acoustic and/or linguistic features. This paper presents a comparison between two speech representation inputs for SER: spectrograms and scalograms. Speech signals from four databases (Emo-DB, RAVDESS, SAVEE, and a mix of all three) were converted into each type of representation and were used to train variations of a convolutional neural network (CNN) VGG16 Model-3. Results show that the scalogram-based models have a higher mean f1-score compared to the spectrogram-based models; however, further analysis indicate that the difference is statistically insignificant at a 95% confidence level. In conclusion, spectrograms and scalograms have statistically the same performance on the systems presented.

Original language	English
Title of host publication	2023 34th Irish Signals and Systems Conference, ISSC 2023
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	1-6
Number of pages	6
ISBN (Electronic)	9798350340570
DOIs	https://doi.org/10.1109/ISSC59246.2023.10162085
Publication status	Published - 2023
Event	34th Irish Signals and Systems Conference, ISSC 2023 - Dublin, Ireland Duration: 13 Jun 2023 → 14 Jun 2023

Publication series

Name	2023 34th Irish Signals and Systems Conference, ISSC 2023

Conference

Conference	34th Irish Signals and Systems Conference, ISSC 2023
Country/Territory	Ireland
City	Dublin
Period	13/06/23 → 14/06/23

Bibliographical note

Funding Information:
This research was conducted with the financial support of Science Foundation Ireland under Grant Agreement No. 13/RC/2106 P2 at the ADAPT SFI Research Centre at University College Dublin. ADAPT, the SFI Research Centre for AI-Driven Digital Content Technology, is funded by Science Foundation Ireland through the SFI Research Centres Programme. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

Publisher Copyright:
© 2023 IEEE.

Access to Document

10.1109/ISSC59246.2023.10162085

Cite this

Enriquez, M. D., Lucas, C. R., & Aquino, A. (2023). Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN. In 2023 34th Irish Signals and Systems Conference, ISSC 2023 (pp. 1-6). (2023 34th Irish Signals and Systems Conference, ISSC 2023). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ISSC59246.2023.10162085

@inproceedings{6a276d5187774c22a01f2f9233307438,

title = "Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN",

abstract = "Speech Emotion Recognition (SER) focuses on understanding the human emotion in a given speech utterance using its acoustic and/or linguistic features. This paper presents a comparison between two speech representation inputs for SER: spectrograms and scalograms. Speech signals from four databases (Emo-DB, RAVDESS, SAVEE, and a mix of all three) were converted into each type of representation and were used to train variations of a convolutional neural network (CNN) VGG16 Model-3. Results show that the scalogram-based models have a higher mean f1-score compared to the spectrogram-based models; however, further analysis indicate that the difference is statistically insignificant at a 95% confidence level. In conclusion, spectrograms and scalograms have statistically the same performance on the systems presented.",

keywords = "CNN, Fourier Transform, Scalogram, SER, Spectrogram, Wavelet Transform",

author = "Enriquez, {Marc Dominic} and Lucas, {Crisron Rudolf} and Angelina Aquino",

note = "Funding Information: This research was conducted with the financial support of Science Foundation Ireland under Grant Agreement No. 13/RC/2106 P2 at the ADAPT SFI Research Centre at University College Dublin. ADAPT, the SFI Research Centre for AI-Driven Digital Content Technology, is funded by Science Foundation Ireland through the SFI Research Centres Programme. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. Publisher Copyright: {\textcopyright} 2023 IEEE.; 34th Irish Signals and Systems Conference, ISSC 2023 ; Conference date: 13-06-2023 Through 14-06-2023",

year = "2023",

doi = "10.1109/ISSC59246.2023.10162085",

language = "English",

series = "2023 34th Irish Signals and Systems Conference, ISSC 2023",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "1--6",

booktitle = "2023 34th Irish Signals and Systems Conference, ISSC 2023",

address = "United States",

}

Enriquez, MD, Lucas, CR & Aquino, A 2023, Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN. in 2023 34th Irish Signals and Systems Conference, ISSC 2023. 2023 34th Irish Signals and Systems Conference, ISSC 2023, IEEE, Institute of Electrical and Electronics Engineers, pp. 1-6, 34th Irish Signals and Systems Conference, ISSC 2023, Dublin, Ireland, 13/06/23. https://doi.org/10.1109/ISSC59246.2023.10162085

Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN. / Enriquez, Marc Dominic; Lucas, Crisron Rudolf; Aquino, Angelina.
2023 34th Irish Signals and Systems Conference, ISSC 2023. IEEE, Institute of Electrical and Electronics Engineers, 2023. p. 1-6 (2023 34th Irish Signals and Systems Conference, ISSC 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper published in Proceedings › peer-review

TY - GEN

T1 - Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN

AU - Enriquez, Marc Dominic

AU - Lucas, Crisron Rudolf

AU - Aquino, Angelina

N1 - Funding Information: This research was conducted with the financial support of Science Foundation Ireland under Grant Agreement No. 13/RC/2106 P2 at the ADAPT SFI Research Centre at University College Dublin. ADAPT, the SFI Research Centre for AI-Driven Digital Content Technology, is funded by Science Foundation Ireland through the SFI Research Centres Programme. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. Publisher Copyright: © 2023 IEEE.

PY - 2023

Y1 - 2023

N2 - Speech Emotion Recognition (SER) focuses on understanding the human emotion in a given speech utterance using its acoustic and/or linguistic features. This paper presents a comparison between two speech representation inputs for SER: spectrograms and scalograms. Speech signals from four databases (Emo-DB, RAVDESS, SAVEE, and a mix of all three) were converted into each type of representation and were used to train variations of a convolutional neural network (CNN) VGG16 Model-3. Results show that the scalogram-based models have a higher mean f1-score compared to the spectrogram-based models; however, further analysis indicate that the difference is statistically insignificant at a 95% confidence level. In conclusion, spectrograms and scalograms have statistically the same performance on the systems presented.

AB - Speech Emotion Recognition (SER) focuses on understanding the human emotion in a given speech utterance using its acoustic and/or linguistic features. This paper presents a comparison between two speech representation inputs for SER: spectrograms and scalograms. Speech signals from four databases (Emo-DB, RAVDESS, SAVEE, and a mix of all three) were converted into each type of representation and were used to train variations of a convolutional neural network (CNN) VGG16 Model-3. Results show that the scalogram-based models have a higher mean f1-score compared to the spectrogram-based models; however, further analysis indicate that the difference is statistically insignificant at a 95% confidence level. In conclusion, spectrograms and scalograms have statistically the same performance on the systems presented.

KW - CNN

KW - Fourier Transform

KW - Scalogram

KW - SER

KW - Spectrogram

KW - Wavelet Transform

UR - http://www.scopus.com/inward/record.url?scp=85166020806&partnerID=8YFLogxK

U2 - 10.1109/ISSC59246.2023.10162085

DO - 10.1109/ISSC59246.2023.10162085

M3 - Conference Paper published in Proceedings

AN - SCOPUS:85166020806

T3 - 2023 34th Irish Signals and Systems Conference, ISSC 2023

SP - 1

EP - 6

BT - 2023 34th Irish Signals and Systems Conference, ISSC 2023

PB - IEEE, Institute of Electrical and Electronics Engineers

T2 - 34th Irish Signals and Systems Conference, ISSC 2023

Y2 - 13 June 2023 through 14 June 2023

ER -

Scalogram vs Spectrogram as Speech Representation Inputs for Speech Emotion Recognition Using CNN

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this