REAL

Aggregation Strategies of Wav2vec 2.0 Embeddings for Computational Paralinguistic Tasks

Kiss-Vetráb, Mercedes and Gosztolya, Gábor (2023) Aggregation Strategies of Wav2vec 2.0 Embeddings for Computational Paralinguistic Tasks. LECTURE NOTES IN COMPUTER SCIENCE, 14338. pp. 79-93. ISSN 0302-9743

[img] Text
2023-specom-aggregation.pdf - Published Version
Restricted to Registered users only

Download (738kB) | Request a copy

Abstract

Throughout the history of computational paralinguistics, numerous feature extraction, preprocessing and classification techniques have been used. One of the important challenges in this subfield of speech technology is handling utterances with different duration. Since standard speech processing features (such as filter banks or DNN embeddings) are typically frame-level ones and we would like to classify whole utterances, a set of frame-level features have to be converted into fixed-sized utterance-level features. The choice of this aggregation method is often overlooked, and simple functions like mean and/or standard deviation are used without solid experimental support. In this study we take wav2vec 2.0 deep embeddings, and aggregate them with 11 different functions. We sought to obtain a subset of potentially optimal aggregation functions, because there are no general rules yet that can be applied universally between subtopics. Besides testing both standard and non-traditional aggregation strategies individually, we also combined them to improve the classification performance. By using multiple aggregation functions, we were able to achieve significant improvements on three public paralinguistic corpora.

Item Type: Article
Additional Information: Export Date: 17 January 2024 Correspondence Address: Vetráb, M.; Institute of Informatics, Hungary; email: vetrabm@inf.u-szeged.hu
Uncontrolled Keywords: Paralinguistics, Wav2vec 2.0, Embeddings, Aggregation
Subjects: P Language and Literature / nyelvészet és irodalom > P0 Philology. Linguistics / filológia, nyelvészet
Q Science / természettudomány > QA Mathematics / matematika > QA75 Electronic computers. Computer science / számítástechnika, számítógéptudomány
Q Science / természettudomány > QA Mathematics / matematika > QA76.76 Software Design and Development / Szoftvertervezés és -fejlesztés
SWORD Depositor: MTMT SWORD
Depositing User: MTMT SWORD
Date Deposited: 22 Aug 2024 08:30
Last Modified: 22 Aug 2024 08:30
URI: https://real.mtak.hu/id/eprint/203111

Actions (login required)

Edit Item Edit Item