Kiss-Vetráb, Mercedes and Gosztolya, Gábor (2023) Aggregation Strategies of Wav2vec 2.0 Embeddings for Computational Paralinguistic Tasks. LECTURE NOTES IN COMPUTER SCIENCE, 14338. pp. 79-93. ISSN 0302-9743
Text
2023-specom-aggregation.pdf - Published Version Restricted to Registered users only Download (738kB) | Request a copy |
Abstract
Throughout the history of computational paralinguistics, numerous feature extraction, preprocessing and classification techniques have been used. One of the important challenges in this subfield of speech technology is handling utterances with different duration. Since standard speech processing features (such as filter banks or DNN embeddings) are typically frame-level ones and we would like to classify whole utterances, a set of frame-level features have to be converted into fixed-sized utterance-level features. The choice of this aggregation method is often overlooked, and simple functions like mean and/or standard deviation are used without solid experimental support. In this study we take wav2vec 2.0 deep embeddings, and aggregate them with 11 different functions. We sought to obtain a subset of potentially optimal aggregation functions, because there are no general rules yet that can be applied universally between subtopics. Besides testing both standard and non-traditional aggregation strategies individually, we also combined them to improve the classification performance. By using multiple aggregation functions, we were able to achieve significant improvements on three public paralinguistic corpora.
Item Type: | Article |
---|---|
Additional Information: | Export Date: 17 January 2024 Correspondence Address: Vetráb, M.; Institute of Informatics, Hungary; email: vetrabm@inf.u-szeged.hu |
Uncontrolled Keywords: | Paralinguistics, Wav2vec 2.0, Embeddings, Aggregation |
Subjects: | P Language and Literature / nyelvészet és irodalom > P0 Philology. Linguistics / filológia, nyelvészet Q Science / természettudomány > QA Mathematics / matematika > QA75 Electronic computers. Computer science / számítástechnika, számítógéptudomány Q Science / természettudomány > QA Mathematics / matematika > QA76.76 Software Design and Development / Szoftvertervezés és -fejlesztés |
SWORD Depositor: | MTMT SWORD |
Depositing User: | MTMT SWORD |
Date Deposited: | 22 Aug 2024 08:30 |
Last Modified: | 22 Aug 2024 08:30 |
URI: | https://real.mtak.hu/id/eprint/203111 |
Actions (login required)
Edit Item |