Wav2vec 2.0 Embeddings Are No Swiss Army Knife : A Case Study for Multiple Sclerosis

Gosztolya, Gábor and Kiss-Vetráb, Mercedes and Svindt, Veronika and Bóna, Judit and Hoffmann, Ildikó (2024) Wav2vec 2.0 Embeddings Are No Swiss Army Knife : A Case Study for Multiple Sclerosis. In: 25th Interspeech Conference (Interspeech 2024). Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH . International Speech Communication Association (ISCA), Dublin, pp. 2499-2503.

Preview

Text 2024-interspeech-wav2vec2-vs-others.pdf - Published Version Download (234kB) | Preview

Abstract

In the past few years, self-supervised learning has revolutionalized automatic speech recognition. Self-supervised models such as wav2vec2, due to their generalization ability on huge unannotated audio corpora, were claimed to be state-ofthe-art feature extractors in paralinguistic and pathological applications as well. In this study we test embeddings extracted from a wav2vec 2.0 model fine-tuned on the target language as features on a multiple sclerosis audio corpus, using three speech tasks. After comparing the resulting classification performances with traditional features such as ComParE functionals, ECAPATDNN and activations of a HMM/DNN hybrid acoustic model, we found that wav2vec2-based models, surprisingly, only produced a mediocre classification performance. In contrast, the decade-old ComParE functionals feature set consistently led to high scores. Our results also indicate that the number of features correlates surprisingly well with classification performance.

Actions (login required)

Edit Item