Investigating the Utility of wav2vec 2.0 Hidden Layers for Detecting Multiple Sclerosis

Gosztolya, Gábor and Tóth, László and Svindt, Veronika and Bóna, Judit and Hoffmann, Ildikó (2025) Investigating the Utility of wav2vec 2.0 Hidden Layers for Detecting Multiple Sclerosis. In: Speech and Computer. Lecture Notes in Computer Science (15299). Springer Nature Switzerland, Cham, pp. 297-308. ISBN 9783031779602; 9783031779619

Text 2024-specom-sm-all-layers.pdf - Published Version Restricted to Repository staff only Download (415kB) | Request a copy

Abstract

Multiple sclerosis (MS) is a chronic autoimmune neurodegenerative disease, affecting the central nervous system. The disease can induce various symptoms, such as adversarily affecting the speech of the subject in various ways, therefore allowing the use of automatic speech analysis for the detection of MS and for monitoring the condition of the patient. Owing to data scarcity, however, deep neural networks are usually not employed for this task as classifiers, but are used as feature extractors. This is the case for self-supervised networks such as wav2vec 2.0 as well, where a straightforward source of embeddings (used as features) are the last layers of the convolutional (lower) and fine-tuned (higher) blocks. In this study we investigate whether extracting the embeddings from some other, inner layer of the fine-tuned (transformer) block can help improve MS detection performance. Tested on two speech tasks, we found that the lowest one-third of the 24 fine-tuned layers proved to be the most suitable for feature extraction, which led to statistically significant improvements in the AUC scores for both speech tasks.

Actions (login required)

Edit Item