SUE: Sparsity-based Uncertainty Estimation via Sparse Dictionary Learning

Ficsor, Tamás and Berend, Gábor (2025) SUE: Sparsity-based Uncertainty Estimation via Sparse Dictionary Learning. In: The 2025 Conference on Empirical Methods in Natural Language Processing. (In Press)

Preview

Text ACL_ARR_2025_May___UE__Camera_Ready_.pdf - Published Version Download (706kB) | Preview

Abstract

The growing deployment of deep learning models in real-world applications necessitates not only high predictive accuracy, but also mechanism to identify unreliable predictions, especially in high-stakes scenarios where decision risk must be minimized. Existing methods estimate uncertainty by leveraging predictive confidence (e.g., Softmax Response), structural characteristics of representation space (e.g., Mahalanobis distance), or stochastic variation in model outputs (e.g., Bayesian inference techniques such as Monte Carlo Dropout). In this work, we propose a novel uncertainty estimation (UE) frameworkbasedonsparse dictionary learning by identifying dictionary atoms associated with misclassified samples. We leverage pointwise mutual information (PMI) to quantify the association between sparse features and predictive failure. Our methodSparsity-based Uncertainty Estimation (SUE)is computationally efficient, offers interpretability via atom-level analysis of the dictionary, has no assumption about the class distribution (unlike Mahalanobis distance). We evaluated SUE on several NLU benchmarks (GLUE and ANLI tasks) and sentiment analysis benchmarks (Twitter, ParaDetox, and Jigsaw). In general, SUE outperforms or matches the performance of other methods. SUE performs particularly well when there is considerable uncertainty in the model, i.e., when the model lacks high precision.

Actions (login required)

View Item