Towards Unbiased Exploration in Partial Label Learning

Zombori, Zsolt and Agapi, Rissaki and Kristóf, Szabó and Wolfgang, Gatterbauer and Michael, Benedikt (2024) Towards Unbiased Exploration in Partial Label Learning. JOURNAL OF MACHINE LEARNING RESEARCH, 412 (Publis). pp. 1-56. ISSN 1532-4435

Preview

Text 23-0868.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

We consider learning a probabilistic classifier from partially-labelled supervision (inputs denoted with multiple possibilities) using standard neural architectures with a softmax as the final layer. We identify a bias phenomenon that can arise from the softmax layer in even simple architectures that prevents proper exploration of alternative options, making the dynamics of gradient descent overly sensitive to initialisation. We introduce a novel loss function that allows for unbiased exploration within the space of alternative outputs. We give a theoretical justification for our loss function, and provide an extensive evaluation of its impact on synthetic data, on standard partially labelled benchmarks and on a contributed novel benchmark related to an existing rule learning challenge.

Actions (login required)

Edit Item