Negative Sampling in Variational Autoencoders

Csiszárik, Adrián and Benkő, Beatrix and Varga, Dániel (2022) Negative Sampling in Variational Autoencoders. In: 2022 IEEE 2nd Conference on Information Technology and Data Science (CITDS). Institute of Electrical and Electronics Engineers (IEEE), Piscataway (NJ), pp. 63-68. ISBN 9781665496537; 9781665496520

Preview

Text 1910.02760v3.pdf - Draft Version Download (153kB) | Preview

Abstract

Modern deep artificial neural networks have achieved great success in the domain of computer vision and beyond. However, their application to many real-world tasks is undermined by certain limitations, such as overconfident uncertainty estimates on out-of-distribution data or performance deterioration under data distribution shifts. Several types of deep learning models used for density estimation through probabilistic generative modeling have been shown to fail to detect out-of-distribution samples by assigning higher likelihoods to anomalous data. We investigate this failure mode in Variational Autoencoder models, which are also prone to this, and improve upon the out-of-distribution generalization performance of the model by employing an alternative training scheme utilizing negative samples. We present a fully unsupervised version: when the model is trained in an adversarial manner, the generator's own outputs can be used as negative samples. We demonstrate empirically the effectiveness of the approach in reducing the overconfident likelihood estimates of out-of-distribution inputs on image data. © 2022 IEEE.

Actions (login required)

View Item