REAL

Uncertainty approximation in neural networks using parameter-space proximity regularization

Vetró, Mihály and Hullám, Gábor (2021) Uncertainty approximation in neural networks using parameter-space proximity regularization. In: Proceedings of the 28th PhD Mini-Symposium. BME MIT, Budapest, pp. 8-11.

[img]
Preview
Text
Vetro_Hullam_Uncertainty_MiniSympMIT2021.pdf

Download (5MB) | Preview

Abstract

A common question regarding the application of neural networks is whether the predictions of the model are reliable, in other words, what is the degree of uncertainty of our model. Generalizing a neural network into a Bayesian neural network is a frequent choice to quantify uncertainty. This means the extension of scalar weights and biases of the network to random variables. In terms of implementation, there are two main approaches: (1) the assumption of an unknown distribution for the random variables (i.e. weights) which are sampled and then utilized to infer the distribution of the output; (2) the assumption of a prior distribution for the random variables, and search for the output in the form of a similar posterior distribution. A common drawback of both approaches is that their scalability is limited, and generally require more computational resources than a simple neural network. In this paper, we introduce a new parameter sampling method, which maximizes the pairwise distance in parameter space between the models in the ensemble, therefore ensuring model diversity. Results indicate that this method generally needs less samples from the parameter space to be effective, thus it surpasses the other investigated approximation methods in terms of scalability. Finally, we apply our method to a real-life problem related to semantic segmentation of objects relevant in urban driving environments.

Item Type: Book Section
Subjects: R Medicine / orvostudomány > RC Internal medicine / belgyógyászat > RC0321 Neuroscience. Biological psychiatry. Neuropsychiatry / idegkórtan, neurológia, pszichiátria
R Medicine / orvostudomány > RM Therapeutics. Pharmacology / terápia, gyógyszertan
SWORD Depositor: MTMT SWORD
Depositing User: MTMT SWORD
Date Deposited: 29 Sep 2022 07:56
Last Modified: 29 Sep 2022 07:56
URI: http://real.mtak.hu/id/eprint/150514

Actions (login required)

Edit Item Edit Item