Composing Diverse Ensembles of Convolutional Neural Networks by Penalization

Harangi, Balázs and Baran, Ágnes and Beregi-Kovács, Marcell and Hajdu, András (2022) Composing Diverse Ensembles of Convolutional Neural Networks by Penalization. Other. University of Debrecen. (Submitted)

Text SPRINGER_Journal_Skin_Lesion_Detection_with_Revised_Correlation_Ensemble_of_Convolutional_Neural_Networks__Copy_.pdf Restricted to Repository staff only Download (3MB)

Abstract

This paper investigates how an efficient ensemble of deep convolutional neural networks (CNNs) can be created by forcing them to adjust their parameters during backpropagation to increase diversity in their decisions. More specifically, we join some standard, well-known neural network architectures via a fully-connected layer and introduce a new term in the loss function as a correlation penalty to obstruct the similar operation of the individual neural networks. With this additional term, we implement the standard guideline of ensemble creation to increase the members’ diversity for CNNs in a more detailed and flexible way than the similar existing techniques. Since our approach is a general one, it can be applied to various classification tasks. Accordingly, we demonstrate its efficiency in challenging medical image analysis and natural image classification problems. Besides the theoretical considerations and foundations, our experimental results suggest that the proposed approach is competitive. Namely, on the one hand, the classification rate of the ensemble trained in this way has outperformed all the individual accuracies of the state-of-the-art member CNNs according to the standard error functions of these application domains. On the other hand, it is also validated that the ensemble members get more diverse, and their accuracies are raised by adding the penalization term. Finally, a comparative study with other state-of-the-art ensemble-based approaches recommended for the same classification tasks has also confirmed the superiority of our method.

Actions (login required)

Edit Item