Web-based facial expression recognition using hybrid deep learning

Hu, Ming and Kovásznai, Gergely (2025) Web-based facial expression recognition using hybrid deep learning. In: Proceedings of the International Conference on Formal Methods and Foundations of Artificial Intelligence. Eszterházy Károly Katolikus Egyetem Líceum Kiadó, Eger, pp. 102-114. ISBN 9789634963035

Preview

Text fmfai2025_pp102-114.pdf - Published Version Download (1MB) | Preview

Abstract

This paper present a hybrid ResNet+FPN+Transformer architecture for facial expression recognition, achieving 80.90% accuracy on FER- 2013 with a browser-based implementation using TensorFlow.js for client-side inference. We compare four model configurations: ResNet50 baseline, ResNet+FPN, ResNet+Transformer, and our full ResNet+FPN+Transformer model. Our hybrid architecture combines ResNet backbone features with Feature Pyramid Networks and transformer components to process facial features at multiple scales simultaneously. Our ResNet+FPN+Transformer model achieves 80.90% mean accuracy on FER-2013 (averaged over 5 independent training runs with different random initializations). Ablation studies confirm both FPN (+2.35%) and Transformer (+2.77%) components improve performance over the ResNet50 baseline (77.69%). Our web application features interactive visualization tools revealing the network’s decision-making process, including feature map animations and 3D neural network visualization. This browser-based implementation uses TensorFlow.js for client-side inference.

Actions (login required)

Edit Item