Beyond JA4+: Flow Statistics vs. TLS Fingerprinting for Encrypted Malware Detection

Lipcsey-Magyar, Márton Pál and Madarász, Attila Ármin and Pekar, Adrian (2026) Beyond JA4+: Flow Statistics vs. TLS Fingerprinting for Encrypted Malware Detection. INFOCOMMUNICATIONS JOURNAL, 18 (1). pp. 27-42. ISSN 2061-2079

Preview

Text ICJ_2026_1_4.pdf - Published Version Download (2MB) | Preview

Abstract

The deployment of Encrypted Client Hello (ECH) challenges TLS fingerprinting, a widely used approach for encrypted malware detection, by encrypting the handshake fields these methods rely on. This paper presents a systematic evaluation of flow-based statistical features as a handshakeindependent alternative to fingerprinting. Through validation against the official JA4+ implementation, we establish limitations in fingerprinting approaches for this corpus: only 64.9% of malware families possess unique signatures, placing an inherent ceiling on achievable recall in our evaluation. We evaluate flow-level features—packet counts, timing patterns, and size distributions—across 27 experimental configurations on a dataset of 16,542 flows spanning 101 families (59 malware and 42 benign applications). Random Forest classifiers using combined flow statistics and sequential packet length features achieve 98.11% F1-score for binary malware detection with 97.22% recall, substantially exceeding fingerprinting’s theoretical recall bound of 64.9%. For fine-grained family identification, we obtain 54.81% macro F1 across 101 classes and 48.71% macro F1 for malwareonly attribution, demonstrating that flow-based methods retain meaningful discriminative power where fingerprinting abstains. Across all tasks, Random Forest consistently outperforms neural networks and k-NN, with performance gaps widening in complex multiclass scenarios. These findings highlight flow-based classification as a practical and reproducible approach that can help maintain network security visibility as ECH deployment progresses, showing that behavioral traffic patterns are expected to provide durable signals for detection even as handshake fields become encrypted

Actions (login required)

Edit Item