Method and model comparison by sum of ranking differences in cases of repeated observations (ties)

Kollár-Hunek, Klára and Héberger, Károly (2013) Method and model comparison by sum of ranking differences in cases of repeated observations (ties). CHEMOMETRICS AND INTELLIGENT LABORATORY SYSTEMS, 127. pp. 139-146. ISSN 0169-7439

Text CHEMOMETRICS AND INTELLIGENT LABORATORY SYSTEMS_2013.pdf Restricted to Registered users only Download (1MB) | Request a copy

Abstract

Significant progress has been achieved since the introduction of the new similarity measure the sum of absolute ranking differences (SRDs) [TrAC – Trends in Anal. Chem. 29 (2010) 101-109]. Empirical evidences were accumulated about scaling, selection of the reference (benchmark) vector, cross-validation and grouping of variables (features, models, methods, etc.). The theory has been developed including the repeated observations (ties): (i) The exact theoretical distribution (null distribution) for 4 < number of objects < 9 has been calculated for SRD treatment of random numbers (it’s a kind of validation, a permutation test). All possible reference vectors with ties imply different distribution. (ii) Above number of objects eight (n > 8) an approximation has been developed using the Gaussian distribution fitted on the SRD distribution given by generating of three million ndimensional random vectors. The validity and features of the SRD methodology with ties are illustrated using two case studies: evaluation of a sensory panel and ranking of financial indicators.

Actions (login required)

Edit Item