Measuring the effect of structural differences between Web of Science and Scopus on research impact assessment

Soós, Sándor and Kiss, Anna and Vida, Zsófia Viktória (2026) Measuring the effect of structural differences between Web of Science and Scopus on research impact assessment. JOURNAL OF INFORMETRICS, 20 (2). No. 101810. ISSN 1751-1577

Preview

Text 1-s20-S175115772600043X-main.pdf - Published Version Available under License Creative Commons Attribution Non-commercial. Download (1MB) | Preview

Abstract

Our work elaborates on the study of how the choice of databases affects the measurement of scientific impact. We address a gap in the analysis of the “crown indicator”, the Normalized Citation Score (NCS), when applied in practical settings in different database contexts. It is shown that database-specific structural features have a profound effect on the differences in NCS values across databases, and that the benchmark component of the metric can be used as a proxy for quantifying the effect of such differences (“benchmark effect”). The primary aim of our analysis was the measurement of the effect of benchmark differences on impact scores at various levels of assessment. To that end, we developed and theoretically validated a novel metric for quantifying the effect of benchmark differences on NCS discrepancies across databases (Citation-Benchmark Differentials Ratio, CBDR). We applied the CBDR metric on the five-year country-level publication output of Hungary, retrieved from WoS and Scopus. Measurement results were subjected to statistical analyses to reveal the potential benchmark effect in various aggregations of the publication record, especially by research fields, research organizations (HEIs), journal quartiles, and publication years. Results show that a considerable fraction of the country-level output is affected by the benchmark effect, and no substantial differences emerge between the units of assessment in any aggregation under study. It is demonstrated that database-specific citation counts and journal coverage only partially account for the differences between WoS- and Scopus-based normalized citation scores.

Actions (login required)

Edit Item