Bibliometric indicators in the context of the evaluation of science and of Brazilian graduate education: the thermometer measures temperature, not distance

Abstract

Bibliometric indicators – the JCR, SNIP, SJR, CiteScore, FWCI, and the percentiles and quartiles of the Web of Science and Scopus indexing databases, along with the alternative metrics such as Altmetrics and PlumX – are currently the main instruments used as proxies for assessing the quality and impact of scientific journals and articles. Despite their wide usage, these indicators are frequently misunderstood and often misused – especially when applied to comparisons between fields with structurally distinct sizes and citation culture. This paper discusses the logic underlying their construction, the equations used to calculate them, and the conditions under which these indicators could be used, with emphasis on the specificities of scientific fields with smaller communities – such as the multidisciplinary sciences, the applied social sciences, and the humanities – which may be placed at a disadvantage by metrics calibrated for fields whose articles have high citation density and large collaboration networks. It argues that the established bibliometric indicators are necessary instruments and that, when combined with rigorous indexing processes, they even serve as a defense against journals whose editorial practices do not ensure the integrity of the publication process – referred to in the international literature as predatory. In this context, however, they must be understood within their limits and applied to the field for which they were designed for, within a framework of responsible and contextualized evaluation. In Brazil, this debate takes on the scenario whereby there is a transition from the article classification formerly associated with the CAPES Qualis methodology, centered mainly on journal percentiles, to the three complementary classification procedures established for the evaluation of graduate programs in the 2025-2028 quadrennium. These procedures incorporate both journal-level bibliometric indicators – percentiles, the CiteScore, the JIF, and, alternatively, the H5 – and bibliometric indicators calculated directly at the article level, such as the FWCI. In this new scenario, understanding these instruments ceases to be merely a desirable technical competence and becomes an operational requirement for researchers and graduate program coordinators.