A Scientometric Method to Evaluate the Academic Research Performance

Cornel RESTEANU¹, Constantin POPESCU², Mădălina Ecaterina POPESCU³

¹ I C I Bucharest
(National Institute for R & D in Informatics)
8-10 Averescu Blvd.
011455 Bucharest 1, Romania
resteanu@ici.ro

² “Valahia” University,
Târgovişte, Romania
constantinpop1967@yahoo.com
³ Bucharest University of Economic Studies,
National Scientific Research Institute for Labour and Social Protection,
madalina.ecaterina.popescu@gmail.com

Abstract: The paper presents a method for evaluating and ranking researchers affiliated with a research and development institute. The method has been applied within the Multi-Attribute Decision Making paradigm, multi-decision maker and mono-state of nature sub-paradigm. The decision-makers are several members of the institute’s Scientific Council, chosen by means of ONICESCU method to make the mathematical modelling. Researchers’ expertise lies in Academic Statistics, belonging to the scientometrics domain, whose weights are constructed by DEMATEL method. The evaluation of researchers uses MAUT method. By adopting the current approach, the ranking of researchers upon their scientific merit is thus made possible.

Keywords: Scientometrics, Scientific Merit, Researchers Evaluation and Ranking, Multi-Attribute Decision Making, ONICESCU, DEMATEL, MAUT Methods.

>>Full text
CITE THIS PAPER AS:
Cornel RESTEANU, Constantin POPESCU, Mădălina Ecaterina POPESCU, A Scientometric Method to Evaluate the Academic Research Performance, Studies in Informatics and Control, ISSN 1220-1766, vol. 25(4), pp. 433-444, 2016. https://doi.org/10.24846/v25i4y201604

Introduction

The topic of Scientometrics was first introduced by Nalimov in 1969 [8]. Since then, there has been a growing interest towards its practical applications, as confirmed by the vast international activity in the field.

Nowadays there are several professional organizations dedicated to this field (such as International Society of Scientometrics and Informetrics (ISSI) or SciBiolMed), as well as scientific journals (Scientometrics, Journal of Scientometric Research, Journal of Informetrics), and international conferences (The 15th ISSI International Conference – „Future of scientometrics”. June 29 – July 4, 2015, Istanbul, Turcia). Also there are even commercial enterprises that develop scientometric software.

The term of Scientometrics can broadly be described as the study of measuring science, technology and innovation. In practice, any search engine that uses data mining techniques can now compute a researcher’s h index. However, not all searches compute the same h index value for the same researcher.

Actually, we are now facing the problem that there is no single metric to evaluate a researcher. It may be impact factor, H-index, g-index or RG score. For instance, if a researcher publishes lots of papers in low impact factor journals it may end up with limited impact on science. On the other hand, if a researcher aims high impact factor journals, he/she may end up with very few papers published because of the high rejection rate of high profile journals. Thus, the research community has not so far agreed on a clear type of score and lots of criticisms are addressed to every existing assessment scheme used to evaluate the performance of the researchers.

Despite the growing interest on the topic of Scientometrics, little has been written so far at national level. Thus, we will try to fill in this gap by presenting in this paper a novel scientometric method for highlighting the researchers’ value. An evaluation and ranking algorithm combines three Multi-Attribute Decision Making (MADM) [10], [11] methods in a very effective manner so as to ensure a clear researchers’ raking based on their scientific merit.

REFERENCES

ANDREICA, M., C. RESTEANU, R. TRANDAFIR, Parallel and Distributed Software Assessment in Multi-Attribute Decision Making Paradigm, Studies in Informatics and Control, vol. 23, Issue 2, 2014, pp. 133-142.
DYER, J. S., MAUT – Multiattribute Utility Theory. Springer New York. Multiple Criteria Decision Analysis: State of the Art Surveys, 2005 pp. 265-292.
DYER, J. S., P. C. FISHBURN, R. E. STEUER, J. WALLENIUS, S. ZIONTS (1992). Multiple Criteria Decision Making, Multi-Attribute Utility Theory: The next ten years, Management Science, Vol. 38, No. 5, pp. 645-654.
FISHBURN, P. C., Utility Theory for Decision Making. Huntington, NY. Robert E. Krieger Publishing Co., 1970. ISBN 978-0-471-26060-8.
FONTELA, E., A. GABUS, DEMATEL, Innovative Methods, Technical report no. 2, Structural analysis of the world problematique, Battelle Geneva Research Institute, 1974.
LEE, H. S., G. H. TZENG, W. YEIH. Y. J. WANG, S. C. YANG, Revised DEMATEL: Resolving the Infeasibility of DEMATEL, Applied Mathematical Modelling 34, Elevier, 2013, ISSN: 0307-904X.
MATEO, J. R. S. C., Multi-attribute Utility Theory, In: Multi Criteria Analysis in the Renewable Energy Industry. Springer, 2012, p. 63-72.
NALIMOV, V. V., Z. M. MULCHENKO, Наукометрия, Изучение развития науки как информационного процесса, Moscow: Nauka, 1969, p. 191.
ONICESCU, O., Procedee de estimare comparativă a unor obiecte purtătoare de mai multe caracteristici, Revista de Statistică, no. 4, 1970 (in Romanian).
TZENG, H., J. J. HUANG, Multiple Attribute Decision Making: Methods and Applications, Chapman & Hall, CRC Press, 2011, ISBN: 1439861579, 9781439861578.
YOON, K. P., C. L. HWANG, Multiple Attribute Decision Making: An Introduction, SAGE Publications, 1995.