Email from Luca Aceto:
I have recently been informed that the editorial board of the journal Mathematical Structures in Computer Science has put together a critical note on the (ab)use of bibliometric data, which will appear in the issue 19.1 of that journal.
The note has been written by the editor in chief, Giuseppe Longo, and subscribed by all the members of the editorial board of that journal.I have made it available at
http://www.ru.is/luca/webnotes/BibliometricsMSCS.pdf.
The note expresses the worries of the scientists in the board about
* the way the evaluation of research activity is evolving in many countries,
* the general trend to use criteria purely based on numbers and citation indexes in judging the quality of researchers and
* the fact that the management of the data used in the numerical evaluations is entrusted to private agencies, whose methodologies and software might be rather dubious or cannot be subjected to scrutiny by the research community.
Similar concerns have been raised by others. See, e.g., the talk "Bibliometric Evaluation of Computer Science - Problems and Pitfalls" by Friedemann Mattern (Institute for Pervasive Computing, Department of Computer Science, ETH Zurich), which is available at
http://www.informatics-europe.org/ECSS08/papers/mattern.pdf.
There is also a well-documented, and very interesting joint report from three mathematical boards (IMU+ICIAM+IMS)
http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf
(notice a reference about the control on citation indices by Thomson Sci. et al on page 17).
Finally, the "Sector Overview Report from the Computer Science and Informatics Sub-Panel (UoA 23)" after the British nationwide "Research Assessment Exercise 2008" (available at http://www.rae.ac.uk) includes the following passage:
".... We frequently found that citation counts were poorly correlated with the sub-panel’s assessment of the impact of the work examined. Citations also varied widely between research areas. For instance, much of the highly significant theoretical research, in which the UK is world leading, typically attracts low citation counts. Despite these low citations, the work is often found to have profound long-term impact on practical aspects of the field. ...."
I hope that some of you will find these interesting.
Cheers,
Luca
I have recently been informed that the editorial board of the journal Mathematical Structures in Computer Science has put together a critical note on the (ab)use of bibliometric data, which will appear in the issue 19.1 of that journal.
The note has been written by the editor in chief, Giuseppe Longo, and subscribed by all the members of the editorial board of that journal.I have made it available at
http://www.ru.is/luca/webnotes/BibliometricsMSCS.pdf.
The note expresses the worries of the scientists in the board about
* the way the evaluation of research activity is evolving in many countries,
* the general trend to use criteria purely based on numbers and citation indexes in judging the quality of researchers and
* the fact that the management of the data used in the numerical evaluations is entrusted to private agencies, whose methodologies and software might be rather dubious or cannot be subjected to scrutiny by the research community.
Similar concerns have been raised by others. See, e.g., the talk "Bibliometric Evaluation of Computer Science - Problems and Pitfalls" by Friedemann Mattern (Institute for Pervasive Computing, Department of Computer Science, ETH Zurich), which is available at
http://www.informatics-europe.org/ECSS08/papers/mattern.pdf.
There is also a well-documented, and very interesting joint report from three mathematical boards (IMU+ICIAM+IMS)
http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf
(notice a reference about the control on citation indices by Thomson Sci. et al on page 17).
Finally, the "Sector Overview Report from the Computer Science and Informatics Sub-Panel (UoA 23)" after the British nationwide "Research Assessment Exercise 2008" (available at http://www.rae.ac.uk) includes the following passage:
".... We frequently found that citation counts were poorly correlated with the sub-panel’s assessment of the impact of the work examined. Citations also varied widely between research areas. For instance, much of the highly significant theoretical research, in which the UK is world leading, typically attracts low citation counts. Despite these low citations, the work is often found to have profound long-term impact on practical aspects of the field. ...."
I hope that some of you will find these interesting.
Cheers,
Luca
