Skip to main content

Library Subject Guides

6. Measure Impact: Other viewpoints

..

Research evaluation has become routine and often relies on metrics. But it is increasingly driven by data and not by expert judgement.

These metrics originally designed to increase the quality of research may be doing the exact opposite....

The use of metrics to measure research comes in for criticism ...

Edwards, M. A., & Roy, S. (2017). Academic research in the 21st century: Maintaining scientific integrity in a climate of perverse incentives and hypercompetition Environmental Engineering Science, 34(1), 51-61. 

Erren, T. C., & Groß, J. V. (2016). Research metrics: What about weighted citations? Scientometrics, 107(1), 315-316. doi: 10.1007/s11192-016-1841-5

Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429. 

Jarwal, S. D., Brion, A. M., & King, M. L. (2009). Measuring research quality using the journal impact factor, citations and ‘Ranked Journals’: Blunt instruments or inspired metrics? Journal of Higher Education Policy and Management, 31(4), 289-300. doi: 10.1080/13600800903191930

MacRoberts, M. H., & MacRoberts, B. R. (2018). The mismeasure of science: Citation analysis. Journal of the Association for Information Science and Technology, 69(3), 474-482. doi: 10.1002/asi.23970

Stephan, P., Veugelers, R., & Wang, J. (2017). Reviewers are blinkered by bibliometrics. Nature, 544(7651), 411-412. doi: 10.1038/544411a

Teixeira da Silva, J. A. (2017). The Journal Impact Factor (JIF): Science Publishing’s Miscalculating Metric. Academic Questions, 30(4), 433-441. doi: 10.1007/s12129-017-9671-3

Who to Contact

From the Independent Review of the Role of Metrics in Research Assessment and Management...

http://www.hefce.ac.uk/pubs/rereports/
year/2015/metrictide/

  • There is considerable scepticism among researchers, universities, representative bodies and learned societies about the broader use of metrics in research assessment and management.
     
  • Peer review commands widespread support as the primary basis for evaluating research outputs, proposals and individuals. However, others are enthusiastic about greater use of metrics.
     
  • Care must be taken when using metrics: other quantitative indicators and qualitative measures that respect research diversity will still be required.
     
  • Some metrics are known to be misused: journal impact factors, university rankings and citation counts are three examples.