Recently I encountered a new metric, the "5 year h-index" which was being used to evaluate someone's research performance. Explicity, one counts the number h of papers published in the last 5 years that have been cited more than h times. Perhaps one might argue that this is a good metric for deciding whether to give someone a grant now. Afterall, just because 10 or 20 years ago they published highly cited papers does not mean that right now they are at the cutting edge. However, I do not agree.
I think this is a highly unreliable metric because there is significant noise. Except for a few rare exceptional papers, citations within a few years of publication will be low (1-10?). Hence, comparing two people with 5 year h-indexes, say one with a 6 and another with a 10, I would contend is meaningless.
Two of the most cited papers in Physical Review journals are the EPR (Einstein Podolsky Rosen) paper and Steven Weinberg's electro-weak interaction paper. The latter attracted about one citation per year for the first 5 years after publication! The former attracted about 10 citations in the first 2 years and then none for more than ten years!
When Jorge Hirsch introduced the h-index the whole point what to find some measure for a lifetime of scientific achievement. I still think it does provide a useful coarse-grained measure for that.
Subscribe to:
Post Comments (Atom)
From Leo Szilard to the Tasmanian wilderness
Richard Flanagan is an esteemed Australian writer. My son recently gave our family a copy of Flanagan's recent book, Question 7 . It is...
-
Is it something to do with breakdown of the Born-Oppenheimer approximation? In molecular spectroscopy you occasionally hear this term thro... -
If you look on the arXiv and in Nature journals there is a continuing stream of people claiming to observe superconductivity in some new mat...
-
I welcome discussion on this point. I don't think it is as sensitive or as important a topic as the author order on papers. With rega...
It is also true that many papers with "pop" have little staying power once the trends of the day have passed. This index would lead to a world populated by Andy Warhol scientists, and not a Van Gough in the bunch.
ReplyDeleteSounds better for ECRs than the usual h-index. On first thought at least.
ReplyDeleteThis argument can apply in reverse as well. Many journals (and their editors) gloat over increases in Impact Factor, which measures an even shorter time the a 5 year h-index. Some journals have started to calculate their 5 year Impact Factor, which includes papers published up to 5 years ago. This seems more sensible than the regular Impact Factor.
ReplyDeleteHaving some kind of time horizon associated with an h-index or an Impact Factor has some value. If a person or a journal made a large impact 10-20 years ago but has done things with little impact (or nothing) since, this is meaningful information. The main question this raises for me is what the appropriate time horizon is.