Recently I encountered a new metric, the "5 year h-index" which was being used to evaluate someone's research performance. Explicity, one counts the number h of papers published in the last 5 years that have been cited more than h times. Perhaps one might argue that this is a good metric for deciding whether to give someone a grant now. Afterall, just because 10 or 20 years ago they published highly cited papers does not mean that right now they are at the cutting edge. However, I do not agree.
I think this is a highly unreliable metric because there is significant noise. Except for a few rare exceptional papers, citations within a few years of publication will be low (1-10?). Hence, comparing two people with 5 year h-indexes, say one with a 6 and another with a 10, I would contend is meaningless.
Two of the most cited papers in Physical Review journals are the EPR (Einstein Podolsky Rosen) paper and Steven Weinberg's electro-weak interaction paper. The latter attracted about one citation per year for the first 5 years after publication! The former attracted about 10 citations in the first 2 years and then none for more than ten years!
When Jorge Hirsch introduced the h-index the whole point what to find some measure for a lifetime of scientific achievement. I still think it does provide a useful coarse-grained measure for that.
Subscribe to:
Post Comments (Atom)
It is also true that many papers with "pop" have little staying power once the trends of the day have passed. This index would lead to a world populated by Andy Warhol scientists, and not a Van Gough in the bunch.
ReplyDeleteSounds better for ECRs than the usual h-index. On first thought at least.
ReplyDeleteThis argument can apply in reverse as well. Many journals (and their editors) gloat over increases in Impact Factor, which measures an even shorter time the a 5 year h-index. Some journals have started to calculate their 5 year Impact Factor, which includes papers published up to 5 years ago. This seems more sensible than the regular Impact Factor.
ReplyDeleteHaving some kind of time horizon associated with an h-index or an Impact Factor has some value. If a person or a journal made a large impact 10-20 years ago but has done things with little impact (or nothing) since, this is meaningful information. The main question this raises for me is what the appropriate time horizon is.