Thursday, July 19, 2012

The best measure of research impact is ...

"whether the candidate's research has changed the community's view of chemistry in a positive way".

This is the main point of a nice Editorial, Assessing Academic Researchers that just appeared in Angewandte Chemie by Richard Zare. He points out that this is the main criteria that is used to decide whether or not Assistant Professors in Chemistry at Stanford get tenure.
We do not look into how much funding the candidate has brought to the university in the form of grants. We do not count the number of published papers; we also do not rank publications according to authorship order. We do not use some elaborate algorithm that weighs publications in journals according to the impact factor of the journal. We seldom discuss h-index metrics, which aim to measure the impact of a researcher’s publications. We simply ask outside experts, as well as our tenured faculty members, whether a candidate has significantly changed how we understand chemistry.
I thank Seth Olsen for bringing the article to my attention.


  1. I suppose Stanford can reasonably expect this.

    However, wouldn't Weinberg's prl only fit this criterion once it had very many citations? In this sense one stellar paper once that changed our view of physics would be enough.

    So I'd suggest that the total number of citations of, say, your top 5 cited papers (or so) would quantify, in very many cases, this criterion stipulated by Zare.

  2. Hi Tony,

    Thanks for the comment.
    I agree that ultimately, landmark work produces lots of citations.
    But, the problem is that on the 5 year timescale of a tenure decision, citations are not a clear measure of long-term impact. Weinberg's case clearly shows this.
    Hence, one is left with the subjective judgement of ten or so experts who can best judge what is important.