Battling High Impact Factor Syndrome II

Last friday we had a great colloquium at UQ from Carl Caves on High-impact-factor syndrome: What, why, and what to do.

Much of the talk followed Carl's article The High-impact-factor syndrome on The Back Page of the American Physical Society News. I posted about it before.

Here are a few new things that emerged.

Reinhardt Werner had a nice piece in Nature, The focus on bibliometrics makes papers less useful. The comments and his responses are worth reading.

Last week Nature published The Leiden Manifesto for Research Metrics.

Nature Publishing Group has launched the Nature Index to rate individuals, departments, institutions, and countries. They claim it is a "global indicator of high quality research". It is based on only 68 journals, including many NPG journals! For example, the only APS journals included are PRL and the Rapid Communications parts of PRA, PRB, and PRD. Journal of Chemical Physics is not included. There are no mathematics journals.
I find this enterprise rather disturbing.

One needs to consider not just the Impact Factor which is a mean (average) but rather the width and the shape of the distribution.
For example, for Nature Physics, the bottom 50 per cent have 7 citations/paper/year. This is the same as the impact factor for PRL. Hence, even if you believe in such citation measures, half of the Nature Physics papers are really just like a PRL!

What are some of the problems with HIF syndrome?

Campbell's law will come into play.

"The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Gaming the system is inevitable.

Scientists surrender their research agenda to the Editors of Nature.

There is a tendency toward short, punchy, "hit and run" papers.

There is a trend towards hype and salemanship and fluff. This means a reduction in the commitment to the search for truth and scientific integrity, two things that set science apart as a social enterprise.