Guilt. I don't spend as much time on each application as perhaps I should. I don't read every word, cross check details, learn the necessary science, find an expert, ...
Guilt. I do sometimes look at metrics. But, in my defense this is very coarse-grained. Most of the time it tells me little or what I can guess already. However, occasionally I think it does help. For example, someone who is decades past their Ph.D and with only single digit citations on their hundred plus papers. Or, someone with a few recent papers with 50-100 citations, may mean something.
Frustration. That I have to assess non-competitive applications. Why did this person waste all the time preparing an application. Did any senior colleague advise against it? Or does the applicant live in a dream world or have a massive ego?
Irritation. Applicants should know their application will be reviewed by a multi-disciplinary panel but still fill it with jargon.
Irritation. Most applicants list journal impact factors to 4 or 5 significant figures. I know journals do this, but it reflects an uncritical acceptance of what the numbers really mean. This reflects poorly on the applicant. If one looks at the broad distribution of citation rates for papers in a specific journal, not just the average citation rates, one can see that at best only two significant figures is meaningful.
Sympathetic. Although I am scathing about those with High Impact Factor syndrome, I do see how seductive it is, particularly when reviewing applications from other fields. It provides an easy way out from the hard work and subjectiveness of a real scientific assessment.
Irritation. Hype about the applicant, research field, or project.
Joy. When I learn something new or when I see someone is doing something cool or worthwhile for global society (or at least appears to be).
Anxious. Writing a grant proposal that sells is not the same skill as doing excellent research. I may be deceived and backing the wrong people [those good at marketing] rather than really creative and solid scientists who don't have the skills or motivation to sell themselves or their project.
Concerned. There is too much hyperactivity (lots of papers, conferences, seminars, grad. students, ...). It seems quantity trumps quality. I desperately seek to find just one significant nugget of scientific knowledge that each applicant has produced.
Relief. When I am done.
But I am still grumpy for a few days afterwards.
I have similar issues when reviewing papers for PRL or higher. Unfortunately a lot of that is marketing, and while it's easier to judge the science (results are there instead of a communication of expectations), it's easy being influenced by the fact that the senior author has already many "high impact" paper (erm, papers in supposedly high impact journals, which is a significantly different animal).
ReplyDeleteBut many of the same issues, and associated human reactions, pop up.
I hear your pain Ross!
ReplyDelete