Wednesday, April 22, 2015

A basic but important research skill, 6: skepticism

Feynman said "The first principle is that you must not fool yourself and you are the easiest person to fool."

Walter Kauzmann emphasised that people will often believe what they want to believe rather than what the evidence before them suggests they should believe.

Students need to learn skepticism. Furthermore, it needs to be modelled to them by their advisors.
In particular, students should not just believe something because

- their advisor/supervisor believes it or tells them it is true
- it has been published, especially if it is in a luxury journal
- someone famous [or a group of famous people] claims it is true
- it is an exciting idea.

Basic but important questions to ask are:

What is the evidence? How reliable is the evidence?
Is there an alternative explanation, particularly a simpler one?

Maybe I am just becoming a grumpy old man, but I think I do increasingly encounter students and young researchers who lack this basic skill.
I fear that this is because of the seductive power of "sexy" explanations and topics. Furthermore, some of the students mentors and role models don't model or practise skepticism, particularly if their career success and funding [or hope thereof] depends on the exotica favoured by the luxury journals.

Good science is just plain hard work and not as exciting or clear cut as we might wish.


  1. I agree.While I explain to my student and postdoc that (me) playing devil's advocate will sharpen their ideas as well as mine (!), the hardest part for me is to see my student or postdoc actually question my ideas.
    This goes to show that personalities play a large role in this (and I recognize mine...), but the end result is that all involved in the conversation learn something - and that's the main goal after all.

    But indeed, I (being younger than you are) see quite a lot of "it's written therefore it's true", and that worries me as it easily result in wild goose chases that end up nowhere.

  2. In a lot of funding systems, there is a clear advantage for chasing exotica and sexy explanations, or even worse, giving new sexy sounding names or unnecessary categorization to things that already existed to make it sound new. The problem is when students mistake the funding game for the science (and at a higher level, that there is such a big difference between the two).

    My finding is if you talk to the authors of luxury papers with controversial or sexy explanations, that they will be the first to admit their own skepticism regarding their explanations of the data. But somehow this skepticism is not conferred to the text, because the luxury journals like clear, concise, authoritative explanations. Most of the details get hashed out later in less prominent, but longer form journals, and these are only followed closely by those within the specific community.

    I guess I'm saying that the students don't have the chance to learn skepticism from the literature, so I agree that it must come from the supervisors.

    Phil Anderson famously complained that later in his career he became less productive because people held him too high, and stopped questioning his crazy ideas, so he chased too many dead ends. (But he was still publishing, and being cited, so from the perspective of the bean counters, there's no real difference.)