Saturday, June 23, 2018

The discipline of defining good research questions

I have a friend who works in a small college that offers Masters degrees in the humanities. In one program each student must do a thesis on a research topic over the course of a year. My friend spends a lot of time with the students, both individually and as a group, posing and refining a single question for each of their research projects. Last year while visiting I observed one of these sessions and also to have some discussions with individual students about their questions.

The stages are roughly this.

1. The student picks a specific research topic.
2. The student proposes a specific question about the topic that they will aim to answer.
3. The student meets with their advisor to refine the question. Often this involves making it more specific and narrow so that it is manageable.
4. The student presents their question to the class (often about five students) who then discuss it and try and refine it further.
5. With this feedback the student again refines it.
6. The student meets their advisor for a final discussion and agreement about the question.
7. The student starts research.

The questions can start with How, What, When, or Why?
Often, Why is preferred, because it may mean going deeper.

Several things struck me about this practise, particularly seeing it first hand.
First, how valuable it was in terms of ending up with questions that were more interesting, precise, valuable, and manageable.
Second, how valuable this was for the students in terms of learning to think more critically.
Third, how little I think we do this in science.

I don't think the key thing here is that it is a humanities practise. Rather, I think it is that the complete ethos of the college is teaching and training students.

My experience is that we tend to just pick topics for students and suggest they measure or calculate something and see what happens. We may mention a question but we don't refine it or keep coming back to it. Similar concerns apply to many grant applications. It is often not clear whether they are really aiming to provide definitive answers to any questions. I think that there are two big obstacles to us following this procedure: it is hard work and the "publish or perish" culture.

Some of this relates to the challenges of falsifiability and the method of multiple alternative hypotheses.

One (maybe) obvious caveat. Although one starts with this question, as the research proceeds, one may choose to or need to modify the question as one learns more.

What do you think?
Is this something we could be doing better?


    Stanford research shows how to improve students' critical thinking about scientific evidence
    Physicists at Stanford and the University of British Columbia have found that encouraging students to repeatedly make decisions about data collected during introductory lab courses improves their critical thinking skills.

    The whole study is available as paper
    This is good detailed study and very essential. There are repetitive papers in good journals which tires the reader and causes irritation.

  2. Education is not the filling of a pail, but rather the lighting of a fire.
    WB Yeats
    In an honest search for knowledge, you quite often have to abide by ignorance for an indefinite period.
    Erwin Schrodinger
    Both not filling of pail and ignorance for indefinite period by Yeats and Erwin Schrodinger probably refers to critical thinking. Is this possible in science when as you have written publish or/and perish your papers in the pail?. Group think as Sabine Hossenfelder points out in science prevents critical thinking.

  3. Science is not the humanities. "Why" in real science cannot mean what it does in the humanities. Even in "social science"
    at best it means "determine which is cause and which is effect". "Why" in real science generally means "find a simplying principle and from that derive a working mathematical model".

    In my department we do not use that series of steps, explicitly, for real research projects. But we certainly do for a class required of PhD students, in which they present their exposition of research from a literature paper or papers on a subject different from their thesis project. It works well.

    Lots of research projects start with a rather amorphous premise. My PhD advisors career was based on this. The idea was "What can we learn about chemical reaction dynamics using the techniques physicists use on particles i.e. scattering experiments". It was not more specific that that. The answer was, quite a lot.

  4. Science is not the humanities. "Why" in real science cannot mean what it does in the humanities.

    Critical thinking is truly restricted to humanities. The list of critical thinkers
    in humanities is huge. From Kant to Derrida via Foucault, the train of critical thinkers is there. BTW, the discovery of quasicrystals by Dan S and the objection by the great Pauling was one great saga of a critical observer Dan S winning over Pauling. Dan S followed his observation and not a chemist , but followed the first principle , The slowest step is the rate determining step and won!!. This is very rare in science , but it happened.

  5. How critical thinking text of E Schrodinger helped solve double helix problem? This abide by ignorance ( critical thinking time) quote of ES posted earlier in your blog is "what is life" did

    Ten years after Schrödinger's brilliant insight, Watson and Crick's second 1953 article on the structure of DNA provided the world with the key to the secret of life, casually employing the new concepts that had been created by cybernetics and propelling biology into the modern age with the words: "it therefore seems likely that the precise sequence of the bases is the code which carries the genetical information."

    'These prophetic words – shorn of the conditional opening phrase – are uttered in biology classes all over the world, every single day"

    Francis Crick with Watson acknowledging E Schrodinger in this letter for his " What is life " monograph. The letter is here below in the twitter handle of Royal Irish academy.

    Crick in the letter uses "aperiodate crystal" propounded by E Schrodinger as the apt one while posting the 1953 historic papers.

    Is there any book in the neoliberal scientific enterprise to match "what is life?