Thursday, February 4, 2016

We should not give up on falsifiability

No, we should redouble our efforts!

Over past few years some scientists, particularly string theorists, have suggested that we should give up on the idea of falsifiability as a criterion for deciding whether or not to accept or reject a specific scientific theory. (A good theory is one that one can perform a specific experiment, whose outcome may lead to the rejection of the theory).
For example, in 2014 in answer to the question, "What scientific idea is due for retirement?" Sean Carroll's answer was Falsifiability. He uses this to justify string theory and the multiverse.

First, I think several important points need to be conceded and acknowledged.

1. There are subtle philosophical issues associated with falsifiability. Popper did not have the last word!

2. In practise, rarely will a theory get rejected just because there is experimental data that is inconsistent with it. Sometimes the data will get rejected. Other times the theory will get modified.

3. In practise, rarely do many scientists actually focus on falsifiability. For example, theorists generally don't write papers or give talks suggesting specific experiments that could be used to falsify their favourite theory or latest calculation.

I would argue that 3. happens partly because it is actually extremely difficult to come up with specific do-able experiments that will give definitive results that can clearly falsify a theory, particularly in condensed matter or theoretical chemistry. As I have said before, good science is hard work. 

But, the solution is not to give up on falsifiability. To me this is throwing the baby out with the bath water. It is a bit like discussions of foreign aid for poverty alleviation. Currently, some programs waste money and others actually cause more harm than good. But, the solution is not to give up but focus on supporting programs that actually do make a positive difference.

I think Einstein's General Relativity does provide a nice example of falsifiability, contrary to what Sean Carroll claims. This is because besides Einstein's theory there are many alternative theories: Newton, Brans-Dicke, Cartan, ...
We are celebrating the centenary of GR and not some alternative theory because they have largely been ruled out by experiment ....

I think that science would be better off if we all worked a little harder and thought a little more critically about how integrate falsifiability more into science.

What do you think?


  1. It seems to me that a close corollary of a theory not being falsifiable is that the same theory is not predictive. From the perspective of someone who is very involved in applied theoretical research, I can say that at least in this general area if a theorist develops a track record for only generating non-predictive work then their impact on the broader community will rapidly converge to zero.

  2. Yes!
    @#2: modifying a theory is fine I think - that amounts to falsifying part of the range of applicability of the theory, and providing an alternative model outside the limits of applicability of the early theory.

    Also Prof. Sholl is quite right as well in my opinion (though predictions on a lengthscale of 10^-34 m won't do either, so "predictive" needs some boundary conditions as well).

    I have learned that some flashy journals currently do not like to publish (condensed matter) work that experimentally falsify main lines of thought in a subject.

    Since this may sound like "the editor does not see the importance of my paper", I want to note that this was not my work (I was not a co-author). We learned this from a discussion of the lead-author with an(other) editor at the journal. This other editor did not handle the paper, but was a friend of the lead-author - hence we asked whether he could provide an explanation of the reasons for the decision. He explained that this is their policy: they want to see papers with "model A is falsified because experiments in this paper, and we propose model B". I.e. they don't want "negative" papers, they want to be able to announce that they published new insights etc.

    What I think is flawed in this is that sometimes (oftentimes?) there is no immediate model B available.
    Falsifying major lines of thought is one of the two ways science really moves forward, with the other one being able to provide alternative models. Being forced to always combine the two ways (falsify A, propose B) encourages speculation beyond what should be done.

    Again, I have no stake in this situation, and I tried to objectively describe what we learned. And what we learned about the way this journal acts is disappointing to me.

    1. pcs, Thanks for your comment and telling the story. I find it very disturbing and disappointing, but not totally surprising. I think the most interesting and important experimental discoveries are what we cannot explain. In fact, I find it a little boring if one can immediately cook up a new theory to explain an unexpected result. I also agree that speculation (beloved by the luxury journals) or even half-baked ideas does not help.

  3. I rather prefer an applied physics and engineering measure that I think is fairly well put in a sloganeering way by Ian Hacking's suggestion that “if you can spray them then they are real” (1983, Google for this quote finds a variety of support and critique for this). That comes significantly later than whether a theory has been falsified or not, and it resists falsification in the sense that a theory can continue to be useful even if its known domain of application is reduced, as for classical mechanics, electrodynamics, etc. This can be seen as having merit in being a relatively pragmatic agreement with the concepts presented by a theory rather than being an exaggerated and polemicizing realism.

  4. Dear Ross,

    I completely agree with you that give up on falsifiability is not the way.

    All the subtleties of falsifiability are related not to the concept itself, but to the difficulty in testing physical theories. The problem solved by Popper so long ago is a very simple one: a proposition whose time extension is not limited cannot be confirmed, it can only be disproved. If it is not disprovable (i.e., falsifiable) it does not make practical difference.

    The (social) act of working in science as a profession means that we sometimes are faced with hypothesis that are not possible to be falsified in practice for some technical limitation, but working with that is fine. The problem is to keep holding an hypothesis which has been proven to be non-falsifiable. In this case, there is no difference from religion or superstition.

    Of course, one can always talk about degrees of belief in a hypothesis, but even Bayes needs falsifiability to be completely sure (although that is a different discussion).

  5. I liquidated the first common for unstable ranting. I just wanted to compliment a lot of good thinking (and wise pitching) on your part. I only came here as a result of randomly clicking at a large blogroll on someone else's blog. Not sure how I came to be there.