Consider scientific productivity as a problem in economics. One has a limited amount of resources (time, money, energy, political capital) and one wants to maximise the scientific output. Here I want to stress that the real output is scientific understanding. This is not the same as numbers of papers, grants, citations, conferences, newspaper articles, ...
The limited amount of resources is relevant at all scales: individual, research group, fields of research, departments, institutions, funding agencies, ...
As time passes one needs to face the problem of diminishing returns with increased resources. Consider the following diverse set of situations.
Adding extra parameters to a theoretical model.
Continuing to work on developing a theory without advances.
Calculating higher order corrections to a theory in the hope of getting better agreement with experiment.
Applying for an extra grant.
Taking on another student.
In quantum chemistry using a larger basis set or a higher level of theory (i.e. more sophisticated treatment of correlations).
Developing new correlation exchange functionals for density functional theory (DFT).
Trying to improve an experimental technique.
Repeating measurements or calculations in the hope of finding errors.
When one starts out it is never clear that these efforts will bear fruit. Sometimes they do. Sometimes they don't. But inevitably, I think one has to face the law of diminishing returns.
These thoughts were stimulated by two events in the last week. One was reading Not Even Wrong: The failure of String Theory and the search for unity in physical law by Peter Woit. The second was being part of a workshop on superconductivity that featured many discussions about the high-Tc cuprate superconductors.
The book chronicles how in spite of thousands of papers over the past thirty years high energy theory has not really produced any ideas beyond the standard model that are relevant to experiment, or even a theory that is coherent.
I don't think the cuprates as a field is in such a dire straight. There are real experiments and concrete theoretical calculations. But it may be debatable whether we are gaining significant new insights. This is a hard problem on which we have made some real progress, but will we make more?
Even when one is making advances one needs to consider the useful economic concept of opportunity cost: if the resources were directed elsewhere would one produce greater scientific gains? This again applies at all scales, from personal to funding agencies.
So how does one decide to move on? When is it time to quit?
I think there is a highly subjective and personal element to deciding at what point one is at the point of diminishing returns.
On also needs to be careful because there are plenty of times in the history of science where individuals perservered for many years without progress, but eventually had a breakthrough.
e.g. Watson and Crick, John Kendrew and the first protein crystal structure, theory of superconductivity, ...
I welcome suggestions.
How do you decide when you are at the point of diminishing returns?
How do you decide when a research field or topic is at that point?