However, some theoretical papers do this too.
There are several reasons why authors may do this.
Laziness. It can be hard work and confusing to work out the actual physical units for some theoretical calculations.
Complexity. For experiments it can be extremely difficult to normalise and calibrate some detectors.
Uncertainty and embarrassment. Parameters such as detector efficiency, sample thickness,
geometric corrections, solution concentration can involve large uncertainties so the horizontal scale may be unknown by as much as an order of magnitude.
But I think these uncertainties should be reported because they present a challenge for improvement.
I realise that the measured "units" may be detector dependent and not particularly useful to experimenters in different labs. For example, the units may be "number of clicks per second in homemade photodetector 3 in Professor Smith's lab".
But I still think those details should be reported.
1. It belies the fundamental fact that any physical quantity does have actual units.
2. For experiments it means that the reported measurements are not reproducible.
i.e. they cannot be checked. The shape of the spectrum can be checked but not the magnitude.
3. This practice limits the comparison of theory and experiment.
The absolute intensity [spectral weight] of a spectrum will be predicted by theory. It is important to test this experimentally. Just because a theoretical calculation gives the correct spectrum does not mean it is correct.
Absolute units allow one to test sum rules.
[Aside: I have a prejudice/suspicion that matrix elements may be generally more theory sensitive than energies. For example, in quantum many-body theory it is possible to get a very accurate ground state energy with a variational wave function that has a small overlap with the true ground state.]
Another example is the temperature dependence of transport properties.
Sometimes resistivity is reported in arbitrary units or the resistance [rather than resistivity is reported].
Simple theories can sometimes get the temperature dependence of the properties such as resistivity and thermopower correct but the absolute magnitude can be off by orders of magnitude.
For a nice example of people working very hard to normalise spectra, test sum rules, and get physical insight, see the paper
Fractional spinon excitations in the quantum Heisenberg antiferromagnetic chain.
A less impressive example [that I was involved in] is in the paper Transition dipole strength of eumelanin.
So, do the hard yards. Don't use "arbitrary units".
If you referee a paper that does, request the authors to do better.
My lab always uses "arbitrary units" for the Y axis of X-ray absorption/dichroism plots. I believe it's meant to emphasize that the actual numbers are unimportant and that the information is contained within the relative changes.
ReplyDeleteHi Ted,
DeleteThanks for the comment.
It highlights my point and how entrenched an undesirable practice is.
I agree that relative changes may be the most important information. However, theory can/should predict the magnitude of an X-ray absorption/dichroism spectrum. Hence, I disagree that the "actual numbers are unimportant".
The other one that really bothers me is colour plots where blue is "low" and red is "high". A plot without a scale is somewhat equivalent to one without units I suppose, but then there's also the question of the subtracted background, whether it's a linear or log plot in whatever unit it is to begin with, and perhaps others.
ReplyDeleteArbitrary units may seem arbitrary to some, but may not be to others. I agree entirely with the point that even if there is some detail, "homemade photodetector 3" being a very nice example, then why not just include this info? Then there is no chance for confusion for someone unfamiliar with your lab/own practices and policies. I had a couple of frustrating bouts of wasted time in my PhD trying to reproduce other peoples' results, finding my plots had no such feature, only to discover that I had to zoom in 100,000X or so to see them.
(the splitting of the massive into three massless Dirac cones in bilayer graphene being one example that I can immediately remember)
DeleteI don't know. Let's say we are talking about a spectrum. You say don't offer undefendable ground. If the shape is defendable, but not the magnitude, should I actually be offering the magnitude at all?
ReplyDeleteIf reproducibility of a particular result (e.g. spectral magnitude) cannot be reasonably expected, should it be reported? Reporting it would be irrelevant and possibly harmful to reproduction (because it would suggest an irrelevant detail is relevant, leading a poor graduate student on a wild goose chase). I counterclaim that one should only report results for which reproducibility can be reasonably expected, because only then is it useful to benchmark an attempt at reproduction.
Thanks for your comment.
DeleteTo me, both your points highlight that more clarifying information should be provided by the authors.
If the magnitude is not defendable that should be stated.
If reproducibility cannot be reasonably expected that should be stated as well.
Nice post. I quite enjoy reading papers and graphs that look like the end of a war: messy with lots of details, error bars criss-crossing all over the place, and lots of spoils for others to pick up.
ReplyDeleteThis is why I ALWAYS include a Supplementary Document with the submission of a paper. All the additional stuff and minutae important for reproducibility go there; the main paper is for making a clear point.
ReplyDeleteI don't agree with your statements when it comes to experiment. Lets take angle resolved photoemission as an example. The measured number (clicks/pixel/ second) depends on too many unknowns to really allow for a reproducible absolute scale. Apart from all the actual experimental problems, there is no real theory of this highly complex process. If I would work really really hard I might publish a figure with a reliable, reproducible scale once every5-10 years, because this is what it would take. i would have to do many additional experiments to test hypothesis required to get a working theory of the particular material I wanted to study
ReplyDeleteI would have to do this for every new material, chemical composition etc etc. It is simply not a feasible thing to do. If you want to remain funded that is. I get measurement time two times one week per synchrotron I visit every year. It would take forever to publish a paper. And the question is really wether we will learn something truly important from all that effort.
ReplyDeleteSTM is the same thing: no decent theory to rule out intrinsic processes that are important but unavoidable ( what is the actual shape of the tip and how does it influence the spectra? That is a complete researchline by itself).
I do agree that it should be done when it can be done ( optical spectroscopy is good example where there is no excuse to publish something in arbitrary units).