At UQ we just had an interesting colloquium from Signe Riemer-Sorensen about Dark matter emission - seeing the invisible. Central to the talk was the data below. Focus on the red data around 3.6 keV.
This has stimulated more than 100 theory papers!
This reminds me of the faster than speed of light neutrinos and the 17 keV neutrino, 500 GeV particles seen by the Fermi gamma ray telescope, BICEP2 "evidence" for cosmic inflation, ....
The above data is discussed in detail here.
I don't want to just pick on my astrophysics and high energy physics colleagues as this happens in condensed matter and chemistry too... remember cold fusion... think about periodic reports of room temperature superconductors!
The painful reality is that cutting edge science is hard. One can be incredibly careful about noise, subtracting background signals, statistical analysis, sample preparation, .... but in the end there is Murphy's law .... things do go wrong .... and crap happens...
Skepticism and caution should always be the default reaction; all the more so the greater the possible significance or surprise of the "observed" result.
I believe in irreproducible results.
Update (14 December).
Clifford Taubes brought to my attention two relevant papers on the possible 3.5 keV line. The first paper rules out a dark matter origin of the line and even mentions Occam's razor. The second has a mundane alternative explanation of the line in terms of charge exchange between hydrogen gas and sulfur ions.