Certain benefits of Bayes

Best wishes for the New Year! One thing I hope to achieve this year is an actual understanding of things "Bayesian".

I am particularly interested because it gives a way to be more quantitative and precise about some of the intuitions that I use in science. For example, I tend to be skeptical of new experimental results (often hyped) that claim to go against well-established theories, regardless of how good the "statistics" of the touted result.

In this vein, Phil Anderson argued that Bayesian methods should have been used to rule out the significance of "discoveries" such as the 10 keV neutrino and the fifth force. In 1992 he wrote a Physics Today column on the subject.

An interesting metric for mathematical formula is the ratio of profound and wide implications to the simplicity of the formula and its derivation. I suspect that Bayes' formula for conditional probabilities would win first place!

P(A|B) denotes the probability of A given B. 

The proof takes about two lines. If you multiply both sides of the equation about by P(B) the identity holds because both sides of the equation are just different ways of writing P(A and B).

My first attempt to understand the applications and implications of Bayes was reading the relevant sections in Phil Nelson's beautiful book, Physical Models of Living Systems. There is a helpful section entitled, "Bayes formula provides a consistent approach to upgrading our degree of belief in light of new data."

More recently, I found this wonderful and short video very helpful, as it clearly defines terms, uses graphical representations, and gives some concrete examples.

 

A Bayesian perspective highlights the importance of reporting negative results and is the basis of a seminal paper

Why Most Published Research Findings Are False by John P. A. Ioannidis

A measure of the profundity of Bayes is that the Stanford Encyclopedia of Philosophy has two articles on the topic

Bayes Theorem

Bayesian Epistemology

Niels Bohr on emergence

Until this week, I did not know that Bohr ever thought about emergence.

Ernst Mayr was one of the leading evolutionary biologists in the twentieth century and was influential in the development of the modern philosophy of biology. He particularly emphasised the importance of emergence and the limitations of reductionism. In the preface to his 1997 book, This is Biology: the Science of the Living World, Mayr recounts the development of his thinking about emergence.

At first I thought that this phenomenon of emergence, as it is now called, was restricted to the living world, and indeed, in a lecture I gave in the early 1950s in Copenhagen, I made the claim that emergence was the one of the diagnostic features of the of the organic world. The whole concept of emergence at the time was considered to be rather metaphysical. When Niels Bohr who was who was in the audience, stood up during the discussion, I was fully prepared for an annihilating refutation. However, much to my surprise, he did not at all object to the concept of emergence, but only to my notion that it provided a demarcation between the physical and the biological sciences. Citing the case of water whose "aquosity" could not be predicted from the characteristics of its two components, hydrogen and oxygen, Bohr stated that emergence is rampant in the inaminate world.  (page xii).

Later in the book Mayr pillars Bohr for his support of vitalism, including claims that vitalism has a "quantum" foundation.

Very Short Introductions Podcast on Condensed Matter Physics

The podcast episode where I talk about my book just came out.

It is available on a range of platforms, listed here, including SoundCloud and YouTube.

Emergence in nuclear physics

Nuclear physics exhibits many characteristics associated with emergent phenomena. These include a hierarchy of scales, effective interactions and theories, and universality.

The table below summarises how nuclear physics is concerned with phenomena that occur at a range of length and number scales. At each level of the hierarchy, there are effective interactions that are described by effective theories. Some of the biggest questions in the field concern how the effective theories that operate at each level are related to the levels above and below.

Moving from the bottom level to the second top level, relevant length scales increase from less than a femtometre to several femtometres.

The challenge in the 1950s was to reconcile the liquid drop model and the nuclear shell model. This led to the discovery of collective rotations and shape deformations. The observed small moments of inertia were explained by BCS theory. Integration of the liquid drop and shell models led to the award of the1975 Nobel Prize in Physics to Aage Bohr, Ben Mottelson, and Rainwater.

Since the 1980s a major challenge is to show how the strong nuclear force between two nucleons can be derived from Quantum Chromodynamics (QCD). The figure below illustrates how the attractive interaction between a neutron and a proton can be understood in terms of the creation and destruction of a down quark-antiquark pair. The figure is taken from here.

An outstanding problem concerns the equation of state for nuclear matter, such as found in neutron stars. A challenge is to learn more about this from the neutron star mergers that are detected in gravitational wave astronomy.

Characteristics of universality are also seen in nuclear physics. Landau’s Fermi liquid theory provides a basis for the nuclear shell model which starts from assuming that nucleons can be described in terms of weakly interacting quasiparticles moving in an average potential from the other nucleons. The BCS theory of superconductivity can be adapted to describe the pairing of nucleons, leading to energy differences between nuclei with odd and even numbers of nucleons. 

Universality is also evident in the statistical distribution of energy level spacings in heavy nuclei. They can be described by random matrix theory which makes no assumptions about the details of interactions between nucleons, only that the Hamiltonian matrix has unitary symmetry. Random matrix theory can also describe aspects of quantum chaos and zeros of the Riemann zeta function relevant to number theory.

Shape memory alloys

Recently I bought a small wire of NiTinol to have fun with and use in demonstrations to kids. This video gives a spectacular demonstration and attempts to explain how it works. I did not know about their use in stents for heart surgery.

I am still struggling to understand exactly how shape-memory alloys work. According to Wikipedia

The shape memory effect occurs because a temperature-induced phase transformation reverses deformation...Typically the martensitic (low-temperature) phase is monoclinic or orthorhombic . Since these crystal structures do not have enough slip systems for easy dislocation motion, they deform by twinning—or rather, detwinning.

Martensite is thermodynamically favored at lower temperatures, while austenite (B2 cubic) is thermodynamically favored at higher temperatures. Since these structures have different lattice sizes and symmetry, cooling austenite into martensite introduces internal strain energy in the martensitic phase. To reduce this energy, the martensitic phase forms many twins—this is called "self-accommodating twinning" and is the twinning version of geometrically necessary dislocations. 

In different words, I think the essential idea may be the following. In most metals large strains are accomodated by topological defects such as dislocations. These become entangled leading to work hardening and irreversible changes is macroscopic shapes. Shape memory alloys are different because of the low symmetry unit cell. The most natural defects are twinning domain walls and they are not topological and so their formation is reversible.

I am looking forward to reading the book chapter Shape memory alloys by Vladimir Buljak, Gianluca Ranzi

Another fascinating phenomena that is related to shape-memory is "superelasticity", which I discussed in an earlier post on organic molecular crystals, and has recently been reviewed.

I welcome clarification  of the essential physics.

An emergentist perspective on public policy issues that divide

How is the whole related to the parts?

Which type of economy will produce the best outcomes: laissez-faire or regulated?

Can a government end an economic recession by "stimulus" spending?  

What is the relative importance of individual agency and social structures in causing social problems such as poverty and racism?

These questions are all related to the first one. Let's look at it from an emergentist perspective, with reference to physics. 

Consider the Ising model in two or more dimensions. The presence of nearest neighbour interactions between spins leads to emergent properties: long-range ordering of the spins, spontaneous symmetry breaking below the critical temperature, and singularities in the temperature dependence of thermodynamic properties such as the specific heat and magnetic susceptibility. Individual uncoupled spins have neither property. Even a finite number of spins do not. (Although, a large number of spins do exhibit suggestive properties such as an enhancement of the magnetic susceptibility near the critical temperature). Thus, the whole system has properties that are qualitatively different from the parts. 

On the other hand, the properties of the parts, such as how strongly the spins couple to an external field and interact with their neighbours, influence the properties of the whole. Some details of the parts matter. Other details don't matter. Adding some interaction with spins beyond nearest neighbours does not change any of the qualitative properties, provided those longer-range interactions are not too large. On the other hand, changing from a two-dimensional rectangular lattice to a linear chain removes the ordered state. Changing to a triangular lattice with an antiferromagnetic nearest-neighbour interaction removes the ordering and there are multiple ground states. Thus, some microscopic details do matter.

For illustrative purposes, below I show a sketch of the temperature dependence of the magnetic susceptibility of the Ising model for three cases: non-interacting spins (J=0), two dimensions (d=2), and one dimension (d=1). This shows how interactions can significantly enhance/diminish the susceptibility depending on the parameter regime.

The main point of this example is to show that to understand a large complex system we have to keep both the parts and the whole in mind. In other words, we need both microscopic and macroscopic pictures. There are two horizons, the parts and the whole, the near and the far. There is a dialectic tension between these two horizons. It is not either/or but both/and.

I now illustrate how this type of tension matters in economics and sociology, and the implications for public policy. If you are (understandably) concerned about whether Ising models have anything to do with sociology and economics, see my earlier posts about these issues. The first post introduced discrete-choice models that are essentially Ising models. A second post discussed how these show how equilibrium may never be reached leading to the insight that local initiatives can "nucleate" desired outcomes. A third post, considered how heterogeneity can lead to qualitative changes including hysteresis so that the effectiveness of "nudges" can vary significantly.

A fundamental (and much debated) question in sociology is the relationship between individual agency and social structures. Which determines which? Do individuals make choices that then lead to particular social structures? Or do social structures constrain what choices individuals make. In sociology, this is referred to as the debate between voluntarism and determinism. A middle way, that does not preference agency or structure, is structuration, proposed by Anthony Giddens.

Social theorists who give primacy to social structures will naturally advocate solving social problems with large government schemes and policies that seek to change the structures. On the other side, those who give primacy to individual agency are sceptical of such approaches, and consider progress can only occur through individuals, and small units such as families and communities make better choices. The structure/agency divide naturally maps onto political divisions of left versus right, liberal versus conservative, and the extremes of communist and libertarian. An emergentist perspective is balanced, affirming the importance of both structure and agency.

Key concepts in economics are equilibrium, division of labour, price, and demand. These are the outcomes of many interacting agents (individuals, companies, institutions, and government). Economies tend to self-organise. This is the "invisible hand" of Adam Smith. Thus, emergence is one of the most important concepts in economics. 

A big question is how the equilibrium state and the values of the associated state variables (e.g., prices, demand, division of labour, and wealth distribution) emerge from the interactions of the agents. In other words, what is the relationship between microeconomics and macroeconomics?

What are the implications for public policy? What will lead to the best outcomes (usually assumed to be economic growth and prosperity for "all")? Central planning (or at least some government regulation) is pitted against laissez-faire. For reasons, similar to the Ising and sociology cases, an emergentist perspective is that the whole and the parts are inseparable. This is why there is no consensus on the answers to specific questions such as, can government stimulus spending move an economy out of a recession? Keynes claimed it could but the debate rages on.

An emergentist perspective tempers expectations about the impact of agency, both individuals and government. It is hard to predict how a complex system with emergent properties will respond to perturbations such as changes in government policy. This is the "law" of unintended consequences.

“The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.”

Friedrich A. Hayek, The Fatal Conceit: The Errors of Socialism

I think this cuts both ways. This is also reason to be skeptical about those (such as Hayek's disciples) who think they can "design" a better society by just letting the market run free.

Diversity is a common characteristic of emergent properties

Consider a system composed of many interacting parts. I take the defining characteristic of an emergent property is novelty. That is, the whole has a property not possessed by the parts alone. I argue that there are five other characteristics of emergent properties. These characteristics are common but they are neither necessary nor sufficient for novelty.

1. Discontinuities

2. Unpredictability

3. Universality

4. Irreducibility

5. Modification of parts and their relations

I now add another characteristic.

6. Diversity

Although a system may be composed of only a small number of different components and interactions, the large number of possible emergent states that the system can take is amazing. Every snowflake is different. Water is found in 18 distinct solid states. All proteins are composed of linear chains of 20 different amino acids. Yet in the human body there are more than 100,000 different proteins and all perform specific biochemical functions. We encounter an incredible diversity of human personalities, cultures, and languages. 

A related idea is that "simple models can describe complex behaviour". Here "complex" is often taken to mean diverse. Examples, how simple Ising models with a few competing interactions can describe a devil's staircase of states or the multitude of atomic orderings found in binary alloys.

Perhaps the most stunning case of diversity is life on earth. Billions of different plant and animal species are all an expression of different linear combinations of the four base pairs of DNA: A, G, T, and C.

One might argue that this diversity is just a result of combinatorics. For example, if one considers a chain of just ten amino acids there are 10^13 different possible linear sequences. But this does not mean that all these sequences will produce a functional protein, i.e., one that will fold rapidly (one the timescale of milliseconds) into a stable tertiary structure, and one that can perform a useful biochemical function.