Statistical mechanics in just one equation

 This semester, I am giving four lectures in a third-year undergraduate course on statistical mechanics. Last year, I gave a guest lecture on the Ising model.

The first thing I want to emphasise is that the whole course is built around just one equation.

exp(−F(T, V)/kT) = ∑_s exp(−E_s/kT) ≡ Z(T, V)

This connects macroscopic thermodynamic properties (contained in the Helmholtz free energy F(T,V)) to microscopic properties (the energies E_s of all possible states s of the system).

[Note that from equilibrium thermodynamics, partial derivatives of F(T,V) give the entropy (and specific heat capacity) and the pressure (equation of state)].

I think that we are so used to this equation that we may miss just how amazing and profound it is.

First, the equation is incredibly simple.

Second, it is universal. It applies to any system in thermodynamic equilibrium regardless of its chemical or physical composition.

Third, in the context of the theory of emergent phenomena, it is exceptional because it provides a robust, tested way to connect the microscopic to the macroscopic. Biology, neuroscience, economics, sociology, and computer science have nothing like it.

Fourth, although the above three points are impressive, the basis of its validity remains an outstanding problem (a mystery?). One can "derive" it and "justify" it by drawing on assumptions such as the fundamental postulate ("in an isolated system all accessible microstates are equally probable"), the principle of maximum entropy, and the validity of equilibrium thermodynamics. But why are they true?

Although the equation is simple, implementing it, particularly for systems of interacting particles, is challenging. This challenge can be broken into five steps. One can get stuck on any one of the steps. People build whole careers on them.

1. For a system of interest, propose microscopic states of each particle and a Hamiltonian for the whole system.

2. Enumerate all possible microscopic states of the system.

3. Evaluate the energy of each of these microstates.

4. Perform the sum over all states in the partition function Z.

5. Take the thermodynamic limit where the system size becomes infinite.

A golden age for precision observational cosmology

Yin-Zhe Ma gave a nice physics colloquium at UQ last week, A Golden Age for Cosmology

I learnt a lot. Too often, colloquia are too specialised and technical for a general audience.

There are three pillars of experimental evidence for the Big Bang model: Hubble expansion of the universe, relative abundance of light nuclei due to nucleosynthesis in the first few minutes, and the Cosmic Microwave Background.

Ma showed Hubble's original data from 1929 for redshift versus distance of galaxies. There was a lot of noise in the data. Nevertheless, Hubble was right.

Big Bang Nucleosynthesis

This was first proposed in 1948 by Ralph Alpher and George Gamow. (Hans Bethe was an honorary author of the paper as a joke so that the author list would sound like the first three letters of the Greek alphabet. Gamow had a mischievous sense of humour.)

The chain of nuclear reactions that will produce the lightest elements and isotopes is shown below.

Because the binding energy of 4He is so large, it could have only been formed at an extremely high temperature of about 10^10 K. (Or is the issue activation energy for formation, not binding energy?)

Detailed calculations using parameters from terrestrial nuclear physics give the observed relative abundances of the elements. In particular, the universe is 74% hydrogen and 24 per cent helium.

The astrophysicist's periodic table showing the origin of the different chemical elements is rather cute.

Giving credit to George Gamow

Gamow, who died in 1968, made impressive contributions to theoretical physics. His Wikipedia page is worth reading. He claimed that he predicted the Cosmic Microwave Background in the late 1940s and did not receive sufficient credit when it was discovered in 1964. The 2019 Nobel Prize citation for James Peebles also minimises Gamow's early contributions. Whether this is fair or not can be debated.

Anisotropies in the Cosmic Microwave Background.

The past two decades have seen amazing advances in precision measurements of these anisotropies. The radiation is isotropic to one part in 25000, with a temperature of 2.72548±0.00057 K.

Measurements of the anisotropies have allowed precise determinations of key cosmological parameters by fitting theoretical predictions to the data shown below from the 2018 Planck collaboration. Different peaks have different physical origins. 

The level of precision in the data is truly amazing.

The solid line is a fit to theory involving six parameters. What would Enrico Fermi say? This is not "making the tale of an elephant wiggle" because the fit parameters are all consistent with independent determination of the cosmological parameters from Hubble expansion and the relative abundance of the light elements.

Aside. The paper from the Planck 2 collaboration has been cited 19000 times, but has almost 200 authors. How does one use that information in evaluating individual authors in job and promotion applications? How are they to be compared to a single-author paper with 100 citations or a five-author paper with 500 citations?

Is this a golden age for cosmology? 

Yes, in terms of precision measurements. 

On the theoretical side, the golden age may have passed. It is not clear that new concepts or theories will emerge. The outstanding questions are:

What is the nature and origin of dark matter? of dark energy? 

Why is the cosmological constant so small? Why is it so fine-tuned?

Can the validity of inflation be pinned down?

Does quantum gravity matter?

A lot of smart people have spent decades on these problems and made little progress. That fact does not preclude the possibility of a theoretical breakthrough. However, it does not make me optimistic. I hope I am wrong.

What is absolute temperature?

The concept and reality of absolute temperature is amazing. It tells us something fundamental about the universe, including physical limits as to what is possible. The existence of absolute temperature is intimately connected with the existence of entropy as a thermodynamic state function. It also hints at the underlying quantum nature of reality.

Aside: Unfortunately, the Wikipedia page on this topic is mediocre and garbled. For example, it continues the myth that temperature is related to kinetic energy.

The zeroth law of thermodynamics allows the definition of empirical temperature. It is an equilibrium state variable that indicates whether a thermodynamic system will remain in the same state upon being brought into thermal contact with another system. Thermometers are systems with a single state variable.

Absolute temperature is a specific temperature scale that is central to thermodynamics and statistical mechanics. 

There are several equivalent definitions of absolute temperature. They start at different points. Except for the first one, the others show that the existence of absolute temperature is intimately connected to the second law and to entropy being an extensive quantity.

This is nicely discussed by Zemansky in chapter 8 of his text Heat and Thermodynamics, Fifth Edition (1968). [This was the text for my second year undergrad thermo course at ANU in 1980. At the time, I did not fully appreciate how profound some of it is. I just enjoyed all the multivariable calculus.] 

1. Ideal gas thermometers.

Consider a fixed mass of ideal gas whose volume is fixed. An ideal gas is defined as any gas at a temperature and pressure much larger than the critical temperature and pressure for the gas-liquid transition. Suppose the system is cooled and heated, and the pressure is measured as a function of the temperature measured by a separate thermometer calibrated by the Celsius scale. The pressure versus temperature curve is a straight line. If this line is extrapolated to zero pressure, this occurs at -273.15 degrees Celsius. The straight line has different slopes for different gases, but they all intercept the x-axis at the same point. Alternatively, one can take the pressure as fixed and measure the volume of the gas versus temperature. Extrapolation to zero volume also occurs at -273.15 degrees. 

This suggests that something special is happening at -273.15 degrees Celsius. One can define a special temperature scale where this temperature is zero. Historically, this was the beginning of the concept of absolute temperature.

However, we should be cautious about this approach. This is just an extrapolation and does not allow for the fact that ideal gases are rather special or that some very different physics might kick in below the critical temperature of helium.

2. The efficiency of Carnot cycles. 

This follows Zemansky (page 208). Consider a Carnot cycle abcda, where b to c and d to a are isothermal processes, between the same two reversible adiabatic surfaces, and involve heat transfers Q and Q_3, respectively. The absolute temperature scale T is defined by 

T/T_3 = Q/Q_3

with T_3 = 273.16, when the process d to a occurs at the triple point of water.

3. Integrating factor for heat

Heat is not a state property. It depends on processes. The first law says Delta Q = Delta U + P Delta V. If we consider a quasi-static process and integrate the heat transfer along the path taken (in state space), the result may depend on the path taken. On the other hand, if one integrates dQ/T, one finds that the result is independent of the path. This can then be used to define a new state variable, the entropy. 

The brief discussion above misses some subtle and profound features that only became clear in the 1960s following the work of Pippard, Turner, Landsberg, and Sears, which was inspired by an axiomatic approach to thermodynamics developed by Caratheodory.

Zemansky states

It is an extraordinary circumstance that not only does an integrating factor exist for the dQ of any system, but this integrating factor is a function of temperature only and is the same function for all systems! This universal character enables us to define an absolute temperature.

4. Applying the second law to a composite system

This treatment follows Schroeder, Thermal Physics (Section 3.1)

Schroeder defines entropy in terms of a multiplicity of states. However, I prefer to define entropy as the state function which tells us whether or not two states are accessible from one another by an adiabatic process. There are multiple possible versions of this empirical entropy state function, but let's choose one that is extensive, i.e., scales with the mass and volume of the system.

Consider an adiabatically isolated system containing an internal partition through which the conduction of heat can occur. Denote the two parts of the system by A and B. The entropy of each part can be written as a function U of its internal energy. 

The total entropy of the system can be written 

S = S_A (U_A) + S_B (U_B)

If the system is in thermal equilibrium, by the second law, the entropy of the whole system must be a minimum as a function of U_A and U_B.

Now, dU_A = - dU_B as the composite system is adiabatically isolated. Hence, we have.

The left-hand (right-hand) side of the equation only depends on the properties of system A (B). Thus, it is an intensive state variable which determines whether the system will be in equilibrium with another system. Hence, by the zeroth law, it defines a temperature scale.

T is the absolute temperature.

What is temperature?

Temperature is NOT the average kinetic energy.

When I taught thermodynamics to second year undergraduates one of the preconceived notions that was hard to dislodge from students was that temperature IS a measure of the average kinetic energy of the atoms or molecules in a system.

First, I will give the merits of this view and then explain why it is problematic.

A profound and important insight from Maxwell's kinetic theory of ideal gases was that the average kinetic energy of the atoms/molecules in the gas is related to the absolute temperature defined by Kelvin. This result was important because it provided a microscopic basis for Joule's discovery of the mechanical equivalence of heat.

The result does not just hold for an ideal gas. Classical statistical mechanics can be used to show that for any system of interacting particles, the average kinetic energy of each particle is 3/2 kT. The proof proceeds in the same manner as the equipartition theorem. In the partition function, the integral over momentum factorises and can be evaluated exactly as it is Gaussian integral.

However, this simple relationship between temperature and kinetic energy does not hold for quantum systems. Consider the case of a harmonic oscillator, with frequency omega. By the virial theorem, the average kinetic energy is equal to the average potential energy. Thus, the average kinetic energy is half of the internal energy U(T), which is a universal function f(T/omega). Thus, if we compare two oscillators with different frequencies, at the same temperature, they will have different kinetic energies.

This problem is not just some quantum exotica that is only relevant at extremely low temperatures. Most solids are "quantum" at room temperature because they have a Debye temperature in the range of 200-1000 K.

Temperature is a macroscopic variable, not a microscopic one. It should be defined in terms of the zeroth law of thermodynamics.

Temperature is a state variable associated with a system in thermal equilibrium. It tells us whether that system will be in thermal equilibrium with another system. Consider two separated systems with temperatures T1 and T2. If they are brought into thermal contact, their states will not change if and only if T1=T2.

A thermometer is a system with a single state variable. The value of that variable is an empirical temperature.

Aside. This view of temperature was used by Planck in his book, Treatise on Thermodynamics, first published in 1905.

I am thankful to my undergraduate mentor, Hans Buchdahl for teaching me that thermodynamics is conceptually coherent and beautiful. 

This discussion illustrates that temperature is an emergent property. It is a property of a macroscopic system that the parts of the system do not have. The temperature is independent of the microscopic composition of the system or its history. This universality is a characteristic of many emergent properties.

In another post, I hope to explain what the absolute temperature, first introduced by Kelvin, is.

Maxwell's demon and the history of the second law of thermodynamics

I recently reread Warmth Disperses and Time Passes: The History of Heat by Hans Christian von Baeyer

As a popular book, it provides a beautiful and enthralling account of the discovery of the first and second laws of thermodynamics. The book is a great companion to teaching and learning thermodynamics and statistical mechanics. The narrative is unified by the puzzle of Maxwell's demon.

Aside: The book was first published in 1998 with the title Maxwell's Demon. My guess is that the publisher changed the title because most people have probably not heard of the demon, unlike Schrodinger's cat.

Baeyer captures both the wonder of the subject and the fascinating story of how the science of thermodynamics developed. He describes quirky personalities and illustrates how science proceeds with a mixture of brilliant insights, clever experiments, false leads, and forgotten discoveries. It is easy and compelling reading.

I appreciated that there is a lack of hype, in contrast to too many popular science books.

The book is enhanced by showing that the story is not over. Many reports of the demise of the demon have been premature. The penultimate chapter discusses Zurek's definition of entropy in terms of algorithmic randomness. The last chapter considers molecular motors, such as kinesin, which can be viewed as ratchets driven by thermal noise.

Physical insights

The first and second laws tell us something about the fundamental nature of the universe. Although they are macroscopic and may have some (debatable) microscopic justification,  they can be viewed as fundamental.

Central to the development of the first law was the notion of the mechanical equivalent of heat.

There are three rather different ways to formulate the second law: a Carnot cycle represents an engine of optimal efficiency, heat never passes from a cold to a hot body, and the arrow of time. It is profound that these formulations are equivalent and not something that was anticipated. We should marvel at this.

Entropy can be viewed as the absence of information. Consequently, the second law can be viewed as statistical.

Things I want to understand

A good book stimulates us to want to engage more with its subject. Some things I want to understand are the entropy of the initial state of the universe, Boltzmann's H theorem, Feynman's ratchet, Shannon's information theory, molecular motors, Zurek's definition of entropy, and Gerald Holton's book, Thematic origins of scientific thought.

A recent tutorial is A Friendly Guide to Exorcising Maxwell’s Demon, by A. de Oliveira Junior, Jonatan Bohr Brask, and Rafael Chaves

Beautiful things missed

As a popular book, I think the length and scope of topics are right. Nevertheless, in a longer book, here are some things I would enjoy reading about: the zeroth and third laws, the contributions of Gibbs, the ergodic hypothesis, Brownian motion and evidence for atoms, the role of thermodynamics (and statistical mechanics) in the development of quantum theory (blackbody radiation, Einstein solid, identical particle statistics, and the Sackur-Tetrode equation) and perhaps phase transitions.

Two quibbles

von Baeyer has a somewhat reductionist perspective that the true nature of thermodynamics was revealed by the microscopic descriptions of Maxwell and Boltzmann.

I will write separate posts on why I am not comfortable with the following two statements.

Temperature IS the average kinetic energy of molecules.

Entropy was mysterious until Boltzmann's definition S=k ln W. 

Elastic interactions and complex patterns in binary systems

One of the many beauties of condensed matter physics is that it can reveal and illuminate how two systems or phenomena that at first appear to be quite different actually involve similar physics. This is an example of universality: for emergent phenomena, many details don't really matter. One example is the similarities between superconductivity and superfluidity. A consequence of universality is that the same concepts, techniques, toy models, and effective theories can be used to describe a wide range of systems.

The complex organometallic molecules, known by the misnomer "spin crossover" compounds, exhibit a rich range of phase transitions and types of spatial order. Key aspects of the physics are the following.

• Each transition metal ion can be in one of two possible states: low-spin or high-spin. 
• The size of each molecular complex depends on the spin state.
• Consequently, the molecules interact with their neighbours via elastic interactions.

A toy model that can describe this is expanding balls connected by springs. Various versions of this type of model are reviewed here. The simplest version is the chain model below.

It turns out there are other classes of systems described by similar models. As far as I am aware, this was first pointed out in Consequences of Lattice Mismatch for Phase Equilibrium in Heterostructured Solids Layne B. Frechette, Christoph Dellago, Phillip L. Geissler

That paper is motivated by experiments on the growth of semiconductor quantum dots, by ion exchange, such as when CdSe is bathed in an Ag-rich solution and Ag2Se is produced with heterostructures (i.e., patterns of Ag and Se ions) that are different from the bulk crystal.

They consider the balls and springs model above on a triangular lattice.

They also point out how similar physics is relevant to binary metal alloys, e.g, AgCu, citing 

Ising model for phase separation in alloys with anisotropic elastic interaction—I. Theory, P. Fratzl and O. Penrose

Those authors consider a square lattice with elastic interactions associated with bond stretching along the edges and diagonals of the squares and bending of the square angles.

Frechette et al. also mention experiments on thin films of  DNA modified metallic nanoparticles. Compared to atomic systems these can tolerate larger lattice-mismatch before the formation of defects due to lattice strain.

Other systems (not mentioned) described by similar Ising models are metal-hydrogen systems, where the Ising pseudospin signifies whether a hydrogen atom is present at a particular site in the metallic crystal.

Frechette et al. start with the ball and springs model and "integrate out" the springs to obtain an effective Hamiltonian, which is an Ising model.

The spatial range of the interaction between Ising spins is shown in the colour-shaded plot below.

The interaction has two components.

One is an infinite range "ferromagnetic" part, seen as the light blue below.

The second is a short-range interaction which is mostly "antiferromagnetic" (i.e., red), but extends over several lattice sites. (Note, this interaction will be frustrated on the triangular lattice).

Using this toy model, Frechette et al. can obtain complex patterns (heterostructures) similar to those seen in quantum dots grown by ion exchange.

There is some subtle (and confusing) physics associated with deriving the Ising model from the ball and springs model. 

Due to the long-range nature of elastic interactions, the boundary conditions matter. 

The infinite range part of the Ising interaction arises from dealing with the lattice constant for the crystal, depending on the net "magnetisation" of the "spins". But that is a story for another day.

Undergraduates need to learn about the Ising model

A typical undergraduate course on statistical mechanics is arguably misleading because (unintentionally) it does not tell students several important things (related to one another).

Statistical mechanics is not just about how to calculate thermodynamic properties of a collection of non-interacting particles.

A hundred years ago, many physicists did not believe that statistical mechanics could describe phase transitions. Arguably, this lingering doubt only ended fifty years ago with Wilson's development of renormalisation group theory.

It is about emergence: how microscopic properties are related to macroscopic properties.

Leo Kadanoff commented, "Starting around 1925, a change occurred: With the work of Ising, statistical mechanics began to be used to describe the behaviour of many particles at once."

When I came to UQ 25 years ago, I taught PHYS3020 Statistical Mechanics a couple of times. To my shame, I never discussed the Ising model. There is a nice section on it in the course textbook, Thermal Physics: An Introduction, by Daniel Schroeder. I guess I did not think there was time to "fit it in" and back then, I did not appreciate how important the Ising model is. This was a mistake.

Things have changed for the better due to my colleagues Peter Jacobson and Karen Kheruntsyan. They now include one lecture on the model, and students complete a computational assignment in which they write a Monte Carlo code to simulate the model.

This year, I am giving the lecture on the model. Here are my slides  and what I will write on the whiteboard or document viewer in the lecture.

Confusing bottom-up and top-down approaches to emergence

Due to emergence, reality is stratified. This is reflected in the existence of semi-autonomous scientific disciplines and subdisciplines. A major goal is to understand the relationship between different strata. For example, how is chemistry related to physics? How is genetics related to cell biology?

Before describing two alternative approaches —top-down and bottom-up —I need to point out that in different fields, these terms are used in opposite senses. That can be confusing!

In the latest version of my review article on emergence, I employ the same terminology traditionally used in condensed matter physics, chemistry, and biology. It is also consistent with the use of the term “downward causation” in philosophy. 

Top-down means going from long-distance scales to short-distance scales, i.e., going down in the diagrams shown in the figure above. In contrast, in the quantum field theory of elementary particles and fields (high-energy physics), “top-down” means the opposite, i.e., going from short to long distance length scales. This is because practitioners in that field tend to draw diagrams with high energies at the top and low energies at the bottom.

Bottom-up approaches aim to answer the question: how do properties observed at the macroscale emerge from the microscopic properties of the system? 

History suggests that this question may often be best addressed by identifying the relevant mesoscale at which modularity is observed and connecting the micro- to the meso- and connecting the meso- to the macro. For example, high-energy degrees of freedom can be "integrated out" to give an effective theory for the low-energy degrees of freedom.

Top-down approaches try to surmise something about the microscopic from the macroscopic. This has a long and fruitful history, albeit probably with many false starts that we may not hear about, unless we live through them or read history books. Kepler's snowflakes are an early example. Before people were completely convinced of the existence of atoms, the study of crystal facets and of Brownian motion provided hints of the atomic structure of matter. Planck deduced the existence of the quantum from the thermodynamics of black-body radiation, i.e. from macroscopic properties. Arguably, the first definitive determination of Avogadro's number was from Perrin's experiments on Brownian motion, which involved mesoscopic measurements. Comparing classical statistical mechanics to bulk thermodynamic properties gave hints of an underlying quantum structure to reality. The Sackur-Tetrode equation for the entropy of an ideal gas hinted at the quantisation of phase space. The Gibbs paradox hinted that fundamental particles are indistinguishable. The third law of thermodynamics hints at quantum degeneracy. Pauling’s proposal for the structure of ice was based on macroscopic measurements of its residual entropy. Pasteur deduced the chirality of molecules from observations of the facets in crystals of tartaric acid. Sometimes a “top-down” approach means one that focuses on the meso-scale and ignores microscopic details.

The top-down and bottom-up approaches should not be seen as exclusive or competitive, but rather complementary. Their relative priority or feasibility depends on the system of interest and the amount of information and techniques available to an investigator. Coleman has discussed the interplay of emergence and reductionism in condensed matter. In biology, Mayr advocated a “dual level of analysis” for organisms. In social science, Schelling discussed the interplay of the behaviour of individuals and the properties of social aggregates. In a classic study of complex organisations in business, understanding this interplay was termed differentiation and integration.

I thank Jeremy Schmit for requesting clarification of this terminology.

The two-state model for spin crossover in organometallics

Previously, I discussed how spin-crossover is a misnomer for organometallic compounds and proposed that an effective Hamiltonian to describe the rich states and phase transitions is an Ising model in "magnetic field".

I introduce the two-state model that defines the model without the Ising interactions. To save me time on formatting in HTML, here is a pdf file that describes the model and what comparisons with experimental data (such as that below) tells us.

Future posts will consider how elastic interactions produce the Ising interaction and how frustrated interactions can produce multi-step transitions.

My review article on emergence

I just posted on the arXiv a long review article on emergence

Emergence: from physics to biology, sociology, and computer science

The abstract is below.

I welcome feedback. 

------

Many systems of interest to scientists involve a large number of interacting parts and the whole system can have properties that the individual parts do not. The system is qualitatively different to its parts. More is different. I take this novelty as the defining characteristic of an emergent property. Many other characteristics have been associated with emergence are reviewed, including universality, order, complexity, unpredictability, irreducibility, diversity, self-organisation, discontinuities, and singularities. However, it has not been established whether these characteristics are necessary or sufficient for novelty. A wide range of examples are given to show how emergent phenomena are ubiquitous across most sub-fields of physics and many areas of biology and social sciences. Emergence is central to many of the biggest scientific and societal challenges today. Emergence can be understood in terms of scales (energy, time, length, complexity) and the associated stratification of reality. At each stratum (level) there is a distinct ontology (properties, phenomena, processes, entities, and effective interactions) and epistemology (theories, concepts, models, and methods). This stratification of reality leads to semi-autonomous scientific disciplines and sub-disciplines. A common challenge is understanding the relationship between emergent properties observed at the macroscopic scale (the whole system) and what is known about the microscopic scale: the components and their interactions. A key and profound insight is to identify a relevant emergent mesoscopic scale (i.e., a scale intermediate between the macro- and micro- scales) at which new entities emerge and interact with one another weakly. In different words, modular structures may emerge at the mesoscale. Key theoretical methods are the development and study of effective theories and toy models. Effective theories describe phenomena at a particular scale and sometimes can be derived from more microscopic descriptions. Toy models involve minimal degrees of freedom, interactions, and parameters. Toy models are amenable to analytical and computational analysis and may reveal the minimal requirements for an emergent property to occur. The Ising model is an emblematic toy model that elucidates not just critical phenomena but also key characteristics of emergence. Many examples are given from condensed matter physics to illustrate the characteristics of emergence. A wide range of areas of physics are discussed, including chaotic dynamical systems, fluid dynamics, nuclear physics, and quantum gravity. The ubiquity of emergence in other fields is illustrated by neural networks, protein folding, and social segregation. An emergent perspective matters for scientific strategy, as it shapes questions, choice of research methodologies, priorities, and allocation of resources. Finally, the elusive goal of the design and control of emergent properties is considered.

Thermodynamics and emergence

Novelty. 

Temperature and entropy are emergent properties. Classically, they are defined by the zeroth and second laws of thermodynamics, respectively. The individual particles that make up a system in thermodynamic equilibrium do not have these properties. Kadanoff provided an example illustrating the qualitative difference between macro- and micro-perspectives. He pointed out how deterministic behaviour can emerge at the macroscale from stochastic behaviour at the microscale. The many individual molecules in a dilute gas can be viewed as undergoing stochastic motion. However, collectively they are described by an equation of state such as the ideal gas law.

 Primas gave a technical argument, involving C* algebras, that temperature is emergent: it belongs to an algebra of contextual observables but not to the algebra of intrinsic observables.44 Following this perspective, Bishop argued that temperature and the chemical potential are (contextually) emergent.

Intra-stratum closure. 

The laws of thermodynamics, the equations of thermodynamics (such as TdS = dU + pdV), and state functions such as S(U,V), provide a complete description of processes involving equilibrium states. A knowledge of microscopic details, such as the atomic constituents or forces of interaction, is not necessary for the description.

Irreducibility. 

A common view is that thermodynamics can be derived from statistical mechanics. However, this is contentious. David Deutsch claimed that the second law of thermodynamics is an “emergent law”: it cannot be derived from microscopic laws, like the principle of testability.

Lieb and Yngvason stated that the derivation from statistical mechanics of the law of entropy increase “is a goal that has so far eluded the deepest thinkers.”  In contrast, Weinberg claimed that Maxwell, Boltzmann, and Gibbs “showed that the principles of thermodynamics could in fact be deduced mathematically, by an analysis of the probabilities of different configurations… Nevertheless, even though thermodynamics has been explained in terms of particles and forces, it continues to deal with emergent concepts like temperature and entropy that lose all meaning on the level of individual particles.” (Dreams of A Final Theory, pages 40-41)

I agree that thermodynamic properties (e.g., equations of state, the temperature dependence of heat capacity, and phase transitions) can be deduced from statistical mechanics. However, thermodynamic principles, such as the second law, are not thermodynamic properties. Furthermore, these thermodynamic principles are required to justify the equations of statistical mechanics, such as the partition function, that are used to calculate thermodynamic properties. 

Macro hints of microscopics.

The Sackur-Tetrode equation for the entropy of an ideal gas hinted at the quantisation of phase space. The Gibbs paradox hinted that fundamental particles are indistinguishable. The third law of thermodynamics hints at quantum degeneracy.

Pattern formation and emergence

Patterns in space and/or time form in fluid dynamics (Rayleigh-Bénard convection and Taylor-Couette flow), laser physics, materials science (dendrites in the formation of solids from liquid melts), biology (morphogenesis), and chemistry (Belousov-Zhabotinsky reactions). External constraints, such as temperature gradients, drive most of these systems out of equilibrium. 

Novelty. 

The parts of the system can be viewed as the molecular constituents or small uniform parts of the system. In either case, the whole system has a property (a pattern) that the parts do not have.

Discontinuity. 

When some parameter becomes larger than a critical value, the system transitions from a uniform state to a non-uniform state. 

Universality. 

Similar patterns, such as convection rolls in fluids, can be observed in diverse systems regardless of the microscopic details of the fluid. Often, there is a single parameter, such as the Reynolds number, which involves a combination of fluid properties, that determines the type of patterns that form. Cross and Hohenberg highlighted how the models and mechanisms of pattern formation across physics, chemistry, and biology have similarities. Turing’s model for pattern formation in biology associated it with concentration gradients of reacting and diffusing molecules. However, Gierer and Meinhardt showed that it is sufficient to have a network with competition between short-range positive feedback and long-range negative feedback. This could occur in a circuit of cellular signals.

Self-organisation. 

The formation of a particular pattern occurs spontaneously, resulting from the interaction of the many components of the system.

Effective theories. 

A crystal growing from a liquid melt can form shapes such as dendrites. This process involves instabilities of the shape of the crystal-liquid interface. The interface dynamics are completely described by a few partial differential equations that can be derived from macroscopic laws of thermodynamics and heat conduction. A helpful review is by Langer. 

Diversity. 

Diverse patterns are observed, particularly in biological systems. In toy models, such as the Turing model, with just a few parameters, a diverse range of patterns, both in time and space, can be produced by varying the parameters. Many repeated iterations can lead to a diversity of structures. This may result from a sensitive dependence on initial conditions and history. For example, every snowflake is different because, as it falls, it passes through a slightly different environment, with small variations in temperature and humidity, compared to others.

Toy models. 

Turing proposed a model for morphogenesis in 1952 that involved two coupled reaction-diffusion equations. Homogeneous concentrations of the two chemicals become unstable when the difference between the two diffusion constants becomes sufficiently large. A two-dimensional version of the model can produce diverse patterns, many resembling those found in animals. However, after more than seventy years of extensive study, many developmental biologists remain sceptical of the relevance of the model, partly because it is not clear whether it has a microscopic basis. Kicheva et al., argue that “pattern formation is an emergent behaviour that results from the coordination of events occurring across molecular, cellular, and tissue scales.” 

Other toy models include Diffusion Limited Aggregation, due to Witten and Sander, and Barnsley’s iterated function system for fractals that produces a pattern like a fern.

Here is a beautiful lecture on Pattern Formation in Biology by Vijaykumar Krishnamurthy

 

Are gravity and space-time emergent?

Attempts to develop a quantum theory of gravity continue to falter and stagnate. Given this, it is worth considering approaches that start with what we know about gravity at the macroscale and investigate whether it provides any hints about some underlying more microscopic theory. One such approach was taken by Thanu Padmanabhan and is elegantly described and summarised in a book chapter.

Gravity and Spacetime: An Emergent Perspective

Insights about microphysics from macrophysics 

Padmanabhan emphasises Boltzmann's insight: "matter can only store and transfer heat because of internal degrees of freedom". In other words, if something has a temperature and entropy then it must have a microstructure.

The approach of trying to surmise something about microphysics from macrophysics has a long and fruitful history, albeit probably with many false starts that we do not hear about. Kepler's snowflakes may have been the first example. Before people were completely convinced about the existence of atoms, the study of crystal facets and of Brownian motion provided hints of the atomic structure of matter. Planck deduced the existence of the quantum from the thermodynamics of black-body radiation.

Arguably, the first definitive determination of Avogadro's number was from Perrin's experiments on Brownian motion which involved macroscopic measurements.

Comparing classical statistical mechanics to bulk thermodynamic properties gave hints of an underlying quantum structure to reality. The Sackur-Tetrode equation for the entropy of an ideal gas hints at the quantisation of phase space. The Gibbs paradox hints that fundamental particles are indistinguishable. The third law of thermodynamics hints at the idea of quantum degeneracy.

Puzzles in classical General Relativity

Padmanabhan reviews aspects of the theory that he considers some consider to be "algebraic accidents" but he suggests that they may be hints to something deeper. These include the role of boundary terms in variational principles and he suggests hint at a classical holography (bulk behaviour is determined by the boundary). He also argues that the metric of space-time should not be viewed as a field, contrary to most attempts to develop a quantum field theory for gravity.

Thermodynamics of horizons

The key idea that is exploited to find the microstructure is that can define a temperature and an entropy for null surfaces (event horizons). These have been calculated for specific systems (metrics) including the following:

For accelerating frames of reference (Rindler) there is an event horizon which exhibits Unruh radiation with a temperature that was calculated by Fulling, Davies and Unruh.

The black hole horizon in the Schwarschild metric has the temperature of Hawking radiation.

The cosmological horizon in deSitter space is associated with a temperature proportional to the Hubble constant H. [This was discussed in detail by Gibbons and Hawking in 1977].

Estimating Avogadro's number for space-time

Consider the number of degrees of freedom on the boundary, N_s, and in the bulk, N_b. 

On the boundary surface, there is one degree of freedom associated with every Planck area (L_p^2) where L_p is the Planck length, i.e,  N_s = A/ L_p^2, where A is the surface area, which is related to the entropy of the horizon (cf. Bekenstein and Hawking).

In the bulk equipartition of energy is assumed so the bulk energy E = N_b k T/2 and he presents an argument for the holographic principle that N_s = N_b.

An alternative perspective on cosmology 

He presents a novel derivation of the dynamic equations for the scale factor R(t) in the Friedmann-Robertson-Walker metric of the universe in General Relativity. His starting point is a simple argument leading to 

V is the Hubble volume, 4pi/3H^3, where H is the Hubble constant, and L_P is the Planck length.

The right-hand side is zero for the deSitter universe, which is predicted to be the asymptotic state of our current universe.

Possible insights about the cosmological constant

One of the biggest problems in theoretical physics is to explain why the cosmological constant has the value that it does.

There are two aspects to the problem.
1. The measured value is so small, 120 orders of magnitude smaller than what one estimates based on the quantum vacuum energy!

2. The measured value seems to be finely tuned (to 120 significant figures!) to the value of the mass energy.

He presents an argument that the cosmological constant is related to the Planck length 

where mu is of order unity.

Details of his proposed solution are also discussed here.

I am not technically qualified to comment on the possible validity or usefulness of Padmanabhan's perspective and results. However, I think it provides a nice example of a modest and conventional scientific alternative to radical approaches, such as the multiverse, or ideas that seem to be going nowhere such as AdS/CFT that are too often invoked or clung onto to address these big questions. 

Aside. In the same book, there is also a short and helpful chapter, Quantum Spacetime on loop quantum gravity by Carlo Rovelli. He explicitly identifies the "atoms" of space-time as the elements of "spin foam".

Is biology better at computing than supercomputers?

Stimulated by discussions about the physics of learning machines with Gerard Milburn, I have been wondering about biomolecular machines such as proteins that do the transcription and translation of DNA in protein synthesis. These are rather amazing machines.

I found an article which considers a problem that is simpler than learning, computation.

The thermodynamic efficiency of computations made in cells across the range of life

Christopher P. Kempes, David Wolpert, Zachary Cohen and Juan Pérez-Mercader

It considers the computation of translating a random set of 20 amino acids into a specific string for a specific protein. Actual thermodynamic values are compared to a generalised Landauer bound for computation. Below is the punchline. (page 9)

Given that the average protein length is about 325 amino acids for 20 unique amino acids, we have that p_i=p=1/20³²⁵=1.46×10⁻⁴²³, where there are 20³²⁵ states, such that the initial entropy is  , which gives the free energy change of kT(S_I−0)=4.03×10⁻¹⁸ (J) or 1.24×10⁻²⁰ (J per amino acid). This value provides a minimum for synthesizing a typical protein. 

We can also calculate the biological value from the fact that if four ATP equivalents are required to add one amino acid to the polymer chain with a standard free energy of 47.7 (kJ mol⁻¹) for ATP to ADP, then the efficiency is 1.03×10⁻¹⁶ (J) or 3.17×10⁻¹⁹ (J per amino acid).  

This value is about 26 times larger than the generalized Landauer bound.

These results illustrate that translation operates at an astonishingly high efficiency, even though it is still fairly far away from the Landauer bound. To put these results in context, it is interesting to note that the best supercomputers perform a bit operation at approximately 5.27×10⁻¹³ (J per bit). In other words, the cost of computation in supercomputers is about eight orders of magnitude worse than the Landauer bound of  (J) for a bit operation, which is about six orders of magnitude less efficient than biological translation when both are compared to the appropriate Landauer bound. Biology is beating our current engineered computational thermodynamic efficiencies by an astonishing degree.

Emergence and the stratification of physics into sub-fields

The concept of emergence is central to understanding sub-fields of physics and how they are related, and not related, to other sub-fields.

The table below shows a stratum of sub-disciplines of physics. For each strata there are a range of length, time, and energy scales that are relevant. There are distinct entities that are composed of the entities from lower strata. These composite entities interact with one another via effective interactions that arise due to the interactions present at lower strata and can be described by an effective theory. Each sub-discipline of physics is semi-autonomous. Collective phenomena associated with a single strata can be studied, described, and understood without reference to lower strata.

Table entries are not meant to be exhaustive but to illustrate how emergence is central to understanding sub-fields of physics and how they are related to one another.

What do you think of the table? Is it helpful? Have you seen something like this before?

I welcome suggestions about entries that I could add.

The incomplete veil: from macroscopic to the microscopic

 It is natural to assume that scientists need to probe a system at the microscopic scale to learn about what is happening at that scale. If we take this view we will necessarily be pessimistic about the "bottom-up" research strategy for quantum gravity advocated by Bei Lok Hu. It goes from macro- to micro-, the opposite to the more popular approaches of string theory and loop quantum gravity. However, the history of science shows that we can learn a lot about microscopics from probing systems at much greater length scales. Here are some examples.

Following Perrin's experiments and Einstein's theory of Brownian motion, almost all scientists believed that atoms were not just a mathematical convenience but did exist and were the basic constituents of liquids and solids. All this was before X-ray diffraction allowed the more direct study of crystals at the atomic scale.

Crystallography was pretty much settled as a field before there was any direct evidence of the atomic constituents and their spatial arrangement. Cleavage of crystals, facets observed in minerals, and group theory provided a complete classification of all possible crystal structures. Observations of crystal facets and different modes of sound can be sufficient to determine (or at least constrain options for) the crystal class. 

Figure from Traité de minéralogie (1801) by Rene Hauy See also this.

In 1935, Linus Pauling proposed the crystal structure of common ice without any information from X-ray crystallography. He only used the measured value of the residual entropy, simple models of hydrogen bonding, and the Bernal-Fowler ice rules.

In 1961, the biochemist Peter Mitchell deduced the mechanism of the synthesis of ATP, the molecule responsible for energy transport in cells, without knowing any details of the molecular structure of cell membranes. He reasoned from thermodynamics and the fact that there was an electric potential across the cell membranes. His work led to the discovery of the enzyme ATP synthase, a molecular motor. The underlying physics is beautifully described by Phil Nelson in his text, Biological Physics.

I see two important and related lessons for today from these historical examples.  

1. We have access to amazing computational power and microscopic probes. However, before rushing off to use them, ponder what constraints on the microscopic might be deduced from macroscopic observations.

2. Given that a quantum theory of gravity seems so elusive more resources might be invested in the macro- to micro- strategy.

Aside: Overall, I think this post is going against the strong claims that Bob Laughlin makes in "The Dark Side of Protection", chapter 12 in A Different Universe.

Philosophers of science on which theories are fundamental

What is real? What is true? These big questions are central to philosophy and issues in the philosophy of science.

Emergent properties of complex systems raise similar philosophical questions such as  "What is fundamental?" and "Are quasiparticles real?".

Robert Batterman is a philosopher of science who is the author of the book,

The Devil in the Details: Asymptotic Reasoning in Explanation, Reduction, and Emergence

In 2017 Batterman wrote an article in an edition of the Journal of Statitiscal Physics that was in memory of Leo Kadanoff. 

Philosophical Implications of Kadanoff’s Work on the Renormalization Group

Below I reproduce some of the text as it provides a helpful (and disturbing) summary of how the philosophy of science has evolved.

There are very few natural philosophers anymore. The fields of philosophy and science parted company at the end of [the nineteenth] century. Philosophers more and more began to turn toward the disciplines of logic and the analysis of language, and their examination of the enterprise of science began to follow a different, less-engaged-with-scientific-detail, direction. They began to try to determine the logic and structure of scientific theorizing in a way that was much more arm-chair and much less concerned with details about individual theories. The aim was to construct or reconstruct the proper logical structures of scientific explanation, confirmation, and theory choice. The philosophical reconstructions were, by and large, designed to fit all empirical science. For example, an explanation in physics should share the same general (logical) form as explanations in biology, chemistry, or sociology. 

I find this problematic because how physicists and biologists do science and the knowledge that they produce is quite different. In fact, similar differences exist between elementary particle physics and condensed matter physics. That also applies to Batterman's next claim.

I think it is fair to say that from a philosophy of science point of view, physical theories are supposed to reflect our best attempts to understand nature. Philosophers are also enamored with the idea that theories have a certain logical structure—they can be written down in some kind of axiomatic form from which, given certain inputs, various features of physical systems (future states, e.g.) can be derived using logic and reasonably straightforward mathematics.

Furthermore, philosophers often distinguish fundamental from nonfundamental (or “phenomenological”) theories. This latter distinction presupposes the idea that fundamental theories are the ones that tell us really what nature is actually like at “bottom.” These presumably include, quantum theory, quantum field theory, maybe a theory of quantum gravity, etc.

In contrast, Bob Laughlin, argues that certain emergent properties are exact [such as quantisation of magnetic flux in a superconductor, hydrodynamics, sound waves] and so they are more fundamental than microscopic theories. [A Different Universe, pp. 36-40].

Batterman continues

Nonfundamental theories such as thermodynamics, continuum mechanics, and fluid dynamics, on the other hand, while pragmatically useful, are in a certain sense (exactly what sense is a matter of serious contention) superfluous. We could, in principle, solve problems involving the elastic bending of beams by starting from the fundamental atomic and subatomic theories of the constituents of the beam.

Nonfundamental theories don’t get nature right. Steel beams are not really the continua whose bending behaviors are described by the Navier–Cauchy equations. Gases are not continuous blobs of stuff. The important theories, according to many philosophers and, I believe, according to many physicists, are those that get the ontology right. In part, the (often unarticulated) reason for preferring fundamental theories over phenomenological theories is a realist presupposition that physical theories must accurately describe the world the way the world really is. 

 Perhaps the view that atoms are real but solids are not is reflected by Bertrand Russell in the opening paragraph of his book, The ABC of Atoms, published in 1923 and intended for popular audiences.

Phenomenological theories are often good for calculating, but they don’t accurately describe the world and so must, in a sense, play second fiddle to their fundamental partners.

This is also contentious. Thermodynamics, elasticity theory, and fluid dynamics are perfectly accurate and never wrong within their domain of validity. Many courses and texts on thermodynamics begin with the following quote from Einstein.

 A theory is the more impressive the greater the simplicity of its premises, the more different kinds of things it relates, and the more extended its area of applicability. Therefore the deep impression that classical thermodynamics made upon me. It is the only physical theory of universal content which I am convinced will never be overthrown, within the framework of applicability of its basic concepts.

I should stress that Batterman is not agreeing with or promoting the views I have questioned above. Rather, he is trying to characterise what many philosophers believe.

Can emergent properties be explained?

An important question about emergent properties is whether they can be explained solely in terms of the properties of the components of the system. Here I explore the question from the point of view of Hempel's covering law of scientific explanation, discussed in my last post.

According to Hempel, a scientific explanation E of a specific phenomena P is a logical argument that starts with some premises, at least one of which is a scientific law L, and which logically implies P.

I now give a version of this that describes a microscopic scientific explanation of some emergent property.

Suppose that a macroscopic system S has property X. S is composed of many interacting microscopic components whose properties, including their interactions, have a finite enumeration x1, x2, x3,...xn. None of these properties is X. Hence, in the sense of novelty, X is an emergent property of S. Let l1, l2, l3,.., lm be a finite number of microscopic laws. Then X has a microscopic scientific explanation if it can be deduced from the x's and l's.

A possible problem with most microscopic "explanations" of emergent properties may be whether they at some point implicitly assume some "emergent" scientific law, such as spontaneous symmetry breaking, or the existence of X. Let me illustrate this possible problem with some examples.

Irreversibility. Microscopic laws are invariant under time-reversal. But macroscopic systems exhibit irreversible behaviour such as the mixing of two distinct fluids. This is encoded in the second law of thermodynamics. This problem of the "arrow of time" is nicely discussed by Tony Leggett in The Problems of Physics, in a chapter entitled "Skeletons in the Cupboard." An alternative perspective is that of Joel Lebowitz, who claims Boltzmann solved the problem.

Superconductivity. One could claim that BCS theory provides a microscopic explanation of superconductivity. We start with the properties of electrons, ions, Coulomb's law, quantum mechanics, and statistical mechanics. These properties and microscopic laws can be used to show that there is an effective attractive interaction between electrons. One then considers the BCS variational wavefunction and calculates the properties of the macroscopic system. They are consistent with experimental observations of superconductivity. It is explained!

However, there are several problems on the way, which all in some sense involve assuming that superconductivity does occur. First, investigating the variational wave function only shows that the superconducting state has lower energy than the normal metallic state. This does not prove it is the true ground state. In fact, in one dimension it is not.

But potentially more fatal to the claimed microscopic explanation is that it assumes that spontaneous symmetry breaking is allowed, including (in some subtle sense that people still argue about) the breaking of the gauge symmetry of electromagnetism. One of the major points that Phil Anderson was trying to make in More is Different is that spontaneous symmetry breaking is a law of nature that should be viewed as of similar status to microscopic laws such as Schrodinger's equation. 

Mean-field theory of antiferromagnetism. One might claim that one can start with a classical Heisenberg or Ising model, and classical statistical mechanics, crank the mathematical handle and get antiferromagnetic. If one does mean-field theory, then one is not really doing statistical mechanics as one is considering a weird ensemble and a Hamiltonian that is no longer microscopic. Suppose instead one does the exact solution of the Ising model. That can give the magnetic state and all the critical exponents. But, it is not clear to me that when one takes the thermodynamic limit, one assumes that the broken symmetry state is allowed. Similar questions arise for me if one does a computer simulation on large lattices and uses clever finite-size scaling techniques to deduce physical properties of the emergent state. Does the assumption of the validity of these techniques amount to some extra (macroscopic) law of nature?  

I wonder whether some of these issues would be clarified (or just muddied) by considering the Thermodynamic Formalism: The Mathematical Structure of Equilibrium Statistical Mechanics by David Ruelle. In particular, does he make clear how an equilibrium broken symmetry magnetic state is fundamentally different from the microscopic equilibrium state associated with a finite number of spins.

I welcome ideas on how to clarify these issues.