The emergence of hadronic matter from interacting quarks and gluons

 A characteristic of emergent phenomena is how novel and complex properties can emerge from apparently simple laws. Quantum ChromoDynamics (QCD) describes the interaction of quarks and gluons. The classical Lagrangian is remarkably simple.

It has an SU(3) local gauge symmetry and the A_v^mu are the associated gauge fields (gluons).

The quarks are fermions with fractional electrical charge. The only parameters in the theory are the coupling constant g_s, which describes the self-interaction of the gluons, and the bare masses of the quarks, m_f. The gluons are massless. In the limit where the bare mass of quarks vanishes, the Lagrangian has chiral symmetry, which transforms quarks with left-handed symmetry into right-handed.

Frank Wilczek states that QCD "is conceptually simple. Its realisation in nature, however, is usually very complex. But not always."

Hadronic matter (nucleons and mesons) has properties that are qualitatively different from its components (interacting quarks and gluons). In other words, it is emergent. In hadronic matter, there are no particles with fractional electrical charge or massless bosons. Chiral symmetry is broken. Consequently, hadrons do not come in pairs with opposite parity and equal energy. Quarks are confined and this is associated with a string tension. The order parameter associated with confinement is the Polyakov (or Wilson) loop. The connection between chiral symmetry breaking and confinement is subtle. For a long time they were thought to be intimately connected but now that is not the case.

Although the underlying Lagrangian is simple, the spectrum of hadrons and their interactions is complex. There is a "zoo" of particles. This all comes from a single coupling constant!

On the one hand, this complexity is surprising. On the other hand, it is similar to how there is a simple coupling constant (the electronic charge) in the Hamiltonian that describes most of chemistry and condensed matter physics. Most of the particles are unstable and can be viewed as quasiparticles as they have a finite lifetime, even in the absence of electroweak interactions. 

The emergent state of hadronic matter only exists at "low" temperatures and densities. It "melts" at the high temperatures associated with the Big Bang, relativistic heavy ion colliders, or the high densities associated with neutron stars. But that and the associated phase diagram of QCD is another story...

Update. I revised this post due to some helpful clarifications from Chris Allton, who was visiting UQ this week.

A new version of my review article on emergence

On the arXiv, I have posted a new version of my review article, Emergence: from physics to biology, sociology, and computer science.

I have added expanded sections on molecular structure, quantitative measures of causal emergence, and biological evolution.

There are also many minor additions and corrections. I hope the hyperlinked Table of Contents is helpful.

I welcome feedback and suggestions. I am sure there is much more to do.

Biological evolution and emergence

 The theory of evolution explains the origin of biological diversity and levels of similarity between species. A characteristic of emergence is that many iterations of a simple law (natural selection of the fittest to reproduce) can produce novel, diverse and rich structures. In biological evolution many generations in a population can produce new traits and species. 

Many of the most debated issues about evolution relate to the different characteristics of emergence and are briefly discussed below.

Scales

Central to emergence are the ideas of “many” and of scales. The former can take two forms: a system composed of many interacting components, or a system that undergoes many iterations according to a rule that is repeated many times. For evolution, both forms of “many” are relevant and have several dimensions. Evolution occurs in a population, i.e., a community of many members of a species living in a specific environment. Each member of the population has a specific genotype (many genes), which largely determines biological characteristics, from proteins to organs, defined as the phenotype. The environment also consists of many interacting species. Natural selection can act at multiple levels: on genes, cells, organisms, species, and groups of species.

Microscopic and macroscopic scales can also manifest in different ways. In terms of length, the micro- and macro- scales can be defined in terms of genotypes and phenotypes, respectively. In terms of time, microevolution and macroevolution roughly correspond to directly observable timescales and geological timescales, respectively. They are associated with the emergence of new traits within a species and new species, respectively.

Novelty

Development of new traits and species occurs over many generations, due to the repetition of the rule of natural selection.

Evolution theory uses concepts such as natural selection, survival of the fittest, niches, and hierarchical trees, that are not present in chemistry and physics. 

Connecting micro- and macro- properties

As for other systems, this is one of the great challenges of emergence. Genotypes and phenotypes are extremely well characterised. Genotype-phenotype maps seek to connect these micro- and macro- levels. A detailed understanding of how microevolution leads to macroevolution is a challenge.

Discontinuities

In microevolution, new traits occur within a species due to (continuous) adaptation to the environment. In contrast, in macroevolution, new organs and species can occur suddenly (at least on geological timescales). An example is the Cambrian explosion of new life forms. Extinctions can also represent discontinuities.

Evolution of a population occurs in response to changes in an environment. New traits, new species, and extinctions can be viewed as qualitative changes due to quantitative changes. For example, small changes in the oxygen concentration in the atmosphere is one (among many) hypotheses for the cause of the Cambrian explosion.

Using techniques from statistical physics, the transition of a species from survival to extinction can be viewed as a non-equilibrium phase transition to an absorbing state. The order parameter is the population and a toy model is directed population.1

Diversity with limitations

All species are based on the same biochemistry of DNA and proteins. Yet from these same building blocks there is an incredible diversity: more than 8 million distinct species, including more than 10,000 species of birds and more than 15,000 species of ants. Darwin said nature produces “endless forms most beautiful.”

But there are limitations. For example, the number of species with more than one head, brain, heart, or liver is limited. There are many more genotypes than phenotypes. 

The dominant view is that evolution is driven by random genetic mutations. Debates have arisen about how much evolution is limited (constrained) by morphology and environment.

Ball stated: (p. 332)

“convergent evolution is often regarded as a sign that certain shapes or structures are ideal adaptations to particular environments for physical reasons: wings consisting of flat, thin membranes are best for flying, torpedo-shaped bodies a streamlined for efficient swimming, and so on… There is a tendency in evolutionary biology to regard natural selection as a process with an infinite palette: anything is possible so long as it doesn't break the laws of physics. But the laws of physics might impose more constraint than that, precisely because biology uses rather than merely suffers them.”

Universality

Not all mutations produce a change in phenotype. There are neutral mutations. There are many more genotypes than phenotypes. In other words, genotype-phenotype maps are many-to-one.

Species that are unrelated or distantly related (in the tree of life) sometimes have traits or behaviours that are similar. Convergent evolution is the hypothesis that natural selection produced the same outcome in a different context. 

Modularity at the mesoscale

The economist Simon pointed out that evolution can occur on much faster time scales than might be expected because of modularity. According to Clune et al.

“A long-standing, open question in biology is how populations are capable of rapidly adapting to novel environments, a trait called evolvability [1]. A major contributor to evolvability is the fact that many biological entities are modular, especially the many biological processes and structures that can be modelled as networks, such as metabolic pathways, gene regulation, protein interactions and animal brains [1–7].”

Ball highlighted how domains in proteins provide functional modules that evolution uses: (pp. 174-5)

“the evolution of metazoan proteins is not so much a slow affair of letting random genetic mutations change one amino acid for another and seeing what effect it produces. Rather, it constitutes a reshuffling of already functional modules to produce multidomain molecules with new potential - a strategy much more likely to yield successful results…  the “unit” of molecular evolution here is not really the base pair of DNA or the amino acid or protein, or the gene itself, by the peers at a scale intermediate between the two: the module of a domain. It seems that this shuffling, rather than the slow mutation of primary base sequences, is what has driven the evolution of animals.”

Johnston et al. considered an algorithmic picture of evolution that 

“suggests that symmetric structures preferentially arise not just due to natural selection but also because they require less specific information to encode and are therefore much more likely to appear as phenotypic variation through random mutations… many genotype–phenotype maps are exponentially biased toward phenotypes with low descriptional complexity. A preference for symmetry is a special case of this bias… Lower descriptional complexity also correlates with higher mutational robustness, which may aid the evolution of complex modular assemblies of multiple components.”

Self-organisation

Complex biological structures, from proteins to organisms, have formed spontaneously due to evolution over millions of years. Their intricacy and functionality have led to claims of purpose and design. However, this is argued to be an “apparent” design, just like an economy whose self-organisation appears “as if” it is guided by an “invisible hand.”

Kauffman claimed that self-organisation is as important as natural selection in driving evolution.

Unpredictability

A contested question about evolution is the role of contingency (historical accidents) and whether the evolution of complex life forms, particularly humans, was an accident of history or inevitable.

Irreducibility

Until recently, evolutionary biology has been dominated by a reductionist gene-centric view, popularised by Dawkins. However, recent discussions about systems biology, evo-devo, and epigenetics have questioned this view. Some characterise these alternative views as a form of structuralism.

Complexity

An algorithmic picture of evolution suggests that simplicity spontaneously emerges as many genotype-phenotype maps may be biased towards phenotypes with low descriptional complexity. 

Toy models

An earlier post discussed the key role that toy models, such as “bean bag” genetics, have played in evolutionary theory.

Cross-fertilisation of fields

Ideas from evolution have stimulated the development of genetic algorithms in computer science.

Drossel has reviewed connections between evolution and statistical physics, including a wide range of toy models. Examples include spin glass models that give rise to rugged landscapes for fitness and can describe hierarchical structures, comparable to Darwin’s tree of life. Goldenfeld and Woese argued that evolution can be viewed as a collective phenomenon far from equilibrium. The toy model central to their discussion is directed percolation.

I welcome comments. My knowledge of biology is limited, and scientifically some the ideas above can be contentious. (Never mind philosophy, politics, or theology!)

Information theoretic measures for emergence and causality

The relationship between emergence and causation is contentious, with a long history. Most discussions are qualitative. Presented with a new system, how does one identify the microscopic and macroscopic scales that may be most useful for understanding and describing the system? Can Judea Pearl’s seminal ideas about causality be implemented practically for understanding emergence?

Broadly speaking, a weakness of discussions of emergence and causality is that it is hard to define these concepts in a rigorous and quantitative manner that makes them amenable to empirical testing, with respect to theoretical models and to experimental data. 

Fortunately, in the past decade, there have been some specific proposals to address this issue, mostly using information theory. A helpful recent review is by Yuan et al. 

“Two primary challenges take precedence in understanding emergence from a causal perspective. The first is establishing a quantitative definition of emergence, whereas the second involves identifying emergent behaviors or phenomena through data analysis.

To address the first challenge, two prominent quantitative theories of emergence have emerged in the past decade. The first is Erik Hoel et al.’s theory of causal emergence [19] whereas the second is Fernando E. Rosas et al.’s theory of emergence based on partial information decomposition [24].

Hoel et al.’s theory of causal emergence specifically addresses complex systems that are modeled using Markov chains. It employs the concept of effective information (EI) to quantify the extent of causal influence within Markov chains and enables comparisons of EI values across different scales [19,25]. Causal emergence is defined by the difference in the EI values between the macro-level and micro-level."

One perspective on causal emergence is that it occurs when the dynamics of a system at the macro-level is described more efficiently by macro-variables than by the dynamics of variables from the micro-level.

Klein et al. used Hoel’s information-theoretic measures of causal emergence to analyse protein interaction networks (interactomes) in over 1800 species, containing more than eight million protein–protein interactions, across different scales. They showed the emergence of ‘macroscales’ that are associated with lower noise and uncertainty. The nodes in the macroscale description of the network are more resilient than those in less coarse-grained descriptions. Greater causal emergence (i.e., a stronger macroscale description) was generally seen in multicellular organisms compared to single-cell organisms. The authors quantified causal emergence in terms of mutual information (between large and small scales) and effective information (a measure of the certainty in the connectivity of a network). Philip Ball (2023) (pages 218-220) gives an account of this work in terms of the emergence of multicellularity in biological evolution. He introduced the term causal spreading (pages 225-7), arguing that over the history of evolution the locus of causation has changed.

Yuan et al. continue

"However, in Hoel’s theory of causal emergence, it is essential to establish a coarse-graining strategy beforehand. Alternatively, the strategy can be derived by maximizing the effective information (EI) [19]. However, this task becomes challenging for large-scale systems due to the computational complexity involved. To address these problems, Rosas et al. introduced a new quantitative definition of causal emergence [24] that does not depend on coarse-graining methods, drawing from partial information decomposition (PID)-related theory. PID is an approach developed by Williams et al., which seeks to decompose the mutual information between a target and source variables into non-overlapping information atoms: unique, redundant, and synergistic information [29]…"

The Figure below is taken from Rosas et al. Xt^j (j=1,…,n) are microscopic variables that define a Markov chain. Vt is a macroscopic variable that is completely determined by the microscopic variables.

“Diagram of causally emergent relationships. Causally emergent features have predictive power beyond individual components. Downward causation takes place when that predictive power refers to individual elements; causal decoupling when it refers to itself or other high-order features.”

Rosas et al. applied the method to specific systems, including Conway’s Game of Life, Reynolds’ flocking model, and neural activity as measured by electrocorticography. More recently, it was used to describe emergence in computer science, including the identification of modular structures. Calculations were performed for specific examples, including Ehrenfest’s urn model for diffusion, the Ising model with Glauber dynamics, a Hopfield neural network model for associative memory.

Yuan et al. also state the following:

"The second challenge pertains to the identification of emergence from data. In an effort to address this issue, Rosas et al. derived a numerical method [24]. However, it is important to acknowledge that this method offers only a sufficient condition for emergence and is an approximate approach. Another limitation is that a coarse-grained macro-state variable should be given beforehand to apply this method."

Sas et al. recently stated

“Empirical applications of this framework to study emergence … including the study of gene regulatory networks [22], the dynamics of the human brain [23], the internal dynamics of reservoir computing [24], and the formation of useful internal representations in machine learning [25].”

Yuan et al. also discuss two significant connections between causal emergence and machine learning. First, machine learning can be used to improve calculations of causal emergence. Second, causal emergence measures can be used to better understand how machine learning works and improve it.

The work described above built on earlier work by Crutchfield, who claimed that the identification of emergence and hierarchies could be made operational, stating that “different scales are delineated by a succession of divergences in statistical complexity at lower levels.” More recently, Rupe and Crutchfield have reported progress towards identifying emergent self-organisation in a system.

Although this work on quantitative measures of emergence based on information theory represents significant progress, there are many open problems. Examples include the extension to non-Markovian systems and the development of computationally feasible methods for large systems. The latter is particularly important in physical systems where spontaneous symmetry breaking occurs, as this only happens in the thermodynamic limit of an infinite system.

There is an unrecognised similarity between the work described above and techniques recently developed to characterise phase transitions in statistical mechanics models such as the Ising model and classical dimer models. Coarse-graining (CG) is optimised by maximising the Real-Space Mutual Information (RSMI) between a spatial block and its distant environment. 

In general, maximising mutual information is notoriously hard but can be done using state-of-the-art machine learning algorithms. Gokmen et al. have developed an algorithm that they claim “can, unsupervised, construct order parameters, locate phase transitions, and identify spatial correlations and symmetries for complex and large-dimensional real-space data.” Furthermore, the optimal CG explicitly identifies the scaling operators associated with the critical point. 

The classical dimer model provides a stringent test as “the relevant low-energy degrees of freedom are profoundly different from the microscopic building blocks of the theory and change qualitatively throughout the phase diagram.” In other words, the emergent entities (quasiparticles such as vortices associated with the height field, which is described by a sine-Gordon field theory) are different from the dimers.

It is encouraging to see that two different scientific communities have developed similar ideas to address this challenging problem of making discussions about emergence and causality more concrete and quantitative.

Reviewing emergent computational abilities in Large Language Models

Two years ago, I wrote a post about a paper by Wei et al, Emergent Abilities of Large Language Models

Then last year, I posted about a paper Are Emergent Abilities of Large Language Models a Mirage? that criticised the first paper.

There is more to the story. The first paper has now been cited over 3,600 times. There is a helpful review of the state of the field.

Emergent Abilities in Large Language Models: A Survey

Leonardo Berti, Flavio Giorgi, Gjergji Kasneci

It begins with a discussion of what emergence is, quoting from Phil Anderson's More is Different article [which emphasised how new properties may appear when a system becomes large] and John Hopfield's Neural networks and physical systems with emergent collective computational abilities, which was the basis of his recent Nobel Prize. Hopfield stated

"Computational properties of use to biological organisms or the construction of computers can emerge as collective properties of systems having a large number of simple equivalent components (or neurons)."

Berti et al. observe, "Fast forward to the LLM era, notice how Hopfield's observations encompass all the computational tasks that LLMs can perform."

They discuss emergent abilities as in-context learning, defined as the "capability to generalise from a few examples to new tasks and concepts on which they have not been directly trained."

Here, I put this review in the broader context of the role of emergence in other areas of science.

Scales. 

Simple scales that describe how large an LLM is include the amount of computation, the number of model parameters, and the size of the training dataset. More complicated measures of scale include the number of layers in a deep neural network and the complexity of the training tasks.

Berti et al. note that the emergence of new computational abilities does not just follow from increases in the simple scales but can be tied to the training process. I note that this subtlety is consistent with experience in biology. Simple scales would be the length of an amino acid chain in a protein or base pairs in a DNA molecule, the number of proteins in a cell or the number of cells in an organism. More subtle scales include the number of protein interactions in a proteome or gene networks in a cell. Deducing what the relevant scales are is non-trivial. Furthermore, as emphasised by Denis Noble and Robert Bishop, context matters, e.g., a protein may only have a specific function if it is located in a specific cell.

Novelty. 

When they become sufficiently "large", LLMs have computational abilities that they were not explicitly designed for and that "small" versions do not have. 

The emergent abilities range "from advanced reasoning and in-context learning to coding and problem-solving."

The original paper by Wei et al. listed 137 emergent abilities in an Appendix!

Berti et al. give another example.

"Chen et al. [15] introduced a novel framework called AgentVerse, designed to enable and study collaboration among multiple AI agents. Through these interactions, the framework reveals emergent behaviors such as spontaneous cooperation, competition, negotiation, and the development of innovative strategies that were not explicitly programmed."

An alternative to defining novelty in terms of a comparison of the whole to the parts is to compare properties of the whole to those of a random configuration of the system. The performance of some LLMs is near-random (e.g., random guessing) until a critical threshold is reached (e.g., in size) when the emergent ability appears.

Discontinuities.

Are there quantitative objective measures that can be used to identify the emergence of a new computational ability? Researchers are struggling to find agreed-upon metrics that show clear discontinuities. That was the essential point of Are Emergent Abilities of Large Language Models a Mirage? 

In condensed matter physics, the emergence of a new state of matter is (usually) associated with symmetry breaking and an order parameter. Figuring out what the relevant broken symmetry and the order parameter often requires brilliant insight and may even lead to a Nobel Prize (Neel, Josephson, Ginzburg, Leggett,...) A similar argument can be made with respect to the development of the Standard Model of elementary particles and gauge fields. Furthermore, the discontinuities only exist in the thermodynamic limit (i.e., in the limit of an infinite system), and there are many subtleties associated with how the data from finite-size computer simulations should be plotted to show that the system really does exhibit a phase transition.

Unpredictability.

The observation of new computational abilities in LLMs was unanticipated and surprised many people, including the designers of the specific LLMs involved. This is similar to what happens in condensed matter physics, where new states of matter have mostly been discovered by serendipity.

Some authors seem surprised that it is difficult to predict emergent abilities. "While early scaling laws provided some insight, they often fail to anticipate discontinuous leaps in performance."

Given the largely "black box" nature of LLMs, I don't find it the unpredictability surprising. It is hard for condensed matter systems, and they are much better characterised and understood.

Modular structures at the mesoscale.

Modularity is a common characteristic of emergence. In a wide range of systems, from physics to biology to economics, a key step in the development of the theory of a specific emergent phenomenon has been the identification of a mesoscale (intermediate between the micro- and macro-scales) at which modular structures emerge. These modules interact weakly with one another, and the whole system can be understood in these terms. Identification of these structures and the effective theories describing them has usually required brilliant insight. An example is the concepts of quasiparticles in quantum many-body physics, pioneered by Landau.

Berti et al. do not mention the importance of this issue. However, they do mention that "functional modules emerge naturally during training" [Ref. 7,43,81,84] and that "specialised circuits activate at certain scaling thresholds [24]".

Modularity may be related to an earlier post, Why do deep learning algorithms work so well? In the training process, a neural network rids noisy input data of extraneous details...There is a connection between the deep learning algorithm, known as the "deep belief net" of Geoffrey Hinton, and renormalisation group methods (which can be key to identifying modularity and effective interactions).

Is emergence good or bad?

Undesirable and dangerous capabilities can emerge. Those observed include deception, manipulation, exploitation, and sycophancy.

These concerns parallel discussions in economics. Libertarians, the Austrian school, and Federich Hayek tend to see the emergence as only producing socially desirable outcomes, such as the efficiency of free markets [the invisible hand of Adam Smith]. However, emergence also produces bubbles and crashes and recessions.

Resistance to control

A holy grail is the design, manipulation, and control of emergent properties. This ambitious goal is promoted in materials science, medicine, engineering, economics, public policy, business management, and social activism. However, it largely remains elusive, arguably due to the complexity and unpredictability of the systems of interest. Emergent properties of LLMs may turn out to offer similar hopes, frustrations, and disappointments. We should try, but have realistic expectations.

Toy models.

This is not discussed in the review. As I have argued before, a key to understanding a specific emergent phenomenon is the development of toy models that illustrate the phenomenon and the possible essential ingredients for it to occur. The following paper may be a step in that direction.

An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem

Yoonsoo Nam, Nayara Fonseca, Seok Hyeong Lee, Chris Mingard, Ard A. Louis

In a similar vein, another possibly relevant paper is the review

Statistical Mechanics of Deep Learning

Yasaman Bahri, Jonathan Kadmon, Jeffrey Pennington1, Sam S. Schoenholz, Jascha Sohl-Dickstein and Surya Ganguli

They considered a toy model for the error landscape for a neural network, and show that the error function for a deep neural net of depth D corresponds to the energy function for a D-spin spherical spin glass. [Section 3.2 in their paper].

Weather, chaos, and emergence

Weather involves many scales of distance, time, and energy. Describing weather means making decisions about what range of scales to focus on. 

BTW. Did you know that there is a cyclone heading for Brisbane right now!

Now, back to physics :)

Key physics involves thermal convection which reflects an interplay of gravity, thermal expansion, viscosity and thermal conduction. This can lead to Rayleigh-Bénard convection and convection cells.

The multiple scales are associated with multiple entities:

-the molecules that make up the fluid

-small volumes of fluid that are in local thermodynamic equilibrium with a well-defined temperature, density, and velocity

-individual convection cells (rolls)

-collections of cells.

At each scale, the corresponding entities can be viewed as emerging from the interacting entities at the next smallest scale. Hence, they are collective degrees of freedom.

In principle, a complete description, including the transition to turbulence, is given by the equations of fluid dynamics, including the Navier-Stokes equation. Despite the apparent simplicity of these equations, making definitive predictions from them remains elusive.

A famous toy model was studied by the meteorologist Edward Lorenz in 1963, in a seminal paper, "Deterministic Nonperiodic Flow." Under the restrictive conditions of considering the dynamics of a single convection roll the model can be derived from the full hydrodynamic equations.

Lorenz's study stimulated the field of chaos theory, and is beautifully described in James Gleick's book Chaos: The Making of a New Science.

Here, I discuss Lorenz's model in the context of emergence.

 

The model consists of (just) three coupled ODEs (ordinary differential equations):

The variables x(t), y(t), and z(t) describe, respectively, the amplitude of the velocity mode, the temperature mode, and the mode measuring the heat flux Nu, the Nusselt number. x and y characterize the roll pattern.

The model has three dimensionless parameters: r, sigma, and b.

r is the ratio of the temperature difference between the hot and cold plate, to its critical value for the onset of convection. It can also be viewed as the ratio of the Rayleigh number to its critical value.

sigma is the Prandtl number, the ratio of the kinematic viscosity to the thermal diffusivity. Sigma is about 0.7 in air and 7 in water. Lorenz used sigma = 10.

b is of order unity and conventionally taken to have the value 8/3. It arises from the nonlinear coupling of the fluid velocity and temperature gradient in the Boussinesq approximation.

The model is a toy model because for values of r larger than r_c (defined below) "the three mode approximation for the PDEs describing thermal convection... ceased to be physically

realistic, but mathematically the model now starts to show its most fascinating properties,.."

Novelty

The model has several distinct types of long-time dynamics: stable fixed points (no convection), limit cycles (convective rolls), and most strikingly a chaotic strange attractor (represented below). The chaos is reflected in the sensitive dependence on initial conditions.

Briefly, a strange attractor is a curve of infinite length that never crosses itself and is contained in a finite volume. This means it has a fractal structure and a non-trivial Hausdorff dimension [calculated in this paper to be 2.0627160].

Discontinuities

Quantitative changes lead to qualitative changes. For r < 1, no convection occurs. For r > 1, convective rolls develop, but these become unstable for 

and a strange attractor develops.

Phase diagram

Lorenz only considered one set of parameter values [r =28, sigma=10, and b=8/3]. This was rather fortunate, because then strange attractor was waiting to be discovered. 

The phase diagram maps out the qualitatively different behaviours that occur as a function of sigma (vertical axis) and r (horizontal axis). 

Different phases are the fixed points P± associated with convective rolls (black), orbits of period 2 (red), period 4 (green), period 8 (blue), and chaotic attractors (white).

EXTENDED PHASE DIAGRAM OF THE LORENZ MODEL

H. R. DULLIN, S. SCHMIDT, P. H. RICHTER, and S. K. GROSSMANN

Universality

The details of the molecular composition of the fluid and the intermolecular interactions are irrelevant beyond how they determine the three parameters in the model. Hence, qualitatively similar behaviour can occur in systems with a wide range of chemical compositions and physical properties.

Unpredictability

Although the system of three ODEs is simple, discovery of the strange attractor and the chaotic dynamics was unanticipated. Furthermore, the dynamics in the chaotic regime are unpredictable, given the sensitivity to initial conditions.

Top-down causation

The properties and behaviour of the system are not just determined by the properties of the molecules and their interactions. The external boundary conditions, the applied temperature gradient and the spatial separation L of the hot and cold plates, are just as important in determining the dynamics of the system, including motion as much smaller length scales.

Colloquium on 2024 Nobel Prizes

This friday I am giving a colloquium for the UQ Physics department.

2024 Nobel Prizes in Physics and Chemistry: from biological physics to artificial intelligence and back

The 2024 Nobel Prize in Physics was awarded to John Hopfield and Geoffrey Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” Half of the 2024 Chemistry prize was awarded to Dennis Hassabis and John Jumper for “protein structure prediction” using artificial intelligence. I will describe the physics background needed to appreciate the significance of the awardees work. 

Hopfield proposed a simple theoretical model for how networks of neurons in a brain can store and recall memories. Hopfield drew on his background in and ideas from condensed matter physics, including the theory of spin glasses, the subject of the 2021 Physics Nobel Prize.

Hinton, a computer scientist, generalised Hopfield’s model, using ideas from statistical physics to propose a “Boltzmann machine” that used an artificial neural network to learn to identify patterns in data, by being trained on a finite set of examples. 

For fifty years scientists have struggled with the following challenge in biochemistry: given the unique sequence of amino acids that make up a particular protein can the native structure of the protein be predicted? Hassabis, a computer scientist, and Jumper, a theoretical chemist, used AI methods to solve this problem, highlighting the power of AI in scientific research. 

I will briefly consider some issues these awards raise, including the blurring of boundaries between scientific disciplines, tensions between public and corporate interests, research driven by curiosity versus technological advance, and the limits of AI in scientific research.

Here is my current draft of the slides.

John Hopfield on what physics is

A decade ago John Hopfield reflected on his scientific life in Annual Reviews in Condensed Matter Physics, Whatever Happened to Solid State Physics?

"What is physics? To me—growing up with a father and mother who were both physicists—physics was not subject matter. The atom, the troposphere, the nucleus, a piece of glass, the washing machine, my bicycle, the phonograph, a magnet—these were all incidentally the subject matter. The central idea was that the world is understandable, that you should be able to take anything apart, understand the relationships between its constituents, do experiments, and on that basis be able to develop a quantitative understanding of its behavior. 

Physics was a point of view that the world around us is, with effort, ingenuity, and adequate resources, understandable in a predictive and reasonably quantitative fashion. Being a physicist is a dedication to the quest for this kind of understanding."

He describes how this view was worked out in his work in solid state theory and moved into biological physics and the paper for which he was awarded the Nobel Prize. 

"Eventually, my knowledge of spin-glass lore (thanks to a lifetime of interaction with P.W. Anderson), Caltech chemistry computing facilities, and a little neurobiology led to the first paper in which I used the word neuron. It was to provide an entryway to working on neuroscience for many physicists..."

After he started working on biological physics in the late 1970s he got an offer from Chemistry and Biology at Caltech and Princeton Physics suggested he take it. 

"In 1997, I returned to Princeton—in the Molecular Biology Department, which was interested in expanding into neurobiology. Although no one in that department thought of me as anything but a physicist, there was a grudging realization that biology could use an infusion of physics attitudes and viewpoints. I had by then strayed too far from conventional physics to be courted for a position in any physics department. So I was quite astonished in 2003 to be asked by the American Physical Society to be a candidate for vice president. And, I was very happy to be elected and ultimately to serve as the APS president. I had consistently felt that the research I was doing was entirely in the spirit and paradigms of physics, even when disowned by university physics departments."

2024 Nobel Prize in Physics

 I was happy to see John Hopfield was awarded the Nobel Prize in Physics for his work on neural networks. The award is based on this paper from 1982

Neural networks and physical systems with emergent collective computational abilities

One thing I find beautiful about the paper is how Hopfield drew on ideas about spin glasses (many competing interactions lead to many ground states and a complex energy landscape).

A central insight is that an efficient way to store the information describing multiple objects (different collective spin states in an Ising model) is in terms of the inter-spin interaction constants (J_ij's) in the Ising model. These are the "weights" that are trained/learned in computer neural nets.

It should be noted that Hopfield's motivation was not at all to contribute to computer science. It was to understand a problem in biological physics: what is the physical basis for associative memory? 

I have mixed feelings about Geoffrey Hinton sharing the prize.  On the one hand, in his initial work, Hinton used physics ideas (Boltzmann weights) to extend Hopfields ideas so they were useful in computer science. Basically, Hopfield considered a spin glass model at zero temperature and Hinton considered it at non-zero temperature. [Note, the temperature is not physical it is just a parameter in a Boltzmann probability distribution for different states of the neural network]. Hinton certainly deserves lots of prizes, but I am not sure a physics one is appropriate. His work on AI has certainly been helpful for physics research. But so have lots of other advances in computer software and hardware, and those pioneers did not receive a prize.

I feel a bit like I did with Jack Kilby getting a physics prize for his work on integrated circuits. I feel that sometimes the Nobel Committee just wants to remind the world how physics is so relevant to modern technology.

Ten years ago Hopfield wrote a nice scientific autobiography for Annual Reviews in Condensed Matter Physics,

Whatever Happened to Solid State Physics?

After the 2021 Physics Nobel to Parisi, I reflected on the legacy of spin glasses, including the work of Hopfield.

Aside: I once pondered whether a chemist will ever win the Physics prize, given that many condensed matter physicists have won the chemistry prize. Well now, we have had an electronic engineer and a computer scientist winning the Physics prize.

Another side: I think calling Hinton's network a Boltzmann machine is a scientific misnomer. I should add this to my list of people getting credit for things that did not do. Boltzmann never considered networks, spin glasses or computer algorithms. Boltzmann was a genius, but I don't think we should be attaching his name to everything that involves a Boltzmann distribution. To me, this is a bit like calling the Metropolis algorithm for Monte Carlo simulations the Boltzmann algorithm. 

The multi-faceted character of emergence (part 2)

In the previous post, I considered five different characteristics that are often associated with emergence and classified them as being associated with ontology (what is real and observable) rather than epistemology (what we believe to be true). 

Below I consider five more characteristics: self-organisation, unpredictability, irreducibility, contextuality and downward causation, and intra-stratum closure.

6. Self-organisation

Self-organisation is not a property of the system but a mechanism that a theorist says causes an emergent property to come into being. Self-organisation is also referred to as spontaneous order. 

In the social sciences self-organisation is sometimes referred to as an endogenous cause, in contrast to an exogenous cause. There is no external force or agent causing the order, in contrast to order that is imposed externally. For example, suppose that in a city there is no government policy about the price of a loaf of sliced wholemeal bread or on how many loaves that bakers should produce. It is observed that prices are almost always in the range of $4 to $5 per loaf, and that rarely are there bread shortages. This outcome is a result of the self-organisation of the free-market, and economists would say the price range and its stability has an endogenous cause. In contrast, if the government legislated the price range and the production levels that would be an exogenous cause. Friedrich Hayek emphasised the role of spontaneous order in economics. In biology, Stuart Kaufmann equates emergence with spontaneous order and self-organisation.

In physics, the periodicity of the arrangement of atoms in a crystal is a result of self-organisation and has an endogenous cause. In contrast, the periodicity of atoms in an optical lattice is determined by the laser physicist who creates the lattice and so has an exogenous cause.

Self-organisation shows how local interactions can produce global properties. In different words, short-range interactions can lead to long-range order. After decades of debate and study, the Ising model showed that this was possible. Other examples of self-organisation, include flocking of birds and teamwork in ant colonies. There is no director or leader but the system acts “as if” there is. 

7. Unpredictability

Ernst Mayr (This is Biology, p.19) defines emergence as “in a structured system, new properties emerge at higher levels of integration that could not have been predicted from a knowledge of the lower-level components.” Philip Ball also defines emergence in terms of unpredictability (Quanta, 2024).

More broadly, in discussions of emergence, “prediction” is used in three different senses: logical prediction, historical prediction, and dynamical prediction.

Logical prediction (deduction) concerns whether one can predict (calculate) the emergent (novel) property of the whole system solely from a knowledge of all the properties of the parts of the system and their interactions. Logical predictability is one of the most contested characteristics of emergence. Sometimes “predict” is replaced with “difficult to predict”, “extremely difficult to predict”, “impossible to predict”, “almost impossible to predict”, or “possible in principle, but impossible in practice, to predict.” 

As an aside, I note that philosophers distinguish between epistemological emergence and ontological emergence. They are associated with prediction that is "possible in principle, but difficult in practice" and "impossible in principle" respectively.

After an emergent property has been discovered experimentally sometimes it can be understood in terms of the properties of the system parts. In a sense “pre-diction” then becomes “post-diction.” An example is the BCS theory of superconductivity, which provided a posteriori, rather than a priori, understanding. In different words, development of the theory was guided by a knowledge of the phenomena that had already been observed and characterised experimentally. Thus, a keyword in the statement above about logical prediction is “solely”. 

Historical prediction. Most new states of matter discovered by experimentalists were not predicted even though theorists knew the laws that the microscopic components of the system obeyed. Examples include superconductivity (elemental metals, cuprates, iron pnictides, organic charge transfer salts, …), superfluidity in liquid 4He, antiferromagnetism, quasicrystals, and the integer and fractional quantum Hall states.

There are a few exceptions where theorists did predict new states of matter. These include are Bose-Einstein Condensates (BECs) in dilute atomic gases and topological insulators, the Anderson insulator in disordered metals, the Haldane phase in even-integer quantum antiferromagnetic spin chains, and the hexatic phase in two dimensions. It should be noted that prediction of BECs and topological insulators were significantly helped that theorists could predict them starting with Hamiltonians of non-interacting particles. Furthermore, all of these predictions involved working with effective Hamiltonians. None started with microscopic Hamiltonians for specific materials.

Dynamical unpredictability concerns what it means in chaotic dynamical systems, where it relates to sensitivity to initial conditions. I do not see this as an example of emergence as it can occur in systems with only a few degrees of freedom. However, some authors do associate dynamical unpredictability with complexity and emergence.

8. Irreducibility and singularities

An emergent property cannot be reduced to properties of the parts, because if emergence is defined in terms of novelty, the parts do not have the property. 

Emergence is also associated with the problem of theory reduction. Formally, this is the process where a more general theory reduces in a particular mathematical limit to a less general theory. For example, quantum mechanics reduces to classical mechanics in the limit where Planck’s constant goes to zero. Einstein’s theory of special relativity reduces to Newtonian mechanics in the limit where the speeds of massive objects become much less than the speed of light. Theory reduction is a subtle philosophical problem that is arguably poorly understood both by scientists [who oversimplify or trivialise it] and philosophers [who arguably overstate the problems it presents for science producing reliable knowledge]. Subtleties arise because the two different theories usually involve language and concepts that are "incommensurate" with one another. 

Irreducibility is also related to the discontinuities and singularities associated with emergent phenomena. As emphasised independently by Hans Primas and Michael Berry, singularities occur because the mathematics of theory reduction involves singular asymptotic expansions. Primas illustrates this by considering a light wave incident on an object and producing a shadow. The shadow is an emergent property, well described by geometrical optics, but not by the more fundamental theory of Maxwell’s electromagnetism. The two theories are related in the asymptotic limit that the wavelength of light in Maxwell’s theory tends to zero. This example illustrates that theory reduction is compatible with the emergence of novelty. Primas also considers how the Born-Oppenheimer approximation, which is central to solid state theory and quantum chemistry, is associated with a singular asymptotic expansion (in the ratio of the mass of an electron to the mass of an atomic nuclei in the system). 

Berry considers several other examples of theory reduction, including going from general to special relativity, from statistical mechanics to thermodynamics, and from viscous (Navier-Stokes) fluid dynamics to inviscid (Euler) fluid dynamics. He has discussed in detail how the caustics that occur in ray optics are an emergent phenomena and are associated with singular asymptotic expansions in the wave theory.

The philosopher of science Jeremy Butterfield showed rigorously that theory reduction occurred for four specific systems that exhibited emergence, defined by him as a novel and robust property. Thus, novelty is not sufficient for irreducibility.

9. Contextuality and downward causation

Any real system has a context. For example, it has boundary and an environment, both in time and space. In many cases the properties of the system are completely determined by the parts of the system and their interactions. Previous history and boundaries do not matter. However, in some cases the context may have a significant influence on the state of the system. An example is Rayleigh-Bernard convection cells and turbulent flow whose existence and nature are determined by the interaction of the fluid with the container boundaries. A biological example concerns what factors determine the structure, properties, and function that a particular protein (linear chain of amino acids) has. It is now known that the only factor is not just the DNA sequence that encodes for the amino acid sequence, in contradiction to some versions of the Central Dogma of molecular biology.  Other factors may be the type of cell that contains the protein and the network of other proteins in which the particular protein is embedded. Context sometimes matters.

Supervenience is the idea that once the micro level is fixed, macro levels are fixed too. The examples above might be interpreted as evidence against supervenience. Supervenience is used to argue against “the possibility for mental causation above and beyond physical causation.” 

Downward causation is sometimes equated with emergence, particularly in debates about the nature of consciousness. In the context of biology, Denis Noble defines downward causation as when higher level processes can cause changes in lower level properties and processes. He gives examples where physiological effects can switch on and off individual genes or signalling processes in cells, including maternal effects and epigenetics.

10. Intra-stratum closure: informational, causal, and computational

The ideas described below were recently developed by Rosas et al. from a computer science perspective. They defined emergence in terms of universality and discussed its relationship to informational closure, causal closure, and computational closure. Each of these are given a precise technical definition in their paper. Here I give the sense of their definitions. In considering a general system they do not pre-define the micro- and macro- levels of a system but consider how they might be defined so that universality holds, i.e., so that properties at the macro-level are independent of the details of the micro-level (i.e., are universal).

Informational closure means that to predict the dynamics of the system at the macroscale an observer does not need any additional information about the details of the system at the microscale. Equilibrium thermodynamics and fluid dynamics are examples. 

Causal closure means that the system can be controlled at the macroscale without any knowledge of lower-level information. For example, changing the software code that is running on a computer allows one to reliably control the microstate of the hardware of the computer regardless of what is happening with the trajectories of electrons in the computer.

Computational closure is a more technical concept, being defined in terms of “a conceptual device called the ε-(epsilon) machine. This device can exist in some finite set of states and can predict its own future state on the basis of its current one... for an emergent system that is computationally closed, the machines at each level can be constructed by coarse-graining the components on just the level below: They are, “strongly lumpable.” "

Rosas et al., show that informational closure and causal closure are equivalent and that they are more restrictive than computational closure. It is not clear to me how these closures relate to novelty as a definition of emergence.

In summary, emergence means different things to different people. I have listed ten different characteristics that have been associated with emergent properties. They are not all equivalent and so when discussing emergence it is important to be clear about which characteristic one is using to define emergence.

The multi-faceted character of emergence (part 1)

There is more to emergence than novel properties, i.e., where a whole system has a property that the individual components of the system do not have. Here I focus on emergent properties, but in most cases “property” might be replaced with state, phenomenon, or entity. I now discuss ten characteristics often associated with emergence, beyond novelty. Some people include one or more of these characteristics in their definitions of emergence. However, I do not include them in my definition because as I explain some of the characteristics are contentious. Some may not be necessary or sufficient for novel system properties.

The first five characteristics discussed below might be classified as objective (i.e., observable properties of the system) and the second five as subjective (i.e., associated with how an investigator thinks about the system). In different words, the first five are mostly concerned with ontology (what is real) and the second five with epistemology (what we know). The first five characteristics concern discontinuities, universality, diversity, mesoscales, and modification of parts. The second five concern self-organisation, unpredictability, irreducibility, downward causation, and closure. 

1. Discontinuities 

Quantitative changes in the system can become qualitative changes in the system. For example, in condensed matter physics spontaneous symmetry breaking only occurs in the thermodynamic limit (i.e., when the number of particles of the system becomes infinite). More is different. Thus, as a quantitative change in the system size occurs the order parameter becomes non-zero. In a system that undergoes a phase transition at a non-zero temperature, a small change in temperature can lead to the appearance of order and to a new state of matter. For a first-order phase transition, there is discontinuity in properties such as the entropy and density. These discontinuities define a phase boundary in the pressure-temperature diagram. For continuous phase transitions the order parameter is a continuous function of temperature, becoming non-zero at the critical temperature. However the derivative with respect to temperature may be discontinuous and/or thermodynamic properties such as the specific heat and susceptibility associated with the order parameter may approach infinite as the critical temperature is approached.

Two different states of a system are said to be adiabatically connected if one can smoothly deform one state into the other and all the properties of the system also change smoothly. The case of the liquid-gas transition illustrates subtle issues about defining emergence. A discontinuity does not imply a qualitative difference (novelty). On the one hand, there is a discontinuity in the density and entropy of the system as the liquid-gas phase boundary is crossed in the pressure-temperature diagram. On the other hand, there is no qualitative difference between a gas and a liquid. There is only a quantitative difference: the density of the gas is less than the liquid. Albeit sometimes the difference is orders of magnitude. The liquid and gas state can be adiabatically connected. There is a path in the pressure-temperature phase diagram that can be followed to connect the liquid and gas states without any discontinuities in properties.

The ferromagnetic state also raises questions, as illustrated by a debate between Rudolf Peierls and Phil Anderson about whether ferromagnetism exhibits spontaneous symmetry breaking. Anderson argued that it did not as, in contrast to the antiferromagnetic state, a non-zero magnetisation (order parameter) occurs for finite systems and the magnetic order does not change the excitation spectrum, i.e., produce a Goldstone boson. On the other hand, singularities in properties at the Curie temperature (critical temperature for ferromagnetism) only exist in the thermodynamic limit. Also, a small change in the temperature, from just above the Curie temperature to below, can produce a qualitative change, a non-zero magnetisation.

2. Universality

Properties often referred to as emergent are universal in the sense that it is independent of many of the details of the parts of the system. There may be many different systems that can have a particular emergent property. For example, superconductivity is present in metals with a diverse range of crystal structures and chemical compositions. 

Robustness is related to universality. If small changes are made to the composition of the system (for example replacing some of the atoms in the system with atoms of different chemical element) the novel property of the system is still present. In elementary superconductors, introducing non-magnetic impurity atoms has no effect on the superconductivity.

Universality is both a blessing and a curse for theory. Universality can make it easier to develop successful theories because it means that many details need not be included in a theory in order for it to successfully describe an emergent phenomenon. This is why effective theories and toy models can work even better than might be expected. Universality can make theories more powerful because they can describe a wider range of systems. For example, properties of elemental superconductors can be described by BCS theory and by Ginzburg-Landau theory, even though the materials are chemically and structurally diverse. The curse of universality for theory is that universality illustrates the problem of “under-determination of theory”, “over-fitting of data” and “sloppy theories” [Sethna et al.]. A theory can agree with the experiment even when the parameters used in the theory may be quite different from the actual ones. For example, the observed phase diagram of water can be reproduced, sometimes with impressive quantitative detail, by combining classical statistical mechanics with empirical force fields that assume water molecules can be treated purely being composed of point charges.

Suppose we start with a specific microscopic theory and calculate the macroscopic properties of the system, and they agree with experiment. It would then be tempting to think that we have the correct microscopic theory. However, universality suggests this may not be the case.

For example, consider the case of a gas of weakly interacting atoms or molecules. We can treat the gas particles as classical or quantum. Statistical mechanics gives exactly the same equation of state and specific heat capacity for both microscopic descriptions. The only difference may be the Gibbs paradox [the calculated entropy is not an extensive quantity] which is sensitive to whether or not the particles are treated as identical or not. Unlike the zeroth, first, and second law of thermodynamics, the third law does require that the microscopic theory be quantum. Laughlin discusses these issues in terms of “protectorates” that hide “ultimate causes” .  

In some physical systems, universality can be defined in a rigorous technical sense, making use of the concepts and techniques of the renormalisation group and scaling. These techniques provide a method to perform coarse graining, to derive effective theories and effective interactions, and to define universality classes of systems. There are also questions of how universality is related to the robustness of strata, and the independence of effective theories from the coarse-graining procedure.

3. Diversity

Even when a system is composed of a small number of different components and interactions, the large number of possible stable states with qualitatively different properties that the system can have is amazing. Every snowflake is different. Water is found in 18 distinct solid states. All proteins are composed of linear chains of 20 different amino acids. Yet in the human body there are more than 100,000 different proteins and all perform specific biochemical functions. We encounter an incredible diversity of human personalities, cultures, and languages. A stunning case of diversity is life on earth. Billions of different plant and animal species are all an expression of different linear combinations of the four base pairs of DNA: A, G, T, and C.

This diversity is related to the idea that "simple models can describe complex behaviour". One example is Conway’s Game of Life. Another example is how simple Ising models with a few competing interactions can describe a devil's staircase of ground states or the multitude of different atomic orderings found in binary alloys.

Goldenfeld and Kadanoff defined complexity [emergence] as “structure with variations”. Holland (VSI) discusses “perpetual novelty” giving the example of the game of chess, where are typical game may involve the order of 1050 move sequences. “Motifs” are recurring patterns (sequences of moves) in games. 

Condensed matter physics illustrates diversity with the many different states of matter that have been discovered. The underlying microscopics is “just” electrons and atomic nuclei interacting according to Coulomb’s law.

The significance of this diversity might be downplayed by saying that it is just a result of combinatorics. But such a claim overlooks the issue of the stability of the diverse states that are observed. In a system composed of many components each of which can take on a few states the number of possible states of the whole system grows exponentially with the number of components. For example, for a chain of ten amino acids there are 1013 different possible linear sequences. But this does not mean that all these sequences will produce a functional protein, i.e., a molecule that will fold rapidly (on the timescale of milliseconds) into a stable tertiary structure and perform a useful biochemical function such as catalysis of a specific chemical reaction or signal transduction.

4. Simple entities at the mesoscale 

A key idea in condensed matter physics is that of quasi-particles. A system of strongly interacting particles may have excitations, seen in experiments such as inelastic neutron scattering and Angle Resolved PhotoElectron Spectroscopy (ARPES), that can be described as weakly interacting quasi-particles. These entities are composite particles, and have properties that are quantitatively different, and sometimes qualitatively different, from the microscopic particles. Sometimes this means that the scale (size) associated with the quasi-particles is intermediate between the micro- and the macro-scales, i.e., it is a mesoscale. The existence of quasi-particles leads naturally to the technique of constructing an effective Hamiltonian [effective theory] for the system where effective interactions describe the interactions between the quasi-particles.

The economist Herbert Simon argued that a characteristic of a complex system is that the system can be understood in terms of nearly decomposable units. Rosas et al., argue that emergence is associated with there being a scale at which the system is “strongly lumpable”. Denis Noble has highlighted how biological systems are modular, i.e., composed of simple interchangeable components.

5. Modification of parts and their relationships

Emergent properties are often associated with the state of the system exhibiting patterns, order, or structure, terms that may be used interchangeably. This reflects that there is a particular relationship (correlation) between the parts which is different to the relationships in a state without the emergent property. This relationship may also be reflected in a generalised rigidity. For example, in a solid applying a force on one surface results in all the atoms in the solid experiencing a force and moving together. The rigidity of the solid defines a particular relationship between the parts of the system.

Properties of the individual parts may also be different. For example, in a crystal single-atom properties such as electronic energy levels change quantitatively compared to their values for isolated atoms. Properties of finite subsystems are also modified, reflecting a change in interactions between the parts. For example, in a molecular crystal the frequencies associated with intramolecular atomic vibrations are different to their values for isolated molecules. However, emergence is a sufficient but not a necessary condition for these modifications. In gas and liquid states, novelty is not present but there are still such changes in the properties of the individual parts.

As stated at the beginning of this section the five characteristics above might be associated with ontology (what is real) and objective properties of the system that an investigator observes and depend less on what an observer thinks about the system. The next five characteristics might be considered to be more subjective, being concerned with epistemology (how we determine what is true). In making this dichotomy I do not want to gloss over the fuzziness of the distinction or of two thousand years of philosophical debates about the relationship between ontology and epistemology, or between reality and theory.

In the next post, I will discuss the remaining five characteristics: self-organisation, unpredictability, irreducibility, contextuality and downward causation, and intra-stratum closure.

Thanks for reading this far!

Clarity about the relationship of emergence, complexity, predictability, and universality

Emergence means different things to different people. Except, that practically everyone likes it! Or at least, likes using the word. Terms associated with emergence include novelty, unpredictability, universality, stratification, and self-organisation. We need to be clearer about what we mean by each of these terms and how they are related or unrelated. Significant progress is reported in a recent preprint.

Software in the natural world: A computational approach to hierarchical emergence

Fernando E. Rosas, Bernhard C. Geiger, Andrea I Luppi, Anil K. Seth, Daniel Polani, Michael Gastpar, Pedro A.M. Mediano

This preprint is the subject of a nice article in Quanta Magazine.

The New Math of How Large-Scale Order Emerges by Philip Ball

Ball defines emergence in terms of unpredictability. He states: 

"Loosely, the behavior of a complex system might be considered emergent if it can’t be predicted from the properties of the parts alone."

He describes the work of Rosas et al. as follows, 

"A complex system exhibits emergence, according to the new framework, by organizing itself into a hierarchy of levels that each operate independently of the details of the lower levels."

This is defining emergence in terms of universality. Rosas et al. use an analogy with software, which runs independently of the details of the hardware of the computer and does not depend on microscopic details such as electron dynamics.

There are three types of closure associated with emergence: informational, causal, and computational.

Informational closure means that to predict the dynamics of the system at the macroscale one does not need any additional  information from the microscale.

Equilibrium thermodynamics is a nice example. 

Causal closure means that the system can be controlled at the macroscale without any knowledge of lower-level information.

"Interventions we make at the macro level, such as changing the software code by typing on the keyboard, are not made more reliable by trying to alter individual electron trajectories."

"...we can use macroscopic variables like pressure and viscosity to talk about (and control) fluid flow, and knowing the positions and trajectories of individual molecules doesn’t add useful information for those purposes. And we can describe the market economy by considering companies as single entities, ignoring any details about the individuals that constitute them."

Computational closure is a more technical concept. 

"a conceptual device called the ε-(epsilon) machine. This device can exist in some finite set of states and can predict its own future state on the basis of its current one. It’s a bit like an elevator, said Rosas; an input to the machine, like pressing a button, will cause the machine to transition to a different state (floor) in a deterministic way that depends on its past history — namely, its current floor, whether it’s going up or down and which other buttons were pressed already. Of course an elevator has myriad component parts, but you don’t need to think about them. Likewise, an ε-machine is an optimal way to represent how unspecified interactions between component parts “compute” — or, one might say, cause — the machine’s future state."

Aside: epsilon-machines featured significantly in my previous post about What is a complex system? 

"Computational mechanics allows the web of interactions between a complex system’s components to be reduced to the simplest description, called its causal state."

"...for an emergent system that is computationally closed, the machines at each level can be constructed by coarse-graining the components on just the level below: They are, in the researchers’ terminology, “strongly lumpable.”"

In some sense, this may be related to the notion of quasiparticles and effective interactions in many-body physics. 

Aside: In 1962, Herbert Simon identified hierarchies as an essential feature of complex systems, both natural and artificial. A key property of a level in the hierarchy is that it is nearly decomposable into smaller units, i.e., it can be viewed as a collection of weakly interacting units. The time required for the evolution of the whole system is significantly decreased due to the hierarchical character. The construction of an artificial complex system, such as a clock, is faster and more reliable if different units are first assembled separately and then the units are brought together into the whole. Simon argues that the reduction in time scales due to modularity is why biological evolution can occur on realistic time scales.  The 1962 article is reprinted in The Sciences of the Artificial.

The paper by Rosas et al. is one of the most important ones I have encountered in the past few years. I am slowly digesting it.

The beauty of the paper that it is mathematically rigorous. All the concepts are precisely defined and the central results are actually theorems. This replaces the vagueness of most discussions of emergence, including by myself.

The paper has helpful figures and considers concrete examples including Ehrenfest's Urn, an Ising model with Glauber dynamics, and a Hopfield neural network model.

I thank Gerard Milburn for bringing the Quanta article to my attention.

Emergence talk at University of the Philippines

Today I am giving a seminar on emergence at the University of Philippines Los Banos, in the Institute of Mathematical Sciences and Physics.

Here are my slides.

The interplay of ecological and evolutionary dynamics: immigration, extinction, and chaos (and DMFT?)

"Ecological and evolutionary dynamics are intrinsically entwined. On short timescales, ecological interactions determine the fate and impact of new mutants, while on longer timescales evolution shapes the entire community."

 Spatiotemporal ecological chaos enables gradual evolutionary diversification without niches or tradeoffs       Aditya Mahadevan, Michael T Pearce, and Daniel S Fisher

Understanding this interplay is "one of the biggest open problems in evolution and ecology."

New experimental techniques for measuring the properties of large microbial ecosystems have stimulated significant theoretical work, including from some with a background in theoretical condensed matter physics. For an excellent accessible introduction see:

Understanding chaos and diversity in complex ecosystems – insights from statistical physics

This is a nice 2.5-page article by Pankaj Mehta at the Journal Club for Condensed Matter. He clearly introduces an important problem in theoretical ecology and evolution and describes how some recent work has provided new insights using techniques adapted from Dynamical Mean-Field Theory, which was originally developed to describe strongly correlated electron systems. 

Here are just a few highlights of the article. It may be better to just read the actual article.

Fifty years ago, Robert May "argued that the more diverse an ecosystem is (roughly defined as the number of species present), the less stable it becomes." He derived this counter-intuitive result using a simple model and results from Random Matrix Theory. This is an example of an emergent property: qualitative difference occurs as a system of interacting parts becomes sufficiently large.

"One major deficiency of May’s argument is that it does not allow for the possibility that complex ecosystems can self organize through immigration and extinction. The simplest model that contains all these processes is the Generalized [to many species] Lotka-Volterra model (GLV)".

"Despite its simplicity, this equation holds many surprises, especially when the number of species is large".

Another case of how simple models can exhibit complex behaviour.

One special case is when the interactions are reciprocal – how species i affects species j is identical to how species j affects species I. "In the presence of non-reciprocity the system can exhibit complex dynamical behavior including chaos." Understanding this case was an open problem until the two papers reviewed by Mehta. For a detailed but pedagogical introduction see:

Les Houches Lectures on Community Ecology: From Niche Theory to Statistical Mechanics, Wenping Cui, Robert Marsland III, Pankaj Mehta

This is relevant to understanding the origin of the fine grained diversity observed in sequencing experiments of microbial ecosystems.

Aside: de Pirey and Bunin "derive analytic expressions for the steady-state abundance distribution and an analogue of the fluctuation-dissipation theorem for chaotic dynamics relating static and dynamics correlation functions."

"Using a DMFT solution, they derive a number of remarkable predictions... in the chaotic system the species fall into two groups: species at high abundances and species at low abundances near the immigration floor. de Pirey and Bunin show that even in the chaotic regime, the number of high abundance species in the ecosystem will always be less than the May stability bound. This result is quite surprising since it suggests that ecosystems self-organize in such a way that the high abundance species still follow May’s diversity bound even when they are chaotic."