Information theoretic measures for emergence and causality

The relationship between emergence and causation is contentious, with a long history. Most discussions are qualitative. Presented with a new system, how does one identify the microscopic and macroscopic scales that may be most useful for understanding and describing the system? Can Judea Pearl’s seminal ideas about causality be implemented practically for understanding emergence?

Broadly speaking, a weakness of discussions of emergence and causality is that it is hard to define these concepts in a rigorous and quantitative manner that makes them amenable to empirical testing, with respect to theoretical models and to experimental data. 

Fortunately, in the past decade, there have been some specific proposals to address this issue, mostly using information theory. A helpful recent review is by Yuan et al. 

“Two primary challenges take precedence in understanding emergence from a causal perspective. The first is establishing a quantitative definition of emergence, whereas the second involves identifying emergent behaviors or phenomena through data analysis.

To address the first challenge, two prominent quantitative theories of emergence have emerged in the past decade. The first is Erik Hoel et al.’s theory of causal emergence [19] whereas the second is Fernando E. Rosas et al.’s theory of emergence based on partial information decomposition [24].

Hoel et al.’s theory of causal emergence specifically addresses complex systems that are modeled using Markov chains. It employs the concept of effective information (EI) to quantify the extent of causal influence within Markov chains and enables comparisons of EI values across different scales [19,25]. Causal emergence is defined by the difference in the EI values between the macro-level and micro-level."

One perspective on causal emergence is that it occurs when the dynamics of a system at the macro-level is described more efficiently by macro-variables than by the dynamics of variables from the micro-level.

Klein et al. used Hoel’s information-theoretic measures of causal emergence to analyse protein interaction networks (interactomes) in over 1800 species, containing more than eight million protein–protein interactions, across different scales. They showed the emergence of ‘macroscales’ that are associated with lower noise and uncertainty. The nodes in the macroscale description of the network are more resilient than those in less coarse-grained descriptions. Greater causal emergence (i.e., a stronger macroscale description) was generally seen in multicellular organisms compared to single-cell organisms. The authors quantified causal emergence in terms of mutual information (between large and small scales) and effective information (a measure of the certainty in the connectivity of a network). Philip Ball (2023) (pages 218-220) gives an account of this work in terms of the emergence of multicellularity in biological evolution. He introduced the term causal spreading (pages 225-7), arguing that over the history of evolution the locus of causation has changed.

Yuan et al. continue

"However, in Hoel’s theory of causal emergence, it is essential to establish a coarse-graining strategy beforehand. Alternatively, the strategy can be derived by maximizing the effective information (EI) [19]. However, this task becomes challenging for large-scale systems due to the computational complexity involved. To address these problems, Rosas et al. introduced a new quantitative definition of causal emergence [24] that does not depend on coarse-graining methods, drawing from partial information decomposition (PID)-related theory. PID is an approach developed by Williams et al., which seeks to decompose the mutual information between a target and source variables into non-overlapping information atoms: unique, redundant, and synergistic information [29]…"

The Figure below is taken from Rosas et al. Xt^j (j=1,…,n) are microscopic variables that define a Markov chain. Vt is a macroscopic variable that is completely determined by the microscopic variables.

“Diagram of causally emergent relationships. Causally emergent features have predictive power beyond individual components. Downward causation takes place when that predictive power refers to individual elements; causal decoupling when it refers to itself or other high-order features.”

Rosas et al. applied the method to specific systems, including Conway’s Game of Life, Reynolds’ flocking model, and neural activity as measured by electrocorticography. More recently, it was used to describe emergence in computer science, including the identification of modular structures. Calculations were performed for specific examples, including Ehrenfest’s urn model for diffusion, the Ising model with Glauber dynamics, a Hopfield neural network model for associative memory.

Yuan et al. also state the following:

"The second challenge pertains to the identification of emergence from data. In an effort to address this issue, Rosas et al. derived a numerical method [24]. However, it is important to acknowledge that this method offers only a sufficient condition for emergence and is an approximate approach. Another limitation is that a coarse-grained macro-state variable should be given beforehand to apply this method."

Sas et al. recently stated

“Empirical applications of this framework to study emergence … including the study of gene regulatory networks [22], the dynamics of the human brain [23], the internal dynamics of reservoir computing [24], and the formation of useful internal representations in machine learning [25].”

Yuan et al. also discuss two significant connections between causal emergence and machine learning. First, machine learning can be used to improve calculations of causal emergence. Second, causal emergence measures can be used to better understand how machine learning works and improve it.

The work described above built on earlier work by Crutchfield, who claimed that the identification of emergence and hierarchies could be made operational, stating that “different scales are delineated by a succession of divergences in statistical complexity at lower levels.” More recently, Rupe and Crutchfield have reported progress towards identifying emergent self-organisation in a system.

Although this work on quantitative measures of emergence based on information theory represents significant progress, there are many open problems. Examples include the extension to non-Markovian systems and the development of computationally feasible methods for large systems. The latter is particularly important in physical systems where spontaneous symmetry breaking occurs, as this only happens in the thermodynamic limit of an infinite system.

There is an unrecognised similarity between the work described above and techniques recently developed to characterise phase transitions in statistical mechanics models such as the Ising model and classical dimer models. Coarse-graining (CG) is optimised by maximising the Real-Space Mutual Information (RSMI) between a spatial block and its distant environment. 

In general, maximising mutual information is notoriously hard but can be done using state-of-the-art machine learning algorithms. Gokmen et al. have developed an algorithm that they claim “can, unsupervised, construct order parameters, locate phase transitions, and identify spatial correlations and symmetries for complex and large-dimensional real-space data.” Furthermore, the optimal CG explicitly identifies the scaling operators associated with the critical point. 

The classical dimer model provides a stringent test as “the relevant low-energy degrees of freedom are profoundly different from the microscopic building blocks of the theory and change qualitatively throughout the phase diagram.” In other words, the emergent entities (quasiparticles such as vortices associated with the height field, which is described by a sine-Gordon field theory) are different from the dimers.

It is encouraging to see that two different scientific communities have developed similar ideas to address this challenging problem of making discussions about emergence and causality more concrete and quantitative.

Responding to scientific uncertainty

Science provides an impressive path to certainty in some areas, particularly in physics. However, as scientists seek to describe increasingly complex entities, moving from chemistry to biology, and then to humans and societies, the level of uncertainty increases.

One observes a wide range of responses to scientific knowledge being uncertain. Here are a few.

Denial. Science is about facts and absolute truth. There really isn’t a problem. We should just trust the scientists.

Minimisation. There is some uncertainty, but it isn’t anything to be concerned about. Some scientists will also minimise any uncertainty about their own research. This may occur because of career ambition. Others will minimise public discussion of uncertainty to try and avoid promoting the science scepticism discussed below.

Optimistic perseverance. The uncertainty is openly acknowledged. Some of the uncertainty does not matter for what we need to know. Other uncertainties can be reduced by further scientific work, such by more precise measurements with new instruments or by developing more sophisticated theories.

Total scepticism. There is a suspicion about the validity of most scientific knowledge, particularly that which is perceived to have philosophical, religious, or political implications.

Suspicion about science

In spite of the success of science at describing the material world and leading to powerful and useful technologies, there is much public suspicion of science. On the one hand, this is understandable given that science has led to technologies with undesirable health, environmental and social consequences. Some scientists, governments and companies have lied about these consequences and hidden them from the public. Human subjects have been abused in medical experiments. Drugs that were claimed to be effective and safe turned out to be ineffective or have undesirable side effects. Science has been used for ideological purposes. Sometimes scientists have faked results to advance their own careers. However, these failures should not undermine our trust in reliable scientific knowledge. Distinctions should be made between the bodies of knowledge, the applications of that knowledge, and the actions of institutions. I now discuss several common claims in public discussion that are used to justify scepticism of scientific knowledge.

Science is always changing. 

One day, scientists tell you that chocolate is good for your health, and the next year they say it is bad for you. And that is just the start. Then there are eggs, wine, running marathons, and cheese. They just can’t make up their mind. So why should we trust them? At one time, they believed in phlogiston and the aether. Now they say they don’t exist. Aristotle was replaced by Newton, who was replaced by Einstein. So why believe in human-induced climate change, biological evolution, vaccines, the Big Bang theory, or Einstein’s theories?

It is true that scientific knowledge does develop and change over time. However, today we have incredibly detailed observations and theories in physics, astronomy, chemistry, biology, and geology. Any future changes will be relatively minor because they will have to be consistent with all the knowledge we have now. Furthermore, when theories change, such as when Einstein superseded Newton, they don’t show that the old theory was completely wrong, but rather that it applied in a limited domain. For example, Newton’s theories of motion and gravity are extremely reliable when it comes to objects that are much larger than atoms, less dense than a black hole, and are moving at speeds less than about 10,000 kilometres per second. This is why engineers spend years learning Newton’s theories, not Einstein’s. If you want to build a good bridge or a rocket, Newton is good enough. He is not wrong.

Update. (Jan. 19). I just discovered that the NY Times had a recent op-ed Science Keeps Changing. So Why Should We Trust It?

“Well, that’s just a theory.” 

In popular debate, such a refrain may be applied to the theory of biological evolution, the Big Bang theory in cosmology, or human-induced climate change. The claimant usually wants to dismiss a particular theory as just idle speculation. Here, the term “theory” is used in the same sense as everyday speculations, such as “I have a theory as to why the president resigned,” or “I have a theory about why my computer is running so slowly.” These are just stories that sound somewhat plausible. In contrast, scientific theories in physics, such as quantum theory and Einstein’s theories of relativity, have precisely defined mathematical formulations that have been checked for logical consistency, made specific predictions, and tested to great precision in experiments. They are not “just theories.” For example, for the Big Bang theory about the beginning of the universe and Darwin’s theory of biological evolution and diversity, there are many independent lines of evidence that are consistent with each theory.  

Scientists cannot be trusted. 

They are not committed to the truth, but rather to their own interests and agendas, related to their careers, politics, and religion. They close ranks and support the status quo of current scientific “dogma”, rather than being open to original thinkers who critique it and propose alternative theories. They don’t want to lose their well-paid jobs and lucrative grants. 

On the one hand, scientists can be conservative and resistant to new ideas. On the other hand, there are significant career incentives to overturn existing knowledge and have your radical new theory accepted. That is how some scientists become famous and win Nobel Prizes. The reasons it does not happen very often are not necessarily for social or ideological reasons. Many of the theories we have today can explain an awful lot. It requires a lot of evidence, carefully acquired and checked, to convince people that those theories need to be modified, let alone abandoned. This may take decades. But it does happen. An example is the Big Bang theory of the universe, whose acceptance was initially resisted because it went against the prevailing view that the universe did not have a beginning. In biology, the discovery in 1970 of the enzyme reverse transcriptase went against a popular version of the “Central dogma” of molecular biology that DNA was always converted to RNA and not the reverse. That discovery led to a Nobel Prize.

I don’t trust scientists. I will do my own research. There is lots of good material from unbiased sources on the internet.

The internet provides a range of information and perspectives on practically any issue imaginable, including science. The material is particularly vast and controversial on biological evolution, the beginning of the universe, fundamental physics, the age of the earth, climate change, and medicine. Since the covid-19 pandemic, scepticism of the effectiveness and safety of vaccines has increased. 

Ivermectin is a drug that was developed as a treatment for parasite worms. Its incredible success was recognised by the award of the 2015 Nobel Prize in Physiology or Medicine to William Campbell and Satoshi Omura, who discovered the drug. During the pandemic, high-profile politicians and social media influencers promoted ivermectin as a treatment for covid-19, even after systematic medical studies showed it was ineffective. Recently, it has gained a reputation as a “miracle” drug that can even cure cancer, but this is being suppressed by the medical establishment. All clinical trials have shown the drug is ineffective for human ailments, beyond deworming. Nevertheless, there are groups on social media with hundreds of thousands of members that discuss the conspiracy, how to get the drug, and the experiences of participants using it to treat a wide range of ailments. Danny Lemoi, a founder of one of the largest groups, died in 2023 after taking massive daily doses of the drug for several years to treat a heart condition. Afterwards, one member of the group wrote “No one can convince me that he died because of ivermectin. He ultimately died because of our failed western medicine which only cares about profits and not the cure.”

Fans of ivermectin claim that they are escaping the biases and vested interests of the medical establishment and Big Pharma as they pursue the truth. However, they are not escaping bias and vested interests. Successful social influencers build their reputations and million-dollar incomes from promoting scepticism. If there is no conspiracy, just scientific uncertainty and occasional incompetence and malpractice, their following collapses. Populist politicians build their careers on criticism of and stoking resentment towards elites, such as the medical establishment. The authority of the medical establishment is replaced with the authority of the popular opinion of a group of people whose views are shaped by social media algorithms, intuition, and anecdotal experience.

My purpose in giving the example of Ivermectin is not to start a detailed critique of science scepticism. Rather, it is to illustrate the role that the interplay of trust, authority, and tradition plays in how we determine what is true and what to act on. There are two competing traditions here: the populism of alternative medicine and the elitism of professional medicine. Each has its own sources of authority. In the end, it boils down to who we trust. We do not have the time, energy, resources or inclination to check the veracity of every single piece of information we have access to. We take shortcuts. This is what tradition does for us, for better and worse. Thus, we cannot escape tradition. We are all swimming in traditions, many of which are in conflict with one another. The question is whether we are aware of it and what we do with that awareness.

Nobel Prize predictions for 2025

 This week Nobel Prizes will be announced. I have not done predictions since 2020. This is a fun exercise. It is also good to reflect on what has been achieved, including outside our own areas, and big advances from the past we may now take for granted.

Before writing this I looked at suggestions from readers of Doug Natelson's blog, nanoscale views, an article in Physics World, predictions from Clarivate based on citations, and recent recipients of the Wolf Prize.

Please enter you own predictions below.

Although we know little about how the process actually works or the explicit criteria used, I have a few speculative suggestions and observations.

1. The Wolf Prize is often a precursor.

2. Every now and then, they seem to surprise us.

3. Every few years, the physics committee seems to go for something technological, sometimes arguably outside physics, perhaps to remind people how important physics is to modern technology and other areas of science.

4. They seem to spread the awards around between different areas of physics.

5. Theory only gets awards when it has led to well-established experimental observations. Brilliant theoretical discoveries that motivate large research enterprises (more theory and experimental searches) are good enough. This is why predictions based on citation numbers may be misleading.

6. Once an award has been made on one topic, it is unlikely that there will be another award for a long time, if ever, on that same topic. In other words, there is a high bar for a second award.

7. I don't think the logic is to pick an important topic and then choose who should get the prize for the topic. This approach works against topics where many researchers independently made contributions that were all important. The awardee needs to be a standout who won't be a debatable choice.

What do you think of these principles?

For some of the above reasons, I discuss below why I am sceptical about some specific predictions.

My top prediction for physics is Metamaterials with negative refractive index, going to John Pendry (theory) and David Smith (experiment). This is a topic I know little about.

Is it just a matter of time before twisted bilayer graphene wins a prize? This might go to Allan MacDonald (theory) and Pablo Jarillo-Herrero (experiment). They recently received a Wolf Prize. One thing that convinced me of the importance of this discovery was a preprint on moiré WSe2 with beautiful phase diagrams such as this one.

The level of control is truly amazing. Helpful background is the recent Physics Today article by Bernevig and Efetov.

This is big enough to overcome 6. and the earlier prize for graphene.

Unfortunately, my past prediction/wish of Kondo and heavy fermions won't happen as Jun Kondo died in 2022. This suggestion also always went against Principle 6, with the award to Ken Wilson citing his solution of the Kondo problem.

The prediction of Berry and Aharonov for topological phases in quantum mechanics is reasonable, except for questions about historical precursors.

The prediction of topological insulators is going against 6. and the award to Haldane in 2016.

Clarivate's predictions of DiVincenzo and Loss (for qubits based on electron spin in quantum dots) goes against 5. and 7. It is just one of many competing proposals for a scaleable quantum computer and a large-scale device is still elusive.

Predictions of a prize for quantum algorithms (Shor, Deutsch, Brassard, Bennett) go against 5. 

Chemistry 

I don't know enough chemistry to make meaningful predictions. On the other hand, in 2019 I did correctly predicted John Goodenough for lithium batteries.  I do like the prediction from Clarivate for Biomolecular condensates (Brangwynne, Hyman, and Rosen). I discussed them briefly in my review article on emergence.

What do you think about my 7 "principles"?

What are your predictions?

Turbulent flows in active matter

The power of toy models and effective theories in describing and understanding emergent phenomena is illustrated by a 2012 study of the turbulence in the fluid flow of swimming bacteria.

Meso-scale turbulence in living fluids

Henricus H. Wensink, Jörn Dunkel, Sebastian Heidenreich, Knut Drescher, Raymond E. Goldstein, Hartmut Löwen, and Julia M. Yeomans

They found that a qualitative and quantitative description of observations of flow patterns, energy spectra, and velocity structure functions was given by a toy model of self-propelled rods (similar to that proposed for flocking of birds) and a minimal continuum model for incompressible flow. For the toy model, they presented a phase diagram (shown below) as a function of the volume fraction of the fluid occupied by rods and the aspect ratio of the rods. There were six distinct phases: dilute state (D), jamming (J), swarming (S), bionematic (B), turbulent (T), and laned (L). The turbulent state occurs for high filling fractions and intermediate aspect ratios, covering typical values for bacteria.

The horizontal axis is the volume fraction, going from 0 to 1.

The figure below compares the experimental data (top right) for the vorticity and the toy model (lower left) and the continuum model (lower right).

Regarding this work, Tom McLeish highlighted the importance of the identification of the relevant mesoscopic scale and the power of toy models and effective theories in the following beautiful commentary taken from his book, The Poetry and Music of Science

“Individual ‘bacteria’ are represented in this simulation by simple rod-like structures that possess just the two properties of mutual repulsion, and the exertion of a constant swimming force along their own length. The rest is simply calculation of the consequences. No more detailed account than this is taken of the complexities within a bacterium. It is somewhat astonishing that a model of the intermediate elemental structures, on such parsimonious lines, is able to reproduce the complex features of the emergent flow structure. 

Impossible to deduce inductively the salient features of the underlying physics from the fluid flow alone—creative imagination and a theoretical scalpel are required: the first to create a sufficient model of reality at the underlying and unseen scale; the second to whittle away at its rough and over-ornate edges until what is left is the streamlined and necessary model. To ‘understand’ the turbulent fluid is to have identified the scale and structure of its origins. To look too closely is to be confused with unnecessary small detail, too coarsely and there is simply an echo of unexplained patterns.”

Confusing bottom-up and top-down approaches to emergence

Due to emergence, reality is stratified. This is reflected in the existence of semi-autonomous scientific disciplines and subdisciplines. A major goal is to understand the relationship between different strata. For example, how is chemistry related to physics? How is genetics related to cell biology?

Before describing two alternative approaches —top-down and bottom-up —I need to point out that in different fields, these terms are used in opposite senses. That can be confusing!

In the latest version of my review article on emergence, I employ the same terminology traditionally used in condensed matter physics, chemistry, and biology. It is also consistent with the use of the term “downward causation” in philosophy. 

Top-down means going from long-distance scales to short-distance scales, i.e., going down in the diagrams shown in the figure above. In contrast, in the quantum field theory of elementary particles and fields (high-energy physics), “top-down” means the opposite, i.e., going from short to long distance length scales. This is because practitioners in that field tend to draw diagrams with high energies at the top and low energies at the bottom.

Bottom-up approaches aim to answer the question: how do properties observed at the macroscale emerge from the microscopic properties of the system? 

History suggests that this question may often be best addressed by identifying the relevant mesoscale at which modularity is observed and connecting the micro- to the meso- and connecting the meso- to the macro. For example, high-energy degrees of freedom can be "integrated out" to give an effective theory for the low-energy degrees of freedom.

Top-down approaches try to surmise something about the microscopic from the macroscopic. This has a long and fruitful history, albeit probably with many false starts that we may not hear about, unless we live through them or read history books. Kepler's snowflakes are an early example. Before people were completely convinced of the existence of atoms, the study of crystal facets and of Brownian motion provided hints of the atomic structure of matter. Planck deduced the existence of the quantum from the thermodynamics of black-body radiation, i.e. from macroscopic properties. Arguably, the first definitive determination of Avogadro's number was from Perrin's experiments on Brownian motion, which involved mesoscopic measurements. Comparing classical statistical mechanics to bulk thermodynamic properties gave hints of an underlying quantum structure to reality. The Sackur-Tetrode equation for the entropy of an ideal gas hinted at the quantisation of phase space. The Gibbs paradox hinted that fundamental particles are indistinguishable. The third law of thermodynamics hints at quantum degeneracy. Pauling’s proposal for the structure of ice was based on macroscopic measurements of its residual entropy. Pasteur deduced the chirality of molecules from observations of the facets in crystals of tartaric acid. Sometimes a “top-down” approach means one that focuses on the meso-scale and ignores microscopic details.

The top-down and bottom-up approaches should not be seen as exclusive or competitive, but rather complementary. Their relative priority or feasibility depends on the system of interest and the amount of information and techniques available to an investigator. Coleman has discussed the interplay of emergence and reductionism in condensed matter. In biology, Mayr advocated a “dual level of analysis” for organisms. In social science, Schelling discussed the interplay of the behaviour of individuals and the properties of social aggregates. In a classic study of complex organisations in business, understanding this interplay was termed differentiation and integration.

I thank Jeremy Schmit for requesting clarification of this terminology.

My review article on emergence

I just posted on the arXiv a long review article on emergence

Emergence: from physics to biology, sociology, and computer science

The abstract is below.

I welcome feedback. 

------

Many systems of interest to scientists involve a large number of interacting parts and the whole system can have properties that the individual parts do not. The system is qualitatively different to its parts. More is different. I take this novelty as the defining characteristic of an emergent property. Many other characteristics have been associated with emergence are reviewed, including universality, order, complexity, unpredictability, irreducibility, diversity, self-organisation, discontinuities, and singularities. However, it has not been established whether these characteristics are necessary or sufficient for novelty. A wide range of examples are given to show how emergent phenomena are ubiquitous across most sub-fields of physics and many areas of biology and social sciences. Emergence is central to many of the biggest scientific and societal challenges today. Emergence can be understood in terms of scales (energy, time, length, complexity) and the associated stratification of reality. At each stratum (level) there is a distinct ontology (properties, phenomena, processes, entities, and effective interactions) and epistemology (theories, concepts, models, and methods). This stratification of reality leads to semi-autonomous scientific disciplines and sub-disciplines. A common challenge is understanding the relationship between emergent properties observed at the macroscopic scale (the whole system) and what is known about the microscopic scale: the components and their interactions. A key and profound insight is to identify a relevant emergent mesoscopic scale (i.e., a scale intermediate between the macro- and micro- scales) at which new entities emerge and interact with one another weakly. In different words, modular structures may emerge at the mesoscale. Key theoretical methods are the development and study of effective theories and toy models. Effective theories describe phenomena at a particular scale and sometimes can be derived from more microscopic descriptions. Toy models involve minimal degrees of freedom, interactions, and parameters. Toy models are amenable to analytical and computational analysis and may reveal the minimal requirements for an emergent property to occur. The Ising model is an emblematic toy model that elucidates not just critical phenomena but also key characteristics of emergence. Many examples are given from condensed matter physics to illustrate the characteristics of emergence. A wide range of areas of physics are discussed, including chaotic dynamical systems, fluid dynamics, nuclear physics, and quantum gravity. The ubiquity of emergence in other fields is illustrated by neural networks, protein folding, and social segregation. An emergent perspective matters for scientific strategy, as it shapes questions, choice of research methodologies, priorities, and allocation of resources. Finally, the elusive goal of the design and control of emergent properties is considered.

Emergence in Chemistry

It is important to be clear what the system is. Most of chemistry is not really about isolated molecules. A significant amount of chemistry occurs in an environment, often within a solvent. Then the system is the chemicals of interest and the solvent. For example, when it is stated that HCl is an acid, this is not a reference to isolated HCl molecules but a solution of HCl in water, and then the HCl dissociates into H+ and Cl- ions. Chemical properties such as reactivity can change significantly depending on whether a compound is in the solid, liquid, or gas state, or on the properties of the solvent in which it is dissolved.

Scales

The time scales for processes, which range from molecular vibrations to chemical reactions, can vary from femtoseconds to days. Relevant energy scales, corresponding to different effective interactions, can vary from tens of eV (strong covalent bonds) to microwave energies of 0.1 meV (quantum tunnelling in an ammonia maser).

Other scales are the total number of atoms in a compound, which can range from two to millions, the total number of electrons, and the number of different chemical elements in the compound. As the number of atoms and electrons increases, so does the dimensionality of the Hilbert space of the corresponding quantum system.

Novelty

All chemical compounds are composed of a discrete number of atoms, usually of different type. For example, acetic acid, denoted CH3COOH, is composed of carbon, oxygen, and hydrogen atoms. The compound usually has chemical and physical properties that the individual atoms do not have.

Chemistry is all about transformation. Reactants combine to produce products, e.g. A + B -> C. C may have chemical or physical properties that A and B did not have.

Chemistry involves concepts that do not appear in physics. Roald Hoffmann argued that concepts such as acidity and basicity, aromaticity, functional groups, and substituent effects have great utility and are lost in a reductionist perspective that tries to define them precisely and mathematicise them.

Diversity

Chemistry is a wonderland of diversity, as it arranges chemical elements in a multitude of different ways that produce a plethora of phenomena. Much of organic chemistry just involves three different atoms: carbon, oxygen, and hydrogen.

Molecular structure

Simple molecules (such as water, ammonia, carbon dioxide, methane, benzene) have a unique structure defined by fixed bond lengths and angles. In other words, there is a well-defined geometric structure that gives the locations of the centres of atomic nuclei. This is a classical entity. This emerges from the interactions between the electrons and nuclei of the constituent atoms.

In philosophical discussions of emergence in chemistry, molecular structure has received significant attention. Some claim it provides evidence of strong emergence. The arguments centre around the fact that the molecular structure is a classical entity and concept that is imposed, whereas a logically self-consistent approach would treat both electrons and nuclei quantum mechanically.

The molecular structure of ammonia (NH3) illustrates the issue. It has an umbrella structure which can be inverted. Classically, there are two possible degenerate structures. For an isolated molecule, quantum tunnelling back and forth between the two structures can occur. The ground state is a quantum superposition of two molecular structures. This tunnelling does occur in a dilute gas of ammonia at low temperature, and an associated quantum transition (at a wavelength of 1.2 cm) is the basis of the maser, the forerunner of the laser. This example of ammonia was discussed by Anderson at the beginning of his seminal More is Different article to illustrate how symmetry breaking leads to well-defined molecular structures in large molecules. 

Figure is taken from here.

Born-Oppenheimer approximation 

Without this concept, much of theoretical chemistry and condensed matter would be incredibly difficult. It is based on the separation of time and energy scales associated with electronic and nuclear motion.  It is used to describe and understand the dynamics of nuclei and electronic transitions in solids and molecules. The potential energy surfaces for different electronic states define effective theory for the nuclei. Without this concept, much of theoretical chemistry and condensed matter would be incredibly difficult.

Singularity. The Born-Oppenheimer approximation is justified by an asymptotic expansion in powers of (m/M)^1/4, where m is the mass of an electron and M the mass of an atomic nucleus in the molecule. This has been discussed by Primas and Bishop.

The rotational and vibrational degrees of freedom of molecules also involve a separation of time and energy scales. Consequently, one can derive separate effective Hamiltonians for the vibrational and rotational degrees of freedom.

Qualitative difference with increase in molecular size

Consider the following series with varying chemical properties: formic acid (CH2O2), acetic acid (C2H4O2), propionic acid (C3H6O2), butyric acid (C4H8O2), and valerianic acid (C5H10O2), whose members involve the successive addition of a CH2 radical. The Marxist Friedrich Engels used these examples as evidence for Hegel’s law: “The law of transformation of quantity into quality and vice versa”.

In 1961, Platt discussed properties of large molecules that “might not have been anticipated” from properties of their chemical subgroups. Table 1 in Platt’s paper lists “Properties of molecules in the 5- to 50-range that have no counterpart in diatomics and many triatomics.” Table 2 lists “Properties of molecules in the 50- to 500-atom range and up that go beyond the properties of their chemical sub-groups.” The properties listed included internal conversion (i.e., non-radiative decay of excited electronic states), formation of micelles for hydrocarbon chains with more than ten carbons, the helix-coil transition in polymers, chromatographic or molecular sorting properties of polyelectrolytes such as those in ion-exchange resins, and the contractility of long chains.

Platt also discussed the problem of molecular self-replication. Until 1951, it was assumed that a machine could not reproduce itself,f and this was the fundamental difference between machines and living systems. However, von Neumann showed that a machine with a sufficient number of parts and a sufficiently long list of instructions can reproduce itself. Platt pointed out that this suggested there is a threshold for autocatalysis: “this threshold marks an essentially discontinuous change in properties, and that fully-complex molecules larger than this size differ from all smaller ones in a property of central importance for biology.” Thus, self-replication is an emergent property. A modification of this idea has been pursued by Stuart Kauffman with regard to the origin of life, that when a network of chemical reactions is sufficiently large, it becomes self-replicating.

Pattern formation and emergence

Patterns in space and/or time form in fluid dynamics (Rayleigh-Bénard convection and Taylor-Couette flow), laser physics, materials science (dendrites in the formation of solids from liquid melts), biology (morphogenesis), and chemistry (Belousov-Zhabotinsky reactions). External constraints, such as temperature gradients, drive most of these systems out of equilibrium. 

Novelty. 

The parts of the system can be viewed as the molecular constituents or small uniform parts of the system. In either case, the whole system has a property (a pattern) that the parts do not have.

Discontinuity. 

When some parameter becomes larger than a critical value, the system transitions from a uniform state to a non-uniform state. 

Universality. 

Similar patterns, such as convection rolls in fluids, can be observed in diverse systems regardless of the microscopic details of the fluid. Often, there is a single parameter, such as the Reynolds number, which involves a combination of fluid properties, that determines the type of patterns that form. Cross and Hohenberg highlighted how the models and mechanisms of pattern formation across physics, chemistry, and biology have similarities. Turing’s model for pattern formation in biology associated it with concentration gradients of reacting and diffusing molecules. However, Gierer and Meinhardt showed that it is sufficient to have a network with competition between short-range positive feedback and long-range negative feedback. This could occur in a circuit of cellular signals.

Self-organisation. 

The formation of a particular pattern occurs spontaneously, resulting from the interaction of the many components of the system.

Effective theories. 

A crystal growing from a liquid melt can form shapes such as dendrites. This process involves instabilities of the shape of the crystal-liquid interface. The interface dynamics are completely described by a few partial differential equations that can be derived from macroscopic laws of thermodynamics and heat conduction. A helpful review is by Langer. 

Diversity. 

Diverse patterns are observed, particularly in biological systems. In toy models, such as the Turing model, with just a few parameters, a diverse range of patterns, both in time and space, can be produced by varying the parameters. Many repeated iterations can lead to a diversity of structures. This may result from a sensitive dependence on initial conditions and history. For example, every snowflake is different because, as it falls, it passes through a slightly different environment, with small variations in temperature and humidity, compared to others.

Toy models. 

Turing proposed a model for morphogenesis in 1952 that involved two coupled reaction-diffusion equations. Homogeneous concentrations of the two chemicals become unstable when the difference between the two diffusion constants becomes sufficiently large. A two-dimensional version of the model can produce diverse patterns, many resembling those found in animals. However, after more than seventy years of extensive study, many developmental biologists remain sceptical of the relevance of the model, partly because it is not clear whether it has a microscopic basis. Kicheva et al., argue that “pattern formation is an emergent behaviour that results from the coordination of events occurring across molecular, cellular, and tissue scales.” 

Other toy models include Diffusion Limited Aggregation, due to Witten and Sander, and Barnsley’s iterated function system for fractals that produces a pattern like a fern.

Here is a beautiful lecture on Pattern Formation in Biology by Vijaykumar Krishnamurthy

 

John Hopfield on what physics is

A decade ago John Hopfield reflected on his scientific life in Annual Reviews in Condensed Matter Physics, Whatever Happened to Solid State Physics?

"What is physics? To me—growing up with a father and mother who were both physicists—physics was not subject matter. The atom, the troposphere, the nucleus, a piece of glass, the washing machine, my bicycle, the phonograph, a magnet—these were all incidentally the subject matter. The central idea was that the world is understandable, that you should be able to take anything apart, understand the relationships between its constituents, do experiments, and on that basis be able to develop a quantitative understanding of its behavior. 

Physics was a point of view that the world around us is, with effort, ingenuity, and adequate resources, understandable in a predictive and reasonably quantitative fashion. Being a physicist is a dedication to the quest for this kind of understanding."

He describes how this view was worked out in his work in solid state theory and moved into biological physics and the paper for which he was awarded the Nobel Prize. 

"Eventually, my knowledge of spin-glass lore (thanks to a lifetime of interaction with P.W. Anderson), Caltech chemistry computing facilities, and a little neurobiology led to the first paper in which I used the word neuron. It was to provide an entryway to working on neuroscience for many physicists..."

After he started working on biological physics in the late 1970s he got an offer from Chemistry and Biology at Caltech and Princeton Physics suggested he take it. 

"In 1997, I returned to Princeton—in the Molecular Biology Department, which was interested in expanding into neurobiology. Although no one in that department thought of me as anything but a physicist, there was a grudging realization that biology could use an infusion of physics attitudes and viewpoints. I had by then strayed too far from conventional physics to be courted for a position in any physics department. So I was quite astonished in 2003 to be asked by the American Physical Society to be a candidate for vice president. And, I was very happy to be elected and ultimately to serve as the APS president. I had consistently felt that the research I was doing was entirely in the spirit and paradigms of physics, even when disowned by university physics departments."

2024 Nobel Prize in Physics

 I was happy to see John Hopfield was awarded the Nobel Prize in Physics for his work on neural networks. The award is based on this paper from 1982

Neural networks and physical systems with emergent collective computational abilities

One thing I find beautiful about the paper is how Hopfield drew on ideas about spin glasses (many competing interactions lead to many ground states and a complex energy landscape).

A central insight is that an efficient way to store the information describing multiple objects (different collective spin states in an Ising model) is in terms of the inter-spin interaction constants (J_ij's) in the Ising model. These are the "weights" that are trained/learned in computer neural nets.

It should be noted that Hopfield's motivation was not at all to contribute to computer science. It was to understand a problem in biological physics: what is the physical basis for associative memory? 

I have mixed feelings about Geoffrey Hinton sharing the prize.  On the one hand, in his initial work, Hinton used physics ideas (Boltzmann weights) to extend Hopfields ideas so they were useful in computer science. Basically, Hopfield considered a spin glass model at zero temperature and Hinton considered it at non-zero temperature. [Note, the temperature is not physical it is just a parameter in a Boltzmann probability distribution for different states of the neural network]. Hinton certainly deserves lots of prizes, but I am not sure a physics one is appropriate. His work on AI has certainly been helpful for physics research. But so have lots of other advances in computer software and hardware, and those pioneers did not receive a prize.

I feel a bit like I did with Jack Kilby getting a physics prize for his work on integrated circuits. I feel that sometimes the Nobel Committee just wants to remind the world how physics is so relevant to modern technology.

Ten years ago Hopfield wrote a nice scientific autobiography for Annual Reviews in Condensed Matter Physics,

Whatever Happened to Solid State Physics?

After the 2021 Physics Nobel to Parisi, I reflected on the legacy of spin glasses, including the work of Hopfield.

Aside: I once pondered whether a chemist will ever win the Physics prize, given that many condensed matter physicists have won the chemistry prize. Well now, we have had an electronic engineer and a computer scientist winning the Physics prize.

Another side: I think calling Hinton's network a Boltzmann machine is a scientific misnomer. I should add this to my list of people getting credit for things that did not do. Boltzmann never considered networks, spin glasses or computer algorithms. Boltzmann was a genius, but I don't think we should be attaching his name to everything that involves a Boltzmann distribution. To me, this is a bit like calling the Metropolis algorithm for Monte Carlo simulations the Boltzmann algorithm. 

The multi-faceted character of emergence (part 2)

In the previous post, I considered five different characteristics that are often associated with emergence and classified them as being associated with ontology (what is real and observable) rather than epistemology (what we believe to be true). 

Below I consider five more characteristics: self-organisation, unpredictability, irreducibility, contextuality and downward causation, and intra-stratum closure.

6. Self-organisation

Self-organisation is not a property of the system but a mechanism that a theorist says causes an emergent property to come into being. Self-organisation is also referred to as spontaneous order. 

In the social sciences self-organisation is sometimes referred to as an endogenous cause, in contrast to an exogenous cause. There is no external force or agent causing the order, in contrast to order that is imposed externally. For example, suppose that in a city there is no government policy about the price of a loaf of sliced wholemeal bread or on how many loaves that bakers should produce. It is observed that prices are almost always in the range of $4 to $5 per loaf, and that rarely are there bread shortages. This outcome is a result of the self-organisation of the free-market, and economists would say the price range and its stability has an endogenous cause. In contrast, if the government legislated the price range and the production levels that would be an exogenous cause. Friedrich Hayek emphasised the role of spontaneous order in economics. In biology, Stuart Kaufmann equates emergence with spontaneous order and self-organisation.

In physics, the periodicity of the arrangement of atoms in a crystal is a result of self-organisation and has an endogenous cause. In contrast, the periodicity of atoms in an optical lattice is determined by the laser physicist who creates the lattice and so has an exogenous cause.

Self-organisation shows how local interactions can produce global properties. In different words, short-range interactions can lead to long-range order. After decades of debate and study, the Ising model showed that this was possible. Other examples of self-organisation, include flocking of birds and teamwork in ant colonies. There is no director or leader but the system acts “as if” there is. 

7. Unpredictability

Ernst Mayr (This is Biology, p.19) defines emergence as “in a structured system, new properties emerge at higher levels of integration that could not have been predicted from a knowledge of the lower-level components.” Philip Ball also defines emergence in terms of unpredictability (Quanta, 2024).

More broadly, in discussions of emergence, “prediction” is used in three different senses: logical prediction, historical prediction, and dynamical prediction.

Logical prediction (deduction) concerns whether one can predict (calculate) the emergent (novel) property of the whole system solely from a knowledge of all the properties of the parts of the system and their interactions. Logical predictability is one of the most contested characteristics of emergence. Sometimes “predict” is replaced with “difficult to predict”, “extremely difficult to predict”, “impossible to predict”, “almost impossible to predict”, or “possible in principle, but impossible in practice, to predict.” 

As an aside, I note that philosophers distinguish between epistemological emergence and ontological emergence. They are associated with prediction that is "possible in principle, but difficult in practice" and "impossible in principle" respectively.

After an emergent property has been discovered experimentally sometimes it can be understood in terms of the properties of the system parts. In a sense “pre-diction” then becomes “post-diction.” An example is the BCS theory of superconductivity, which provided a posteriori, rather than a priori, understanding. In different words, development of the theory was guided by a knowledge of the phenomena that had already been observed and characterised experimentally. Thus, a keyword in the statement above about logical prediction is “solely”. 

Historical prediction. Most new states of matter discovered by experimentalists were not predicted even though theorists knew the laws that the microscopic components of the system obeyed. Examples include superconductivity (elemental metals, cuprates, iron pnictides, organic charge transfer salts, …), superfluidity in liquid 4He, antiferromagnetism, quasicrystals, and the integer and fractional quantum Hall states.

There are a few exceptions where theorists did predict new states of matter. These include are Bose-Einstein Condensates (BECs) in dilute atomic gases and topological insulators, the Anderson insulator in disordered metals, the Haldane phase in even-integer quantum antiferromagnetic spin chains, and the hexatic phase in two dimensions. It should be noted that prediction of BECs and topological insulators were significantly helped that theorists could predict them starting with Hamiltonians of non-interacting particles. Furthermore, all of these predictions involved working with effective Hamiltonians. None started with microscopic Hamiltonians for specific materials.

Dynamical unpredictability concerns what it means in chaotic dynamical systems, where it relates to sensitivity to initial conditions. I do not see this as an example of emergence as it can occur in systems with only a few degrees of freedom. However, some authors do associate dynamical unpredictability with complexity and emergence.

8. Irreducibility and singularities

An emergent property cannot be reduced to properties of the parts, because if emergence is defined in terms of novelty, the parts do not have the property. 

Emergence is also associated with the problem of theory reduction. Formally, this is the process where a more general theory reduces in a particular mathematical limit to a less general theory. For example, quantum mechanics reduces to classical mechanics in the limit where Planck’s constant goes to zero. Einstein’s theory of special relativity reduces to Newtonian mechanics in the limit where the speeds of massive objects become much less than the speed of light. Theory reduction is a subtle philosophical problem that is arguably poorly understood both by scientists [who oversimplify or trivialise it] and philosophers [who arguably overstate the problems it presents for science producing reliable knowledge]. Subtleties arise because the two different theories usually involve language and concepts that are "incommensurate" with one another. 

Irreducibility is also related to the discontinuities and singularities associated with emergent phenomena. As emphasised independently by Hans Primas and Michael Berry, singularities occur because the mathematics of theory reduction involves singular asymptotic expansions. Primas illustrates this by considering a light wave incident on an object and producing a shadow. The shadow is an emergent property, well described by geometrical optics, but not by the more fundamental theory of Maxwell’s electromagnetism. The two theories are related in the asymptotic limit that the wavelength of light in Maxwell’s theory tends to zero. This example illustrates that theory reduction is compatible with the emergence of novelty. Primas also considers how the Born-Oppenheimer approximation, which is central to solid state theory and quantum chemistry, is associated with a singular asymptotic expansion (in the ratio of the mass of an electron to the mass of an atomic nuclei in the system). 

Berry considers several other examples of theory reduction, including going from general to special relativity, from statistical mechanics to thermodynamics, and from viscous (Navier-Stokes) fluid dynamics to inviscid (Euler) fluid dynamics. He has discussed in detail how the caustics that occur in ray optics are an emergent phenomena and are associated with singular asymptotic expansions in the wave theory.

The philosopher of science Jeremy Butterfield showed rigorously that theory reduction occurred for four specific systems that exhibited emergence, defined by him as a novel and robust property. Thus, novelty is not sufficient for irreducibility.

9. Contextuality and downward causation

Any real system has a context. For example, it has boundary and an environment, both in time and space. In many cases the properties of the system are completely determined by the parts of the system and their interactions. Previous history and boundaries do not matter. However, in some cases the context may have a significant influence on the state of the system. An example is Rayleigh-Bernard convection cells and turbulent flow whose existence and nature are determined by the interaction of the fluid with the container boundaries. A biological example concerns what factors determine the structure, properties, and function that a particular protein (linear chain of amino acids) has. It is now known that the only factor is not just the DNA sequence that encodes for the amino acid sequence, in contradiction to some versions of the Central Dogma of molecular biology.  Other factors may be the type of cell that contains the protein and the network of other proteins in which the particular protein is embedded. Context sometimes matters.

Supervenience is the idea that once the micro level is fixed, macro levels are fixed too. The examples above might be interpreted as evidence against supervenience. Supervenience is used to argue against “the possibility for mental causation above and beyond physical causation.” 

Downward causation is sometimes equated with emergence, particularly in debates about the nature of consciousness. In the context of biology, Denis Noble defines downward causation as when higher level processes can cause changes in lower level properties and processes. He gives examples where physiological effects can switch on and off individual genes or signalling processes in cells, including maternal effects and epigenetics.

10. Intra-stratum closure: informational, causal, and computational

The ideas described below were recently developed by Rosas et al. from a computer science perspective. They defined emergence in terms of universality and discussed its relationship to informational closure, causal closure, and computational closure. Each of these are given a precise technical definition in their paper. Here I give the sense of their definitions. In considering a general system they do not pre-define the micro- and macro- levels of a system but consider how they might be defined so that universality holds, i.e., so that properties at the macro-level are independent of the details of the micro-level (i.e., are universal).

Informational closure means that to predict the dynamics of the system at the macroscale an observer does not need any additional information about the details of the system at the microscale. Equilibrium thermodynamics and fluid dynamics are examples. 

Causal closure means that the system can be controlled at the macroscale without any knowledge of lower-level information. For example, changing the software code that is running on a computer allows one to reliably control the microstate of the hardware of the computer regardless of what is happening with the trajectories of electrons in the computer.

Computational closure is a more technical concept, being defined in terms of “a conceptual device called the ε-(epsilon) machine. This device can exist in some finite set of states and can predict its own future state on the basis of its current one... for an emergent system that is computationally closed, the machines at each level can be constructed by coarse-graining the components on just the level below: They are, “strongly lumpable.” "

Rosas et al., show that informational closure and causal closure are equivalent and that they are more restrictive than computational closure. It is not clear to me how these closures relate to novelty as a definition of emergence.

In summary, emergence means different things to different people. I have listed ten different characteristics that have been associated with emergent properties. They are not all equivalent and so when discussing emergence it is important to be clear about which characteristic one is using to define emergence.

The multi-faceted character of emergence (part 1)

There is more to emergence than novel properties, i.e., where a whole system has a property that the individual components of the system do not have. Here I focus on emergent properties, but in most cases “property” might be replaced with state, phenomenon, or entity. I now discuss ten characteristics often associated with emergence, beyond novelty. Some people include one or more of these characteristics in their definitions of emergence. However, I do not include them in my definition because as I explain some of the characteristics are contentious. Some may not be necessary or sufficient for novel system properties.

The first five characteristics discussed below might be classified as objective (i.e., observable properties of the system) and the second five as subjective (i.e., associated with how an investigator thinks about the system). In different words, the first five are mostly concerned with ontology (what is real) and the second five with epistemology (what we know). The first five characteristics concern discontinuities, universality, diversity, mesoscales, and modification of parts. The second five concern self-organisation, unpredictability, irreducibility, downward causation, and closure. 

1. Discontinuities 

Quantitative changes in the system can become qualitative changes in the system. For example, in condensed matter physics spontaneous symmetry breaking only occurs in the thermodynamic limit (i.e., when the number of particles of the system becomes infinite). More is different. Thus, as a quantitative change in the system size occurs the order parameter becomes non-zero. In a system that undergoes a phase transition at a non-zero temperature, a small change in temperature can lead to the appearance of order and to a new state of matter. For a first-order phase transition, there is discontinuity in properties such as the entropy and density. These discontinuities define a phase boundary in the pressure-temperature diagram. For continuous phase transitions the order parameter is a continuous function of temperature, becoming non-zero at the critical temperature. However the derivative with respect to temperature may be discontinuous and/or thermodynamic properties such as the specific heat and susceptibility associated with the order parameter may approach infinite as the critical temperature is approached.

Two different states of a system are said to be adiabatically connected if one can smoothly deform one state into the other and all the properties of the system also change smoothly. The case of the liquid-gas transition illustrates subtle issues about defining emergence. A discontinuity does not imply a qualitative difference (novelty). On the one hand, there is a discontinuity in the density and entropy of the system as the liquid-gas phase boundary is crossed in the pressure-temperature diagram. On the other hand, there is no qualitative difference between a gas and a liquid. There is only a quantitative difference: the density of the gas is less than the liquid. Albeit sometimes the difference is orders of magnitude. The liquid and gas state can be adiabatically connected. There is a path in the pressure-temperature phase diagram that can be followed to connect the liquid and gas states without any discontinuities in properties.

The ferromagnetic state also raises questions, as illustrated by a debate between Rudolf Peierls and Phil Anderson about whether ferromagnetism exhibits spontaneous symmetry breaking. Anderson argued that it did not as, in contrast to the antiferromagnetic state, a non-zero magnetisation (order parameter) occurs for finite systems and the magnetic order does not change the excitation spectrum, i.e., produce a Goldstone boson. On the other hand, singularities in properties at the Curie temperature (critical temperature for ferromagnetism) only exist in the thermodynamic limit. Also, a small change in the temperature, from just above the Curie temperature to below, can produce a qualitative change, a non-zero magnetisation.

2. Universality

Properties often referred to as emergent are universal in the sense that it is independent of many of the details of the parts of the system. There may be many different systems that can have a particular emergent property. For example, superconductivity is present in metals with a diverse range of crystal structures and chemical compositions. 

Robustness is related to universality. If small changes are made to the composition of the system (for example replacing some of the atoms in the system with atoms of different chemical element) the novel property of the system is still present. In elementary superconductors, introducing non-magnetic impurity atoms has no effect on the superconductivity.

Universality is both a blessing and a curse for theory. Universality can make it easier to develop successful theories because it means that many details need not be included in a theory in order for it to successfully describe an emergent phenomenon. This is why effective theories and toy models can work even better than might be expected. Universality can make theories more powerful because they can describe a wider range of systems. For example, properties of elemental superconductors can be described by BCS theory and by Ginzburg-Landau theory, even though the materials are chemically and structurally diverse. The curse of universality for theory is that universality illustrates the problem of “under-determination of theory”, “over-fitting of data” and “sloppy theories” [Sethna et al.]. A theory can agree with the experiment even when the parameters used in the theory may be quite different from the actual ones. For example, the observed phase diagram of water can be reproduced, sometimes with impressive quantitative detail, by combining classical statistical mechanics with empirical force fields that assume water molecules can be treated purely being composed of point charges.

Suppose we start with a specific microscopic theory and calculate the macroscopic properties of the system, and they agree with experiment. It would then be tempting to think that we have the correct microscopic theory. However, universality suggests this may not be the case.

For example, consider the case of a gas of weakly interacting atoms or molecules. We can treat the gas particles as classical or quantum. Statistical mechanics gives exactly the same equation of state and specific heat capacity for both microscopic descriptions. The only difference may be the Gibbs paradox [the calculated entropy is not an extensive quantity] which is sensitive to whether or not the particles are treated as identical or not. Unlike the zeroth, first, and second law of thermodynamics, the third law does require that the microscopic theory be quantum. Laughlin discusses these issues in terms of “protectorates” that hide “ultimate causes” .  

In some physical systems, universality can be defined in a rigorous technical sense, making use of the concepts and techniques of the renormalisation group and scaling. These techniques provide a method to perform coarse graining, to derive effective theories and effective interactions, and to define universality classes of systems. There are also questions of how universality is related to the robustness of strata, and the independence of effective theories from the coarse-graining procedure.

3. Diversity

Even when a system is composed of a small number of different components and interactions, the large number of possible stable states with qualitatively different properties that the system can have is amazing. Every snowflake is different. Water is found in 18 distinct solid states. All proteins are composed of linear chains of 20 different amino acids. Yet in the human body there are more than 100,000 different proteins and all perform specific biochemical functions. We encounter an incredible diversity of human personalities, cultures, and languages. A stunning case of diversity is life on earth. Billions of different plant and animal species are all an expression of different linear combinations of the four base pairs of DNA: A, G, T, and C.

This diversity is related to the idea that "simple models can describe complex behaviour". One example is Conway’s Game of Life. Another example is how simple Ising models with a few competing interactions can describe a devil's staircase of ground states or the multitude of different atomic orderings found in binary alloys.

Goldenfeld and Kadanoff defined complexity [emergence] as “structure with variations”. Holland (VSI) discusses “perpetual novelty” giving the example of the game of chess, where are typical game may involve the order of 1050 move sequences. “Motifs” are recurring patterns (sequences of moves) in games. 

Condensed matter physics illustrates diversity with the many different states of matter that have been discovered. The underlying microscopics is “just” electrons and atomic nuclei interacting according to Coulomb’s law.

The significance of this diversity might be downplayed by saying that it is just a result of combinatorics. But such a claim overlooks the issue of the stability of the diverse states that are observed. In a system composed of many components each of which can take on a few states the number of possible states of the whole system grows exponentially with the number of components. For example, for a chain of ten amino acids there are 1013 different possible linear sequences. But this does not mean that all these sequences will produce a functional protein, i.e., a molecule that will fold rapidly (on the timescale of milliseconds) into a stable tertiary structure and perform a useful biochemical function such as catalysis of a specific chemical reaction or signal transduction.

4. Simple entities at the mesoscale 

A key idea in condensed matter physics is that of quasi-particles. A system of strongly interacting particles may have excitations, seen in experiments such as inelastic neutron scattering and Angle Resolved PhotoElectron Spectroscopy (ARPES), that can be described as weakly interacting quasi-particles. These entities are composite particles, and have properties that are quantitatively different, and sometimes qualitatively different, from the microscopic particles. Sometimes this means that the scale (size) associated with the quasi-particles is intermediate between the micro- and the macro-scales, i.e., it is a mesoscale. The existence of quasi-particles leads naturally to the technique of constructing an effective Hamiltonian [effective theory] for the system where effective interactions describe the interactions between the quasi-particles.

The economist Herbert Simon argued that a characteristic of a complex system is that the system can be understood in terms of nearly decomposable units. Rosas et al., argue that emergence is associated with there being a scale at which the system is “strongly lumpable”. Denis Noble has highlighted how biological systems are modular, i.e., composed of simple interchangeable components.

5. Modification of parts and their relationships

Emergent properties are often associated with the state of the system exhibiting patterns, order, or structure, terms that may be used interchangeably. This reflects that there is a particular relationship (correlation) between the parts which is different to the relationships in a state without the emergent property. This relationship may also be reflected in a generalised rigidity. For example, in a solid applying a force on one surface results in all the atoms in the solid experiencing a force and moving together. The rigidity of the solid defines a particular relationship between the parts of the system.

Properties of the individual parts may also be different. For example, in a crystal single-atom properties such as electronic energy levels change quantitatively compared to their values for isolated atoms. Properties of finite subsystems are also modified, reflecting a change in interactions between the parts. For example, in a molecular crystal the frequencies associated with intramolecular atomic vibrations are different to their values for isolated molecules. However, emergence is a sufficient but not a necessary condition for these modifications. In gas and liquid states, novelty is not present but there are still such changes in the properties of the individual parts.

As stated at the beginning of this section the five characteristics above might be associated with ontology (what is real) and objective properties of the system that an investigator observes and depend less on what an observer thinks about the system. The next five characteristics might be considered to be more subjective, being concerned with epistemology (how we determine what is true). In making this dichotomy I do not want to gloss over the fuzziness of the distinction or of two thousand years of philosophical debates about the relationship between ontology and epistemology, or between reality and theory.

In the next post, I will discuss the remaining five characteristics: self-organisation, unpredictability, irreducibility, contextuality and downward causation, and intra-stratum closure.

Thanks for reading this far!

Biology is about emergence in subtle ways

Biology is a field that is all about emergence. It exhibits a hierarchy of structures from DNA to proteins to cells to organs to organisms. Phenotypes emerge from genotypes. At each level of the hierarchy (stratum) there are unique entities, phenomena, principles, methods, theories, and sub-fields. But there is more to the story. 

Philip Ball is probably my favourite science writer. Earlier this year, he gave a beautiful lecture at The Royal Institution, What is Life and How does it Work?

The lecture presents the main ideas in his recent book,  How Life Works: A User's Guide to the New Biology

Here are a few things that stood out for me from the lecture.

1. The question, "What is life?" has been and continues to be notoriously difficult to answer.

2. The Central Dogma of Molecular Biology is wrong.

It was originally stated by Francis Crick, and some commonly assumed corollaries of it are wrong. In simple terms, the Dogma states that DNA makes RNA and RNA makes proteins. This is a unique and unidirectional process. For example, a specific code (string of the letters A,G,T, and C) will produce a specific protein (sequence of amino acids) which will naturally fold into a unique structure with a specific biochemical function. 

The central dogma has undergirded the notion that genes determine everything in biology. Everything is bottom-up.

However, Ball gives several counterexamples.

A large fraction of our DNA does not code for proteins.

Many proteins are disordered, i.e., they do not have a unique folded structure.

Aside: An earlier failure of (some versions of) the central dogma was the discovery of reverse transcriptase by the obscure virus club, essential for the development of HIV drugs and covid-19 vaccines.

3. The role of emergence can be quantified in terms of information theory, helping to understand the notion of causal emergence: the cause of large-scale behaviour is not just a sum of micro-causes, i.e., the properties of and interactions between the constituents at smaller scales. Entities at the level of the phenomena are just as important as what occurs at lower levels.

(page 214 in the book). Causal emergence is concerned with fitting the scale of the causes to the scale of the effects.

The figure above is taken from this paper from 2021.

Evolution and emergence: higher order information structure in protein interactomes across the tree of life 

Brennan Klein, Erik Hoel, Anshuman Swain, Ross Griebenow, Michael Levin

The authors quantify casual emergence in protein networks in terms of mutual information (between large and small scales) and effective information (a measure of the certainty in the connectivity of a network).

Aside: These quantitative notions of emergence have been developed more in recent work by Fernando Rosas and collaborators and discussed in a Quanta article by Philip Ball.

4. Context matters.  A particular amino acid sequence does not define a unique protein structure and function. They may depend on the specific cell in which the protein is contained.

5. Causal spreading.  Causality happens at different levels. It does not always happen at the bottom (genetic level). Sometimes it happens at higher levels. And, it can flow up or down.

6. Levels of description matter. This is well illustrated by morphology and the reasons that we have five fingers. This is not determined by genes.

7. Relevance to medicine. There has been a focus on the genetic origin of diseases. However, many diseases, such as cancer, do not predominantly happen at the genetic level. There has been a prejudice to focus on the genetic level, partly because that is where most tools are available. For cancer, focussing on other levels, such as the immune system, may be more fruitful.

8. Metaphors matter. Biology has been dominated by  metaphors such as living things are "machines made from genes" and "computers running a code". However, metaphors are metaphors. They have limitations, particularly as we learn more. All models are wrong, but some are useful. Ball proposes that metaphors from life, including the notion of agency, may be more fruitful.

9. The wisdom of Michael Berry. Ball ends with Berry's saying that the biggest unsolved problem in physics is not about dark matter (or some similar problem), but rather, "If all matter can be described by quantum theory, where does the aliveness of living things come from?" In other words, "Why is living matter so different from other matter?"

There is also an interesting episode of the How To Academy podcast, where Ball is interviewed about the book.

The interplay of ecological and evolutionary dynamics: immigration, extinction, and chaos (and DMFT?)

"Ecological and evolutionary dynamics are intrinsically entwined. On short timescales, ecological interactions determine the fate and impact of new mutants, while on longer timescales evolution shapes the entire community."

 Spatiotemporal ecological chaos enables gradual evolutionary diversification without niches or tradeoffs       Aditya Mahadevan, Michael T Pearce, and Daniel S Fisher

Understanding this interplay is "one of the biggest open problems in evolution and ecology."

New experimental techniques for measuring the properties of large microbial ecosystems have stimulated significant theoretical work, including from some with a background in theoretical condensed matter physics. For an excellent accessible introduction see:

Understanding chaos and diversity in complex ecosystems – insights from statistical physics

This is a nice 2.5-page article by Pankaj Mehta at the Journal Club for Condensed Matter. He clearly introduces an important problem in theoretical ecology and evolution and describes how some recent work has provided new insights using techniques adapted from Dynamical Mean-Field Theory, which was originally developed to describe strongly correlated electron systems. 

Here are just a few highlights of the article. It may be better to just read the actual article.

Fifty years ago, Robert May "argued that the more diverse an ecosystem is (roughly defined as the number of species present), the less stable it becomes." He derived this counter-intuitive result using a simple model and results from Random Matrix Theory. This is an example of an emergent property: qualitative difference occurs as a system of interacting parts becomes sufficiently large.

"One major deficiency of May’s argument is that it does not allow for the possibility that complex ecosystems can self organize through immigration and extinction. The simplest model that contains all these processes is the Generalized [to many species] Lotka-Volterra model (GLV)".

"Despite its simplicity, this equation holds many surprises, especially when the number of species is large".

Another case of how simple models can exhibit complex behaviour.

One special case is when the interactions are reciprocal – how species i affects species j is identical to how species j affects species I. "In the presence of non-reciprocity the system can exhibit complex dynamical behavior including chaos." Understanding this case was an open problem until the two papers reviewed by Mehta. For a detailed but pedagogical introduction see:

Les Houches Lectures on Community Ecology: From Niche Theory to Statistical Mechanics, Wenping Cui, Robert Marsland III, Pankaj Mehta

This is relevant to understanding the origin of the fine grained diversity observed in sequencing experiments of microbial ecosystems.

Aside: de Pirey and Bunin "derive analytic expressions for the steady-state abundance distribution and an analogue of the fluctuation-dissipation theorem for chaotic dynamics relating static and dynamics correlation functions."

"Using a DMFT solution, they derive a number of remarkable predictions... in the chaotic system the species fall into two groups: species at high abundances and species at low abundances near the immigration floor. de Pirey and Bunin show that even in the chaotic regime, the number of high abundance species in the ecosystem will always be less than the May stability bound. This result is quite surprising since it suggests that ecosystems self-organize in such a way that the high abundance species still follow May’s diversity bound even when they are chaotic."