Pattern formation and emergence

Patterns in space and/or time form in fluid dynamics (Rayleigh-Bénard convection and Taylor-Couette flow), laser physics, materials science (dendrites in the formation of solids from liquid melts), biology (morphogenesis), and chemistry (Belousov-Zhabotinsky reactions). External constraints, such as temperature gradients, drive most of these systems out of equilibrium. 

Novelty. 

The parts of the system can be viewed as the molecular constituents or small uniform parts of the system. In either case, the whole system has a property (a pattern) that the parts do not have.

Discontinuity. 

When some parameter becomes larger than a critical value, the system transitions from a uniform state to a non-uniform state. 

Universality. 

Similar patterns, such as convection rolls in fluids, can be observed in diverse systems regardless of the microscopic details of the fluid. Often, there is a single parameter, such as the Reynolds number, which involves a combination of fluid properties, that determines the type of patterns that form. Cross and Hohenberg highlighted how the models and mechanisms of pattern formation across physics, chemistry, and biology have similarities. Turing’s model for pattern formation in biology associated it with concentration gradients of reacting and diffusing molecules. However, Gierer and Meinhardt showed that it is sufficient to have a network with competition between short-range positive feedback and long-range negative feedback. This could occur in a circuit of cellular signals.

Self-organisation. 

The formation of a particular pattern occurs spontaneously, resulting from the interaction of the many components of the system.

Effective theories. 

A crystal growing from a liquid melt can form shapes such as dendrites. This process involves instabilities of the shape of the crystal-liquid interface. The interface dynamics are completely described by a few partial differential equations that can be derived from macroscopic laws of thermodynamics and heat conduction. A helpful review is by Langer. 

Diversity. 

Diverse patterns are observed, particularly in biological systems. In toy models, such as the Turing model, with just a few parameters, a diverse range of patterns, both in time and space, can be produced by varying the parameters. Many repeated iterations can lead to a diversity of structures. This may result from a sensitive dependence on initial conditions and history. For example, every snowflake is different because, as it falls, it passes through a slightly different environment, with small variations in temperature and humidity, compared to others.

Toy models. 

Turing proposed a model for morphogenesis in 1952 that involved two coupled reaction-diffusion equations. Homogeneous concentrations of the two chemicals become unstable when the difference between the two diffusion constants becomes sufficiently large. A two-dimensional version of the model can produce diverse patterns, many resembling those found in animals. However, after more than seventy years of extensive study, many developmental biologists remain sceptical of the relevance of the model, partly because it is not clear whether it has a microscopic basis. Kicheva et al., argue that “pattern formation is an emergent behaviour that results from the coordination of events occurring across molecular, cellular, and tissue scales.” 

Other toy models include Diffusion Limited Aggregation, due to Witten and Sander, and Barnsley’s iterated function system for fractals that produces a pattern like a fern.

Here is a beautiful lecture on Pattern Formation in Biology by Vijaykumar Krishnamurthy

 

Emergence in classical optics: caustics and rainbows

                                                                Photo by Chris Lawton on Unsplash

I love seeing patterns such as those above in bodies of water. I did not know that they are an example of emergence, according to Michael Berry, who states:

“A caustic is a collective phenomena, a property of a family of rays that is not present in any individual ray. Probably the most familiar example is the rainbow.”

Caustics are envelopes of families of rays on which the intensity diverges. They occur in media where the refractive index is inhomogeneous. In the image above, there is an interplay of the uneven air-water interface and the difference in the refractive index between air and water. For rainbows, key parameters are the refractive index of the water droplets and the size of the droplets. The caustic is not the "rainbow", i.e., the spectrum of colours, but rather the large light intensity associated with the bow. The spectrum of colours arises because of dispersion (i.e., the refractive index of water depends on the wavelength of the light).

Caustics illustrate several characteristics of emergence properties: novelty, singularities, hierarchies, new scales, effective theories, and universality. 

Novelty. The whole system (a family of light rays) has a property (infinity intensity) that individual light rays do not.

Discontinuities. A caustic defines a spatial boundary across which there are discontinuities in properties.  

Irreducibility and singular limits. Caustics only occur in the theory of geometrical optics which corresponds to the limit where the wavelength of light goes to zero in a wave theory of light. Caustics (singularities) are not present in the wave theory.

Hierarchies. 

a. Light can be treated at the level of rays, scalar waves, and vector waves. At each level, there are qualitatively different singularities: caustics, phase singularities (vortices, wavefront dislocations, nodal lines), and polarisation singularities. 

b. Treating caustics at the level of wave theory, as pioneered by George Bidell Airy, reveals a hierarchy of non-analyticities, and an interference pattern, reflected in the supernumerary part of a rainbow.

New (emergent) scales. An example, is the universal angle of 42 degrees subtended by the rainbow, that was first calculated by Rene Descartes. Airy's wave theory showed that the spacing of the interference fringes shrinks as lambda^2/3.

Effective theories. At each level of the hierarchy, one can define and investigate effective theories. For ray theory, the effective theory is defined by the spatially dependent refractive index n(R)  and the ray action.

Universality. Caustics exist for any kind of waters: light, sound, and matter. They exhibit "structural stability". They fall into equivalence (universality) classes that are defined by the elementary catastrophes enumerated by Rene Thom and Vladimir Arnold and listed in the Table below. Any two members of a class can be smoothly deformed into one another.

The first column in the Table below is the name of the class given by Thom, and the second is the symbol used by Arnold. K is the number of parameters needed to define the class and the associated polynomial, which is given in the last column. 

For this post, I have drawn on several beautiful articles by Michael Berry.  A good place to start may be 

Nature's optics and our understanding of light (2015), which contains the figure I used above of the rainbow.

There is a beautiful description of some of the history and basic physics of the rainbow in Rainbows, Snowflakes, and Quarks: Physics and the World Around Us by Hans Christian Von Baeyer.

The pictures of the rainbow above were taken on a recent visit to Port Moresby in Papua New Guinea. 

Did Turing really "explain" pattern formation?

 Exactly seventy years ago, Alan Turing published a seminal article, in which he proposed a simple reaction-diffusion model for pattern formation in biological systems. The basic idea is that there are two molecules (morphogens) that react with one another chemically and also diffuse through the system.

The potential relevance of the model can be seen by comparing the lower panels below. The left panel is a real fish and the right panel shows the results of a simulation. The figure above is taken from a beautiful review article published a decade ago.

Reaction-Diffusion Model as a Framework for Understanding Biological Pattern Formation  Shigeru Kondo and Takashi Miura

The authors state that the model is not accepted by many experimental biologists and hope their review will lead to a greater engagement with it. Some of the reasons are related to issues in the philosophy of science and how to model complex systems. What is an explanation? What is the role of simple models for complex systems that ignore so many details?

Kondo and Miura point out the universality of the reaction-diffusion model in the sense that a similar model can be derived where the "molecules" are instead a circuit of cellular signals. Diffusion can be replaced by a relay of signals between cells. Alfred Gierer and  Hans Meinhardt in 1972 showed that all that is required is a network with "a short-range positive feedback [competing] with a long-range negative feedback."

A short video from the Sante Fe Institute also provides a helpful introduction including some simulations.

 

There is another problem with Turing's model that is succinctly described in the opening paragraph of a recent PRL. In a system with two molecular species, patterns only form when there is a large difference between the diffusivity of the two molecules. However, this seems unrealistic because one expects the molecules to have comparable diffusivities.

Turing’s Diffusive Threshold in Random Reaction-Diffusion Systems 

Pierre A. Haas and Raymond E. Goldstein 

 In 1952, Turing described the pattern-forming instability that now bears his name [1]: diffusion can destabilize a fixed point of a system of reactions that is stable in well-mixed conditions. Nigh on threescore and ten years on, the contribution of Turing’s mechanism to chemical and biological morphogenesis remains debated, not least because of the diffusive threshold inherent in the mechanism: chemical species in reaction systems are expected to have roughly equal diffusivities, yet Turing instabilities cannot arise at equal diffusivities [2,3]. It remains an open problem to determine the diffusivity difference required for generic systems to undergo this instability, yet this diffusive threshold has been recognized at least since reduced models of the Belousov–Zhabotinsky reaction [4,5] only produced Turing patterns at unphysically large diffusivity differences.

I first became aware of this paper through a commentary by Changbong Hyeon, at the Journal Club for Condensed Matter. It is also helpful because it explains the simple mathematics behind the threshold value of the model parameters for pattern formation. 

Self-organisation in complex fluids

 I am at the beach this week and so a lot of time is spent staring at waves, clouds, sunsets, and patterns in the sand. There is a lot of beauty and a lot of beautiful science, most of which I know only a little about. For example, what is the essential physics and simplest theory that can explain the patterns below?

To start understanding the beautiful patterns seen in natural systems I have found helpful the two-page Quick Study in Physics Today

The universe in a cup of coffee by John Wettlaufer

Your morning java or tea is a rotating, cooling laboratory that reflects the physics of such large-scale phenomena as stellar dynamics and energy transport in Earth’s atmosphere and oceans. 

A nice demonstration is to put the hot liquid in a glass jar and then just add a few drops of cold milk and see the beautiful patterns that emerge.

The key idea is there is a balance between thermal bouyancy (hot air rises) and viscous stresses. This balance can lead to symmetry breaking and self-organisation. In planetary systems rotation can play a significant role, particularly when there is a balance of viscous forces and the coriolis force. This can lead to the formation of vortices. The Quick Study includes snapshops from a video that is worth watching,  supplementary material from this PRL.

The article also discusses the importance of Rayleigh-Benard convection in many geophysical phenomena. Something interesting I learnt is that this is actually a misnomer, as is often the case in science. According to Wikipedia, 

This pattern of convection, whose effects are due solely to a temperature gradient, was first successfully analyzed in 1916 by Lord Rayleigh (1842–1919).^[16] Rayleigh assumed boundary conditions in which the vertical velocity component and temperature disturbance vanish at the top and bottom boundaries (perfect thermal conduction). Those assumptions resulted in the analysis losing any connection with Henri Bénard's experiment. This resulted in discrepancies between theoretical and experimental results until 1958, when John Pearson (1930– ) reworked the problem based on surface tension.^[9] This is what was originally observed by Bénard.

Mathematics, biology, and emergence

Last night I heard a model public lecture about science. The School of Mathematics and Physics at UQ hosted a public lecture at the Queensland State Library. Holly Krieger, a pure mathematician at Cambridge, spoke on the Mathematics of Life. This is part of a biannual lecture series endowed by Kurt Mahler.

The lecture was amazing, both in content and presentation. It was engaging for high school students, and stimulating for experts. I wish I had a video or a copy of the slides. Krieger is well known to some through her Numberphile videos on YouTube. Here are a few things I learned in the lecture.

Mathematics is the language of relationships and patterns.

We forget how even the concept of numbers is abstract. The notion of functions is even more so.

An underlying theme of the lecture was that of emergence: a simple rule describing the interactions between the components of a system lead to collective behaviour (complexity) of the whole system.

Examples were given from biological systems that raise the question: how does the system know to do this?

Swarms of starlings were shown in the short film, The art of flying by Jan van IJken.
How do they move in concert when there is no leader?



Other examples included ant bridges, an experiment with a slime mould that was able to replicate the Japanese transport network (here is the Canadian version), stripes and spots on animals (pattern formation explained with coupled reaction-diffusion equations by Alan Turing).

To illustrate how simple rules lead to complex behaviour, several cellular automata were demonstrated starting with Pascal's triangle and Sierpinski triangle. The latter was connected to biology through the pattern on the shell of a (poisonous) cone snail.

Rule 30 produces patterns similar to those found on the shell. It has periodic patterns such as stripes and aperiodic chaotic patterns.
It seems the new Cambridge train station also has this pattern!

Rule 184 can describe traffic including jamming for medium traffic densities.
The occurrence of a traffic jam does not depend on the initial state or a particular car, but only depends on the density of cars and the interaction (rule) between cars.

A nice video was shown of a traffic shockwave.
When water flows from a tap (faucet) and hits a flat sink bottom at right angles it may produce a "hydraulic jump" such as that shown below.

That is just the first half of the lecture. I may blog later about the second half which concerned chaos, defined as small initial changes leading to significant changes in outcome.

One of the most interesting things for me about the lecture was Krieger's claim that "Emergent complexity isn't everywhere. It can be hard to detect or confirm.'' i.e., just because we see complex behaviour (patterns) does not mean that it is due to emergence. In question time she said that this was in response to some of Wolfram's grand claims in A New Kind of Science, along the lines that everything (consciousness, gravity, continuity, free will, ...) could be explained in terms of discrete computational models such as cellular automata.

I think a more nuanced view is necessary. I agree, along with many others, that Wolfram's grand claims are not justified. But, I do not equate emergent complexity solely with simple rule-based computational models such as cellular automata. Different people do define emergence differently. For example, Sophia Kivelson and Steve Kivelson propose the following definition.

An emergent behavior of a physical system is a qualitative property that can only occur in the limit that the number of microscopic constituents tends to infinity.

This would rule out classifying most of the phenomena described in the lecture as emergent. I disagree with this definition. On the other hand, I am not sure I agree with Krieger's claim. I do think almost anything interesting is emergent: consciousness, critical phenomena, the vacuum in quantum field theory, superconductivity, ...