Weather, chaos, and emergence

Weather involves many scales of distance, time, and energy. Describing weather means making decisions about what range of scales to focus on. 

BTW. Did you know that there is a cyclone heading for Brisbane right now!

Now, back to physics :)

Key physics involves thermal convection which reflects an interplay of gravity, thermal expansion, viscosity and thermal conduction. This can lead to Rayleigh-Bénard convection and convection cells.

The multiple scales are associated with multiple entities:

-the molecules that make up the fluid

-small volumes of fluid that are in local thermodynamic equilibrium with a well-defined temperature, density, and velocity

-individual convection cells (rolls)

-collections of cells.

At each scale, the corresponding entities can be viewed as emerging from the interacting entities at the next smallest scale. Hence, they are collective degrees of freedom.

In principle, a complete description, including the transition to turbulence, is given by the equations of fluid dynamics, including the Navier-Stokes equation. Despite the apparent simplicity of these equations, making definitive predictions from them remains elusive.

A famous toy model was studied by the meteorologist Edward Lorenz in 1963, in a seminal paper, "Deterministic Nonperiodic Flow." Under the restrictive conditions of considering the dynamics of a single convection roll the model can be derived from the full hydrodynamic equations.

Lorenz's study stimulated the field of chaos theory, and is beautifully described in James Gleick's book Chaos: The Making of a New Science.

Here, I discuss Lorenz's model in the context of emergence.

 

The model consists of (just) three coupled ODEs (ordinary differential equations):

The variables x(t), y(t), and z(t) describe, respectively, the amplitude of the velocity mode, the temperature mode, and the mode measuring the heat flux Nu, the Nusselt number. x and y characterize the roll pattern.

The model has three dimensionless parameters: r, sigma, and b.

r is the ratio of the temperature difference between the hot and cold plate, to its critical value for the onset of convection. It can also be viewed as the ratio of the Rayleigh number to its critical value.

sigma is the Prandtl number, the ratio of the kinematic viscosity to the thermal diffusivity. Sigma is about 0.7 in air and 7 in water. Lorenz used sigma = 10.

b is of order unity and conventionally taken to have the value 8/3. It arises from the nonlinear coupling of the fluid velocity and temperature gradient in the Boussinesq approximation.

The model is a toy model because for values of r larger than r_c (defined below) "the three mode approximation for the PDEs describing thermal convection... ceased to be physically

realistic, but mathematically the model now starts to show its most fascinating properties,.."

Novelty

The model has several distinct types of long-time dynamics: stable fixed points (no convection), limit cycles (convective rolls), and most strikingly a chaotic strange attractor (represented below). The chaos is reflected in the sensitive dependence on initial conditions.

Briefly, a strange attractor is a curve of infinite length that never crosses itself and is contained in a finite volume. This means it has a fractal structure and a non-trivial Hausdorff dimension [calculated in this paper to be 2.0627160].

Discontinuities

Quantitative changes lead to qualitative changes. For r < 1, no convection occurs. For r > 1, convective rolls develop, but these become unstable for 

and a strange attractor develops.

Phase diagram

Lorenz only considered one set of parameter values [r =28, sigma=10, and b=8/3]. This was rather fortunate, because then strange attractor was waiting to be discovered. 

The phase diagram maps out the qualitatively different behaviours that occur as a function of sigma (vertical axis) and r (horizontal axis). 

Different phases are the fixed points P± associated with convective rolls (black), orbits of period 2 (red), period 4 (green), period 8 (blue), and chaotic attractors (white).

EXTENDED PHASE DIAGRAM OF THE LORENZ MODEL

H. R. DULLIN, S. SCHMIDT, P. H. RICHTER, and S. K. GROSSMANN

Universality

The details of the molecular composition of the fluid and the intermolecular interactions are irrelevant beyond how they determine the three parameters in the model. Hence, qualitatively similar behaviour can occur in systems with a wide range of chemical compositions and physical properties.

Unpredictability

Although the system of three ODEs is simple, discovery of the strange attractor and the chaotic dynamics was unanticipated. Furthermore, the dynamics in the chaotic regime are unpredictable, given the sensitivity to initial conditions.

Top-down causation

The properties and behaviour of the system are not just determined by the properties of the molecules and their interactions. The external boundary conditions, the applied temperature gradient and the spatial separation L of the hot and cold plates, are just as important in determining the dynamics of the system, including motion as much smaller length scales.

Self-organised criticality and emergence in economics

A nice preprint illustrates how emergence is central to some of the biggest questions in economics and finance. Emergent phenomena occur as many economic agents interact resulting in a system with properties that the individual agents do not have.

The Self-Organized Criticality Paradigm in Economics & Finance

Jean-Philippe Bouchaud

The paper illustrates several key characteristics of emergence (novel properties, universality, unpredictability, ...) and the value of toy models in elucidating it. Furthermore, it illustrates the elusive nature of the "holy grail" of controlling emergent properties. 

The basic idea of self-organised criticality

"The seminal idea of Per Bak is to think of model parameters themselves as dynamical variables, in such a way that the system spontaneously evolves towards the critical point, or at least visits its neighbourhood frequently enough"

A key property of systems exhibiting criticality is power laws in the probability distribution of a property. This means that there are "fat tails" in the probability distribution and extreme events are much more likely than in a system with a Gaussian probability distribution.

Big questions

The two questions below are similar in that they concern the puzzle of how markets produce fluctuations that are much larger than expected when one tries to explain their behaviour in terms of the choices of individual agents.

A big question in economics

"A longstanding puzzle in business cycle analysis is that large fluctuations in aggregate economic activity sometimes arise from what appear to be relatively small impulses. For example, large swings in investment spending and output have been attributed to changes in monetary policy that had very modest effects on long-term real interest rates."

This is the "small shocks, large business cycle puzzle", a term coined by Ben Bernanke, Mark Gertler and Simon Gilchrist in a 1996 paper. It begins with the paragraph above. [Bernanke shared the 2022 Nobel Prize in Economics for his work on business cycles].

A big question in finance

The excess volatility puzzle in financial markets was identified by Robert Shiller: The volatility "is at least five times larger than it "should" be in the absence of feedback". In the views of some, this puzzle highlights the failings of the efficient market hypothesis and the rationality of investors, two foundations of neoclassical economics. [Shiller shared the 2013 Nobel Prize in Economics for this work]. 

"Asset prices frequently undergo large jumps for no particular reason, when financial economics asserts that only unexpected news can move prices. Volatility is an intermittent, scale-invariant process that resembles the velocity field in turbulent flows..." (page 2)

Emergent properties

Close to a critical point, the system is characterised by fat-tailed fluctuations and long memory correlations.

Avalanches. They allow very small perturbations to generate large disruptions.

Dragon Kings

Minsky moment

The holy grail: control of emergent properties

It would be nice to understand superconductivity well enough  to design a room-temperature superconductor. But, this pales in significance compared to the "holy grail" of being about to manage economic markets to prevent bubbles, crashes, and recessions.

Bouchaud argues that  the quest for efficiency and the necessity of resilience may be mutually incompatible. This is because markets may tend towards self-organised criticality which is characterised by fragility and unpredictability (Black swans).

The paper has the following conclusion

"the main policy consequence of fragility in socio-economic systems is that any welfare function that system operators, policy makers of regulators seek to optimize should contain a measure of the robustness of the solution to small perturbations, or to the uncertainty about parameters value.

Adding such a resilience penalty will for sure increase costs and degrade strict economic performance, but will keep the solution at a safe distance away from the cliff edge. As argued by Taleb [159], and also using a different language in Ref. [160], good policies should ideally lead to “anti-fragile” systems, i.e., systems that spontaneously improve when buffeted by large shocks."

Toy models

Toy models are key to understanding emergent phenomena. They ignore almost all details to the point that critics claim that the models are oversimplified. The modest goal of their proponents is simply to identify what ingredients may be essential for a phenomenon to occur. Bouchaud reviews several such models. All provide significant insight.

A trivial example (Section 2.1)

He considers an Ornstein-Uhlenbeck process for a system relaxing to equilibrium. As the damping rate tends to zero [κ⋆ → 0], the relaxation time and the variance of fluctuations diverge at the same rate. In other words, "in the limit of marginal stability κ⋆ →0, the system both amplifies exogenous shocks [i.e., those originating outside the system] and becomes auto-correlated over very long time scales."

The critical branching transition (Section 2.2)

The model describes diverse systems: "sand pile avalanches, brain activity, epidemic propagation, default/bankruptcy waves, word of mouth,..."

The model involves the parameter R0 which became famous during the COVID-19 pandemic. R0 is the average number of uninfected people who become infected due to contact with an infected individual. For sand piles R0 is the average number of grains that start rolling in response to a single rolling grain.

when R0 = 1 the distribution of avalanche sizes is a scale-free, power-law distribution 1/S^3/2, with infinite mean.

"most avalanches are of small size, although some can be very large. In other words, the system looks stable, but occasionally goes haywire with no apparent cause."

A generalised Lotka-Volterra model (Sections 3.3 and 4.2) 

This provides an analogue between economic production networks and ecology. Last year I reviewed recent work on this model, concerning how to understand the interplay of evolution and ecology.

A key result is how in the large N limit (i.e., a large number of interacting species/agents) qualitatively different behaviour occurs. Ecosystems and economies can collapse. 

 "any small change in the fitness of one species can have dramatic consequences on the whole system – in the present case, mass extinctions...

"most complex optimisation systems are, in a sense, fragile, as the solution to the optimisation problem is highly sensitive to the precise value of the parameters of the specific instance one wants to solve, like the Aij entries in the Lotka-Volterra model. Small changes of these parameters can completely upend the structure of the optimal state, and trigger large-scale rearrangements,..." 

Balancing stick problem (Section 3.4)

 The better one is able to stabilize the system, the more difficult it becomes to predict its future evolution! 

Propagation of production delays along the supply chain (Section 4.1)

An agent-based firm network model (Section 4.3)

This has the phase diagram shown below. The horizontal axis is the strength of forces counteracting supply/demand and profit imbalances. The vertical axis is the perishability of goods.

There are four distinct phases.

Leftmost region (a, violet): the economy collapses; 

Middle region (b, blue): the economy reaches equilibrium relatively quickly;

Right region (c, yellow): the economy is in perpetual disequilibrium, with purely endogenous fluctuations. 

The green vertical sliver (d) corresponds to a deflationary equilibrium

Phase diagrams illustrate how quantitative changes can produce qualitative differences.

Universality

The toy models considered describe emergent phenomena in diverse systems, including in fields other than economics and finance. 

Here are a few other recent papers by Bouchaud that are relevant to this discussion.

Navigating through Economic Complexity: Phase Diagrams & Parameter Sloppiness

From statistical physics to social sciences: the pitfalls of multi-disciplinarity

This includes the opening address from a workshop on "More is Different" at the College de France in 2022.

Emergence and the stratification of physics into sub-fields

The concept of emergence is central to understanding sub-fields of physics and how they are related, and not related, to other sub-fields.

The table below shows a stratum of sub-disciplines of physics. For each strata there are a range of length, time, and energy scales that are relevant. There are distinct entities that are composed of the entities from lower strata. These composite entities interact with one another via effective interactions that arise due to the interactions present at lower strata and can be described by an effective theory. Each sub-discipline of physics is semi-autonomous. Collective phenomena associated with a single strata can be studied, described, and understood without reference to lower strata.

Table entries are not meant to be exhaustive but to illustrate how emergence is central to understanding sub-fields of physics and how they are related to one another.

What do you think of the table? Is it helpful? Have you seen something like this before?

I welcome suggestions about entries that I could add.

Emergence and singular asymptotic expansions, II

When is a phenomena truly emergent?

Is there some objective quantitative criteria that one might use to decide?

This is an issue because sometimes discussions of emergence are pretty fuzzy and even flaky.

I have now read the full article Asymptotics, Singularities, and the Reduction of Theories 

by Michael Berry that I mentioned in passing in a previous post.

I highly recommend the article as I think it has a very important insight: singular asymptotic expansions provide a concrete criteria for emergence.

Berry considers the specific problem:

He then discusses these examples in detail, including discussions of the asymptotic expansions.

I recommend reading this article before the one by Hans Primas (reviewed in the previous post) as the latter is more technical and philosophical than Berry's.

One thing I think this highlights is that the problem of emergence in quantum systems is neither more or less challenging or interesting than in classical systems, something I argued before.

I have one minor addition to Berry. In quantum many-body systems the singular parameter delta may not just be 1/N, where N = number of particles. It can also be the coupling constant, lambda.  Emergent phenomena are associated with non-perturbative effects. Concrete examples are in the BCS theory of superconductivity and the Kondo effect. In both there is an emergent energy scale  exp(-1/lambda). There is no convergent expansion in powers of lambda. Taylor series around lambda =0 is singular.

Is hydrodynamics ever relevant in metals?

This is a very subtle question. I learnt a lot from a nice talk that Steve Kivelson gave on the subject.

In a single component fluid [e.g. water or a gas] a hydrodynamic approach works because one has local conversation of energy and momentum. Then ALL transport properties are determined by just three quantities: the shear viscosity (eta), the second viscosity (zeta), and the thermal conductivity (kappa).

However, the electron fluid in a solid can exchange energy and momentum with the lattice and impurities. Hence, hydrodynamics is not relevant.

What about temperature ranges where electron-electron scattering dominates? Well then one has resistivity as a result of Umklapp scattering, which means that there is momentum exchange with the lattice. [I never understand all the subtle details of that]. Hence, one still does not have conditions under which hydrodynamics may apply.

So when might hydrodynamics apply? Kivelson suggests it may be relevant in 2DEGS [2-Dimensional Electron Gases] in semiconductor heterostructures close to the metal-insulator transition. Then the lattice is irrelevant and the electron-electron scattering is dominant. What causes resistivity? It can be collision of the fluid with "large" objects such as some slowly varying background impurity potential. Andreev, Kivelson, and Spivak calculated the two-dimensional resistivity as

where the averaging is over space and n0 and s0 denote the equilibrium density and entropy. An important point is that the resistivity is proportional to the viscosity. In simple kinetic theory this is proportional to the scattering time. In a Fermi liquid this will increase with decreasing temperature. Thus the resistivity will have the opposite temperature dependence to a conventional metal!

Hydrodynamics in metals opens the possibility of turbulence. Signatures of this will be nonlinear I-V characteristics, dependence on geometry, and noise.

Kivelson also highlighted relevant recent work by his string theory colleagues on breakdown of the Wiedemann-Franz law in non-Fermi liquids.

A previous post considered the problem of the viscosity of bad metals.

Distinguishing quantum and classical turbulence

Classical turbulence is hard enough to understand. How about turbulence in a quantum fluid such as superfluid helium?
Is there any difference?
There is a nice viewpoint Reconnecting to superfluid turbulence which is a commentary on the 2008 PRL Velocity Statistics Distinguish Quantum Turbulence from Classical Turbulence.
A key difference between the quantum and classical case concerns the reconnection of vortices.

RG theory of turbulence

The renormalisation group and scaling has proven to be an extremely powerful technique in theoretical physics. It has even been succesfully  applied to turbulence. A paper by Yakhot and Orczag in the Journal of Scientific Computing (!) is widely cited.

Can elephants fly?

This week BIPH3001 is reading Life in the slow lane: The Low Reynolds-Number World,

chapter 5 in Biological Physics: Energy, Information, Life, by Phil Nelson.

He begins with the following great quote

Nobody is silly enough to think that an elephant will only fall under gravity if its genes tell it to do so, but the same underlying error can easily be made in less obvious circumstances. So [we must] distinguish between how much behavior, and what part, has a genetic origin, and how much comes solely because an organism lives in the physical universe and is therefore bound by physical laws.

– Ian Stewart, Life’s Other Secret

As with each chapter Nelson begins with a Biological question and a Physical idea:

Biological question: Why do bacteria swim differently from fish?

Physical idea: The equations of motion appropriate to the nanoworld behave differently under time reversal from those of the macroworld.

Figure 5.1 is a picture showing the peculiar character of laminar flow characteristic of a Reynolds number less than one. A really cool video of the same experiment is here.

I also enjoyed a video on Reynolds Number from Sixty Symbols which includes the image above of vortex-antivortex pairs created after a volcano eruption, taken by NASA.

A turbulent claim?

On Friday we had a nice clear and stimulating physics colloquium, Turbulent times in quantum physics from Brian Anderson.

What are unique characteristics of turbulence?

A beautiful video of a dragon fly in fluid flow was shown to illustrate this.

1. continuous flow

2. unpredictable flow details

3. eddy formation, interaction

4. rapid mixing

5. energy input at one length scale and energy dissipation at another length scale.

The latter is described in a landmark paper from 1941 by Kolmorgorov. He used dimensional analysis to show that the kinetic energy spectrum

E(k) ~ k^-5/3 where k is the wave vector.

A superfluid has no viscosity. But turbulence is still possible. Feynman suggested in 1955 that this could arise as a disordered tangle of vortices.

Three features of quantum turbulence

1. dynamics is described by a quantum dynamical equation (e.g., a non-linear Schrodinger equation) rather than the Navier-Stokes equation.

2. Kolmogorov scaling (this was observed in 1998)

3. disordered tangled arrangement of vortices

BECs have "high potential" for step-by-step construction of a quantum turbulent state.

There are only a million atoms in the BECs studied here.

[But isnt this just 100^3? What is the max. no of vortices one could put in such a small system, 100?]

Spontaneous vortices can be produced with a temperature quench.

It was claimed that dissociation of vortex-antivortex pairs is related to quantum turbulence. However, in two dimensions this dissociation is just the Kosterlitz-Thouless transition which I doubt this has anything to do with quantum turbulence.

Quantum vs. classical turbulence in two dimensions was discussed.

Jupiter's great red spot is considered to be an example of the latter.

Two dimensions leads to different kinetic energy scaling for quantum turbulence, E(k) ~ k^ -3 for large k

Numerical simulations claim to see a crossover to this scaling [However, the graph shown did not appear to have a horizontal scale and so one could not see how may decades of k this covered].

The take home point of the talk was meant to be:

Atomic quantum fluids are enabling advances in difficult physics problems that are relevant beyond quantum physics labs.

However, I failed to see these advances from the talk. The experiments are beautiful and fascinating. But, I could not see how the experiments or simulations have led to any new insights or advances beyond those from Kolmogorov in 1941 and Feynman in 1955. To me this is another example of how people in the BEC community oversell the significance of their work. Potential advances and hoped for insights are not the same as real advances and insights.

For an example of a real advance in a difficult problem which spread across disciplines consider the case of the Hopfield net, which was influenced by ideas from spin glasses in condensed matter physics. This had a large influence on neural networks in computer science and biology. The fact that Hopfield is now a Professor of Molecular Biology at Princeton is a testimony to the advances he made.

Chemical Engineering departments now regularly hire faculty who do research using density functional theory (DFT). This is testimony to the advances that have been made in modelling real materials and chemical processes using quantum chemical methods.

When departments of Aeronautical and Mechanical Engineering hire people to work on quantum turbulence will be a real sign of a significant contribution.