What is a complex system?

What do we mean when we say a particular system is "complex"? We main have some intuition that it means there are many degrees of freedom and/or that it is hard to understand. "Complexity" is sometimes used as a buzzword, just like "emergence." There are many research institutes that claim to be studying "complex systems" and there is something called "complexity theory". Complexity seems to mean different things to different people.

I am particularly interested in understanding the relationship between emergence and complexity. To do this we first need to be more precise about what we mean by both terms. A concrete question is the following. Consider a system that exhibits emergent properties. Often that will be associated with a hierarchy of scales. For example: atoms, molecules, proteins, DNA, genes, cells, organs, people. The corresponding hierarchy of research fields is physics, chemistry, biochemistry, genetics, cell biology, physiology, psychology. Within physics a hierarchy is quarks and leptons, nuclei and electrons, atoms, molecules, liquid, and fluid. 

In More is Different, Anderson states that as one goes up the hierarchy the system scale and complexity increases. This makes sense when complexity is defined in terms of the number of degrees of freedom in the system (e.g., the size of the Hilbert space needed to describe the complete state of the system). On the other hand, the system state and its dynamics become simpler as one goes up the hierarchy.  The state of the liquid can be described completely in terms of the density, temperature, and the equation of state. The dynamics of the fluid can be described by the Navier-Stokes equation. Although that is hard to solve in the regime of turbulence, the system is still arguably a lot simpler than quantum chromodynamics (QCD)! Thus, we need to be clearer about what we mean by complexity.

To address these issues I found the following article very helpful and stimulating.

What is a complex system? by James Ladyman, James Lambert, and Karoline Wiesner 

It was published in 2013 in a philosophy journal, has been cited more than 800 times, and is co-authored by two philosophers of science and a physicist.

[I just discovered that Ladyman and Wiesner published a book with the same title in 2020. It is an expansion of the 2013 article.].

In 1999 the journal Science had a special issue that focussed on complex systems, with an Introduction entitled, Beyond Reductionism. Eight survey articles covered complexity in physics, chemistry, biology, earth science, and economics.

Ladyman et al., begin by pointing out how each of the authors of these articles chooses different properties to define what complexity is associated. These characteristics include non-linearity, feedback, spontaneous order, robustness and lack of central control, emergence, hierarchical organisation, and numerosity.

The problem is that these characteristics are not equivalent. If we do choose a specific definition for a complex system, the difficult problem then remains of determining whether each of the characteristics above is necessary, sufficient, both, or neither for the system to be complex (as defined). This is similar to what happens with attempts to define emergence.

Information content is sometimes used to quantify complexity. Shannon entropy and Kolmogorov complexity (Sections 3.1, 3.2) are discussed. The latter is also known as algorithmic complexity. This is the length of the shortest computer program (algorithm) that can be written to produce the entity as output. A problem with both these measures are they are non-computable.

Deterministic complexity is different from statistical complexity (Section 3.3). A deterministic measure treats a completely random sequence of 0s and 1s as having maximal complexity. A statistical measure treats a completely random sequence as having minimal complexity. Both Shannon and algorithmic complexity are deterministic.

Section 4 makes some important and helpful distinctions about different measures of complexity.

3 targets of measures: methods used, data obtained, system itself

3 types of measures: difficulty of description, difficulty of creation, or degree of organisation

They then review three distinct measures that have been proposed logical depth (Charles Bennett), thermodynamic depth (Seth Lloyd and Heinz Pagels), and effective complexity (Murray Gell-Mann).

Logical depth and effective complexity are complementary quantities. The Mandelbrot set is example of a system (set of data) that exhibits a complex structure that has a high information content. It is difficult to describe. It has a large logical depth.

Created by Wolfgang Beyer with the program Ultra Fractal 3. 

On the other hand, the effective complexity of the set is quite small since it can be generated using the simple equation

z_n+1 = c + z_n^2

c is a complex number and the Mandelbrot set is the values of c for which the iterative map is bounded.

Ladyman et al, prefer the definition of a complex system below, but do acknowledge its limitations.

(Physical account) A complex system is an ensemble of many elements which are interacting in a disordered way, resulting in robust organisation and memory. 

(Data-driven account) A system is complex if it can generate data series with high  statistical complexity. 

What is statistical complexity? It relates to degrees of pattern and some they refer to as causal state reconstruction. It is applied to data sets, not systems or methods. Central to their definition is the idea of the epsilon-machine, something introduced in a long and very mathematical article from 2001, Computational Mechanics: Pattern and Prediction, Structure and Simplicity, by Shalizi and Crutchfield.

The article concludes with a philosophical question. Do patterns really exist? This relates to debates about scientific realism versus instrumentalism. The authors advocate something known as "rainforest realism", that has been advanced by Daniel Dennett, Don Ross, and David Wallace. A pattern is real if one can construct an epsilon-machine that can simulate the phenomena and predict its behaviour.

I don't have a full appreciation or understanding of where the article ends up. Nevertheless, the journey there is helpful as it clarifies some of the subtleties and complexities (!) of trying to be more precise about what we mean by a "complex system".

A few things condensed matter physics has taught me about science (and life)

We all have a worldview, some way that we look at life and what we observe. There are certain assumptions we tend to operate from, often implicitly. Arguably, our worldview is shaped by our experiences: family, friendships, education, jobs, community organisations, and our cultural context (political, economic, and social).

A significant part of my life experience has been working in universities as a condensed matter physicist and being part of a broader scientific community. Writing a Condensed Matter Physics: A Very Short Introduction crystallised some of my thoughts about what CMP might mean in broader contexts. I am more aware of how my experience in CMP has had a significant influence on the way I view not just the scientific enterprise, but also broader philosophical and social issues. Here are a few concrete examples.

Complex systems. The objects studied in condensed matter physics have many interacting components (atoms). Further, there is an incredible diversity of systems (materials and phenomena) that are studied. Many different properties and parameters are needed to characterise a system and its possible states. There are many different ways of investigating each system. Similarly, almost everything else of interest in science and life is a complex system.

Emergence. This is central to CMP. The whole is greater than the sum of the parts. The whole is qualitatively different from the parts. Related features include robustness, universality, surprises, and the difficulty of making predictions. An emergent perspective can provide insights into other complex systems: from biology to psychology to politics.

Differentiation and integration. A key aspect of describing and understanding a complex system is conceptually breaking it into smaller parts (differentiation), determining how those parts interact with one another, and determining how those interacting parts combine to produce properties of the whole system (integration).

Diversity: The value of multiple perspectives and methods. Due to the complexity of condensed matter systems, multiple methods are needed to characterise their different properties. Due to emergence, there are various scales and hierarchies present. Investigating and describing the system at these different scales provides different perspectives on the system. What does the scientist do with all these different perspectives? Interpretation and synthesis are needed. That is not an easy or clearcut enterprise.

 Navigating the middle ground. The most interesting CMP occurs in an intermediate interaction regime that is challenging theoretically. Insight can be gained by considering two extremes that are more amenable to analysis: weak interaction and strong interaction. I had fun using conservative-liberal political tensions as a metaphor for divisions in the strongly correlated electron community.

The Art of Interpretation. Everything requires interpretation: a phone text message, a newspaper article, a novel, a political event, data from a science experiment, and any scientific theory. With interpretation, we assign meaning and significance to something. How we do this is complex and draws on our worldview, both explicitly and implicitly. Regardless of our best intentions, interpretation always has subjective elements.

Synthesis. Given the diversity of data, perspectives, and interpretation, it is a challenge to synthesise them into some coherent and meaningful whole. All the pieces are rarely consistent with one another. Some will be ignored, some discarded, some considered peripheral, and others central. This synthesis is also an act of interpretation.

All models are wrong but some are useful. One way to understand complex systems is in terms of "simple" models that aim to capture the essential features of certain phenomena. In CMP significant progress (and many Nobel Prizes) has resulted from the proposal and study of such models. There is a zoo of them. Many are named after their main inventor or proponent: Ising, Anderson, Hubbard, Heisenberg, Landau, BCS,... All theories in CMP are also models since they involve some level of approximation, at least in their implementation. These models are all wrong, in the sense that they fail to describe all features and phenomena of the system. But, the best models are useful. Their simplicity makes them amenable to understanding, mathematical analysis, or computer simulation. Furthermore, the models can give insight into the essential physics underlying phenomena, predict trends, or be used to analyse experimental data. 

The autonomy of academic disciplines. Reality is stratified. At each level of the hierarchy, one has unique phenomena, methods, concepts, and theories. Most of these are independent of the details of what happens at lower levels of the hierarchy. Given the richness at each level, I do not preference one discipline as more fundamental or important than the others.

Pragmatic limits to knowledge. We know so much.  We know so little. On the one hand, it is amazing to me how successful CMP has been. We have achieved an excellent understanding, at least qualitatively of many emergent phenomena in systems that are chemically and structurally complex (e.g., liquid crystals and superconductivity in crystals involving many chemical elements). On the other hand, there are systems such as glasses and cuprate superconductors that have been incredibly resistant to understanding. Good research is very hard, even for the brilliant. Gains are often incremental and small. This experience leads me to have sober expectations about what is possible, particularly as one moves from CMP to more complex systems such as human societies, national economies, and brains.

Science is a human endeavour. Humans can be clever, creative, insightful, rational, objective, cooperative, fiercely independent and capable of great things. The achievements of science are a great testimony to the human spirit. Humans can also be stubborn, egotistical, greedy, petty, irrational, ruthlessly competitive, and prone to fads, mistakes and social pressures. Science always happens in a context: social, political, cultural, and economic. Context does not determine scientific outcomes but due to human nature, it can corrupt how science is done.

The humanity of scientists leads to a lack of objectivity captured in Walter Kauzmann's maxim: people will tend to believe what they want to believe rather than what the evidence before them suggests that they should believe. My decades of experience working as a scientist leads me to scepticism about extravagant claims that some scientists make, particularly hype about the potential significance (scientific, technological, or philosophical) of their latest discovery or their field of research. Too often such claims do not stand the test of time.

Humility. This brings together practically everything above. The world is complex, people are complex, and human-world interactions are complex. It is easy to be wrong. We often have a pretty limited perspective of what is going on. 

Are gravity and spacetime really emergent in AdS-CFT?

There is an interesting Scientific American article by Adam Becker

What Is Spacetime Really Made Of?

Spacetime may emerge from a more fundamental reality. Figuring out how could unlock the most urgent goal in physics—a quantum theory of gravity

It considers two different approaches to quantum gravity (loop quantum gravity and AdS-CFT beloved by string theorists). Compared to some Scientific American articles it is moderately balanced and low on hype. The article has a nice engagement with some philosophers of physics. It is clear to me how loop quantum gravity has a natural interpretation that gravity and space-time are emergent. However, that is not clear for AdS-CFT.

 The following paragraph is pertinent.

But there are other ways to interpret the latest findings. The AdS/CFT correspondence is often seen as an example of how spacetime might emerge from a quantum system, but that might not actually be what it shows, according to Alyssa Ney, a philosopher of physics at the University of California, Davis. 

“AdS/CFT gives you this ability to provide a translation manual between facts about the spacetime and facts of the quantum theory,” Ney says. “That’s compatible with the claim that spacetime is emergent, and some quantum theory is fundamental.” 

But the reverse is also true, she says. The correspondence could mean that quantum theory is emergent and spacetime is fundamental—or that neither is fundamental and that there is some even deeper fundamental theory out there. Emergence is a strong claim to make, Ney says, and she is open to the possibility that it is true. “But at least just looking at AdS/CFT, I’m still not seeing a clear argument for emergence.”