The Science of Complexity
Self Organization
Nature is awe-inspiring in its splendor. The closer we inspect, the more miraculous it seems. Rainbows form. Acorns turn into oak trees. On the one hand, it seems inadequate to attribute all this to a miracle. For most of human history, that idea endured. But on the other, we can’t help but wonder how things happen on their own. Things don’t just happen. A sandpile doesn’t organize into a silicon microchip, crunching numbers at staggering speeds. We have to make it so. But wait. Did we make the Amazon jungle happen? No, some things do happen on their own. When we pause to consider, they call for an explanation other than a miracle. Of the things that spontaneously happen, they display several unique characteristics — contingency and self-assembly. An oak tree is a chain of events and conditions that conspired to bring about its existence. It’s in it’s very nature to produce acorns. But an oak tree can’t exist forever. It’s bound to perish. Self-organization is a process of self-assembly that occurs all on its own, puts on a spectacular display, and eventually fades. It’s contingent upon conditions being just right for its emergence or unraveling. It’s qualitatively more than the sum of its parts or the unwritten rules that made the whole assembly possible.
Nature’s Laws
Natural laws govern everything with no exception. They allow self-organization to occur. Though a mathematical model remains elusive, an outline is clear. Self-organization is a sign of complex ordering from a cosmic point of view. Order spontaneously emerges and sustains itself in pockets as disorder expands overall in the cosmos — from quasars and clusters of galaxies to the formation of our solar system and emergence of myriad life forms on Earth. The rules of self-organization, like factory presets, are Mother Nature’s inventions. The rest, like the game of soccer or jurisprudence, are human inventions. We don’t choose to beat our hearts or prefer to heal broken bones. They happen. An acorn explodes into a majestic oak tree in defiance of gravity under the right conditions. Whatever the unspoken rules, a murmuration of starlings wards off predatory falcons, bees swarm and build hives, and neurons self-organize into a three-pound brain that lights up with consciousness and wonders how it can wonder, and so on, all under the right conditions.
Just Right
What are these conditions just right for the complexity to emerge? For instance, we haven’t discovered complex life elsewhere in the cosmos. Can they self-organize with “simple” rules? We don’t have all the answers. But nature offers clues. Consider complex life on Earth, for instance. It requires energy in a usable form for continuous action. It’s necessary for survival. Life seems to thrive at the edge of the critical boundary between order and disorder, dissipating energy. No organism grows or lasts forever. Its growth span is a small fraction of its lifespan. Growth bumps against an invisible ceiling and halts. Organisms pivot to upkeep mode as a lion’s share of consumed energy gets shunted into maintenance instead of growth. As an organism ages, even that gets overwhelmed. Life is an inherent struggle to keep disorder at bay. The ever-encroaching disorder unravels the system and ultimately wins. Life gets snuffed in this struggle. Its vitality away from the encroaching grip of death is a signature of life’s complexity.
How Rules Emerged
Nature has dealt her hand in inventing rules. She has enabled all experiments permissible by her bespoke rules to forge the cosmos. And over here, in our corner, for multicellular and complex life to emerge. The rules have run their course by variation and natural selection. In the fullness of time, some rules stuck to form patterns like a rut formed by a wheelbarrow. The patterns in nature are evidence of her rules. From octopi to primates, oak trees to quasars — they are abundant with knowledge encoded by her rules. We have only to reach out and grasp them. And we have barely begun in our quest.
Toy Model
Discovering nature’s rules involves constructing a toy model. All human knowledge boils down to models. All models are wrong, but some are useful. They all have inherent limitations. We carry a mental model of the world throughout our lives, whether or not we realize it. We invariably rely on it for survival. But it’s incomplete and flawed. Our mental model is not the world. Simple and Complex are subjective qualifiers applicable to the model, not the world. The world is what it is — predictably unpredictable. We get humbled in revising our model to accommodate each new surprise it foists upon us. Simplifying amounts to stripping away what we consider irrelevant. Sometimes, we get lucky. But often, our assumption of spherical cows is all too relevant but won’t suffice. Nature mocks our sincere attempt to corral it. Earthquakes, hurricanes, human brains, the global economy and climate, and oak trees all lie in that category.
Better Models
The most successful model ever proposed is the Standard Model of particle physics. Its phenomenal success is an accurate and comprehensive picture of the fundamental reality of nature, couched in a mathematical framework. It’s not without flaws or limitations. But it sets a very high bar to model after. Before maturing into a framework, it was a grab-bag of ideas that helped understand seemingly unrelated observed phenomena.
We hope complex phenomena fit neatly into an explanatory framework someday, but there’s no guarantee they should. The ones that abstract the essence of nature’s rules as applicable to a broad category of phenomena are the most successful. Most find utility until they don’t. New evidence forces us to reconsider or abandon them in favor of new ones we invent. Science is a messy, winding path toward better mathematical models of nature. The sciences of complexity are no exception to this ideal. They are worth persisting in humanity’s quest to unravel the elegant rules governing her astounding majesty.
Toy Sandpile
The sandpile can’t self-organize into a computer’s microchip but does a decent job of a toy-model to emulate complexity. As with a grab-bag of ideas, it is the simplest computational model to wrap our heads around. It offers a mechanism for how self-organization could occur. It falls under a class of computational models called Cellular Automata (CA), which are fun to play using a modern computer. Thanks to technology, they make light work of the enormous computations involved. Can we avoid the computational burden? Sandpile resists a reductionist divide-and-conquer approach to isolate parts (i.e., grains of sand) as a stand-in for the system (i.e., the pile). A reductionist approach can get away assuming one “and” one make two. Combining individual grains of sand doesn’t get us the pile’s behavior.
Whole > Sum of Its Parts
One and one make eleven when “and” implies something other than addition operation. Each grain, taken alone, obeys the laws of gravity and quantum mechanics at the fundamental level. But there are unwritten rules by which grains interact and collectively obey. Avalanches, storms, dunes, and so on play by these rules. It forces us to compute every step of every grain. In describing a grain’s behavior, it isn’t enough to know local activity of grains in its vicinity. We must appeal to global activity of the entire pile. Not only at this moment but the history of the pile’s existence. If that sounds outlandish, it is. Each step in the pile’s evolution follows logically from the previous step. But there’s no killer Lagrangian that can predict the system’s evolution in time and space. It forces us to drag the entire pile step by step along its evolutionary course. That’s complexity.
Complexity
What is a complex system? There is little consensus among experts in the field because, let’s face it, it’s complex! We know it when we see it. It displays telltale signs. It’s typically a hierarchical network of independent parts organized to form a whole system. The system shows novelty betrayed by parts alone or rules alone — emergent self-similarity or fractal patterns in space (i.e., self-similarity or space-filling) and in time (i.e., pink or 1/f noise) and power laws exhibiting the system’s statistical behavior.
Fractals
When a part of a system resembles the whole at different magnifications, it exhibits fractal spatial patterns. A broccoli is such a pattern. Many florets make a bud. Many buds make a whole head. A tiny floret, a bud, and the head, are similar but each other’s scaled versions. If I showed you a picture, you can’t tell if I cheated by zooming in on a floret. Without placing a penny for scale, I could have photographed a floret instead of a whole head. It’s called scale invariance. When we encounter an invariance in physics, some symmetry lurks within, and a conservation law usually follows. What laws underlie scale invariance?
This remarkable resemblance at each scale is the idea of self-similarity or fractal geometry. A river gets fed by a fractal network of tributaries fed by a network of streams. Water erodes soil. A rut deepens into a spectacular canyon exhibiting fractal patterns, all under just the right conditions.
Other networks, such as a mosaic of faults in tectonic plates, are fractal-like. They release pent-up stress as earthquakes when it exceeds a critical value. Minor rumbles are all too frequent but barely registered. Significant fissures are less frequent. The catastrophic ones that cause tsunamis are once-in-a-lifetime events. It’s all about the frequency of events.
Power Laws
A power law is a mathematical statement of the frequency of events with size. It is indicative of their scale-free nature by degrees of magnification, just like in the image of broccoli. The jagged edges of coastlines, crackles of lightning bolts, clusters of stars, mountain ranges, and clouds are all fractal-like. Biology has an abundance of fractals in trees, heads of broccoli, the cardiovascular system, an immense network of neurons and synapses that form a brain, metabolic rates — the list is endless. Fractal phenomena are widespread. A good model must capture fractals and power laws.
Pink Noise
Imagine streaming your favorite song on your app at different speeds. It’s low and hollow at half or a quarter of its recorded speed (or tempo). Doubling or quadrupling makes it sound like high-pitched squeaks. But what if it sounded similar regardless of the speed? There are such sounds in nature — rumbling thunder, howling wind, the patter of rain, waterfalls, a gentle breeze and so on. Such sounds are complex. In a spectrum of sounds named after colors, white, pink, and brown noise have distinct signatures based on their frequency.
White noise, with its random notes, sounds like a hiss. Brown noise is like Brownian motion. Like pollen suspended in water, it staggers around like a drunk, retaining only a short-term memory of where it’s been. Pink noise lies between the two and is a fingerprint of complexity. It’s a fractal pattern in time, sounding similar at different scales. It’s ubiquity, from heartbeats to firing of neurons, waterfalls to light emitted by quasars, is a sign of complexity. It’s also called flicker noise. Not all sounds are fractals. As does broccoli with magnification, so does the pink noise with tempo. A model worthy of consideration must mimic pink noise.
Escape to Criticality
Life self-organizes to a critical state, all on its own. Imagine a cliff. It escapes farthest from the equilibrium to the ledge and remains in a meta-stable state, just at the edge of tipping over into the abyss below. On one side, a safe distance from the ledge, is a state of equilibrium. It’s boring — easy to retain the memory of events. Nothing much happens. The ocean below, is its opposite. It’s in a state of dynamic equilibrium, a seething hotbed of activity. So much that it’s intractably hard to keep events straight. It resists all memory creation. That is disorder. Without order, there is no retelling a sequence of events as a coherent story. Memory demands a medium of storage and an orderliness of events. Getting pedantic, the dynamic equilibrium of highest disorder, can display no sign of complexity. Objects such as brains or computers cannot exist to perform ( i.e., compute) the act of remembering (or retaining a state). In the far future, that’s the inevitable fate of the cosmos, according to our best understanding. That’s a bleak prospect of the abyss.
Before taking the plunge, let’s pause at the precipice. It’s a state of non-equilibrium. That’s the critical boundary where complexity emerges and sustains. John von Neumann doubted a viable theory of non-equilibrium could exist, dubbing it the Theory of non-elephants. He may have been quick to dismiss that critical but meta-stable ledge. How does a system escape to that ledge? How can it remain there without plunging into the abyss? Meta-stable does not imply instability. The system hangs in there, but only just. Even a tiny push could trigger large-scale changes to bring the system back to criticality by releasing energy. A local nudge may trigger a global domino effect. But, the system prefers to remain there for an extended period, full of vitality and self-organized activity.
Cellular Automata
Most Cellular Automata (CA) models have stuff happening on a line or a two-dimensional grid. It’s just convenience. They can extend to higher dimensions as well. That’s our stick-figure starter pack to build the toy model. What rules apply to the stuff on the grid? That’s guesswork by reasoning and creativity to mimic Mother Nature’s rule-book. Then, let the model evolve. It gets intractable fast to manage many moving parts. Resorting to computer simulation becomes necessary. They may produce interesting patterns or none at all, based on the rules we set. There are many examples of Cellular Automata out there. And they are fun to play with. And literature proving their computational universality abounds. But a single CA model that simulates fractals, pink noise, and criticality is tricky.
There was one bold proposal in 1987. It goes by the title of Abelian Sandpile Model (ASM). Its originators, Per Bak, Chao Tang, and Kurt Wiesenfeld (BTW), called it the Self-Organized Criticality (SOC) model. It has garnered thousands of citations from the scientific community since its publication. It has also stirred up quite a controversy. And it has continued to garner adherents and detractors over the decades. Its detractors owe it to themselves to propose viable alternatives, which is easier said than done. But in the true spirit of Science, constructive criticism is necessary. And it has attracted criticism in spades. That’s the model we will turn to next.
The Sandpile Paradigm
A flat, empty surface serves as a two-dimensional grid. The size of the cells forming the grid is irrelevant, but say they are uniform. The grid can be infinite or finite, such as a tabletop, with its edges serving as a sink for the grains to roll off. Grains of dry sand can drop anywhere but slowly. How slowly doesn’t matter, but we can stipulate it does grain by grain. Grains settle on the table. We can also make another simplifying assumption that grains are identical. If we imagine a grid, each cell (or site) collects sand grains. Some may have none. Grains bouncing on the surface get ignored. We count the grains in each cell at each step of the simulation. The process continues indefinitely. A real sandpile grows, becomes unstable, sinks under its weight, and slides. But in our toy model, we can introduce simple rules of the pile’s growth and slide. The model’s objectives are:
- to know how long a cascade lasts,
- how large the blast radius of a cascade gets, and
- predict where and when a large cascade may occur
That’s the process in a nutshell. Can our simple model emulate criticality, pink noise, fractals, and power laws? As the authors who proposed the model found out, it can.
Here’s the list of rules to introduce:
- Criticality Rule: If the sand-grain-tower grows taller than N grains (= 4, 5, etc.) in any cell on the grid, it must topple (or tumble). Let’s take N = 3 for our model. Once the simulation stops at any arbitrary time, no cell has a tower exceeding N=3 anywhere in the grid.
- Toppling (or Sliding) Rule: Since grains are continuously getting added, there are moments when a cell has N=4 grains and becomes unstable. What’s the rule for toppling or sliding? We stipulate that the cell gives up all grains (N=4), pushing them off, one in each cardinal direction (east, west, south, north) to its neighboring cells. Each neighbor gains a grain. The original cell loses four. If the cells are near the edge, one or more grains could leave the grid, which is allowed.
Simulating the process K x K grid between two arbitrary starting and stopping points reveals patterns worth exploring. The initial configuration can be any random arrangement of grains, but only entries N = {0, 1, 2, 3} are allowed at each cell.
Abelian Group
From an initial grid configuration, which cell gets picked first? We identify all sites with N≥4. There could be more than one cell. In what order (or sequence) should tumbling proceed? Order doesn’t matter. Its insensitivity to order is a pleasant feature — symmetry of the system. The system doesn’t care if this tower topples before that or if ordering gets flipped. It’s a property of Abelian Group, named after its creator, mathematician Niels Henrik Abel. One can get fancy with commutation operators, but all said and done, the system evolves to a steady state because ordering is immaterial. A set of stable configurations fall neatly into two classes — the system is either transient or recurrent. Recurrent implies repeated operations on a configuration will restore the grid to where it started. All intermediate states en route are transient. None of the transient states can be a state of self-organized criticality.
Steady State
The system may never reach a steady state if grains aren’t allowed to leave the grid. Conversely, if the edges leak sand, the grid must arrive at some stable configuration of N < 4 in all cells. It becomes apparent as we observe the central cell(s) and move in an outward spiral crossing each cell. Each must eventually end up with N < 4 grains as sand spreads outward. It may happen after many tumbles, but it’s bound to. And the edges lose the grains. Stability does not depend on which cells are getting piled on. It’s a nice feature for the system not to care where and how much sand is getting poured as long as it’s anywhere on the grid! It’s not too picky. It doesn’t demand finagling by fine-tuning. It eventually relaxes, releasing (or dissipating) energy (the sand grains contain as they make their way out) after piling stops. It’s ready to absorb more energy (sand) after settling down. If we expect it to settle down to a smooth stable state, we are mistaken. Each iteration yields intricate fractal patterns.
Non-local and Deterministic
The system radiates sand out to the edges. A nudge to a tower (N=4) anywhere may result in a large blast radius of cells tumbling and spreading sand. A global domino effect is never far away, just round the corner, as it hinges on a local wisp of nudge. Its dynamics are non-local. That doesn’t imply the cascade is chaotic. Logical rules determine how the system evolves at each step. At any moment, what cells tumble next is predictable. The system is deterministic. Despite that, we can’t know or predict ahead of time (a) how long a cascade may persist or (b) how large the blast radius gets. It’s a hallmark of natural processes like earthquakes. Small fluctuations accumulated over long periods can set off lasting catastrophic consequences, when the system relaxes.
Stationary Density
It’s apparent by the construction of the model (0 ≤ N ≤ 3) that no cell can have negative entries. Some cascades (i.e, sequences) to stable end-states are impossible regardless of the initial configuration. For instance, the grid will never get flushed out completely. If two adjacent cells weren’t empty when the process began, they can never both be empty in the future since one cell tumbles and offers its neighbor a grain. What’s the average cell density (i.e., height) after a long time? It must rise rapidly but cannot exceed 4. Peter Grassberger conjectured that a cell’s density averaged over an ensemble of recurrent configurations to be 𝝆 = 17/8 = 2.125 grains.
Blast Radius and Cascade Duration
Starting from an initial cell that tumbles, what is its blast radius? The blast radius are the cells affected in that cascade (or avalanche). Though we cannot know apriori, we can estimate its lower bound. The probability that it contains K cells is proportional to 1/√K. This estimate gets more accurate with large values of K. It implies small piles are scaled-down versions of large ones by that scaling factor. A sandpile in a dimension (say d = 3) looks like a stack of lower dimensional slices (d=2) except at the center of the grid. The tendency of sand grains to radiate outward may have something to do with this effect. But it’s a curious artifact of the sandpile dynamics. We can’t know how long the cascade lasts (i.e., its lifetime), but the chance it lasts longer than a duration t is proportional to 1/t. Long relaxation times, t are rare. Shorter ones are all too frequent. The exponents (β = -0.5 and β = -1) are power laws governing its (size and duration).
Resilience
Continuously adding sand reliably brings the pile back to a meta-stable state without fail. This process is robust. On average, sand gets pushed out to the edges as the system relaxes. In its relaxation, it passes through accessible transient configurations, settling into a robust recurrent configuration no matter where or how many grains enter and leave. The system gets attracted to the recurrent configurations, which are critical. Their criticality implies the tendency of causing large scale changes when disturbed from those states by addition of more grains. The transient configurations are either sub-critical or super-critical, and the duration or size of the chain reactions are unpredictable but follow power laws. The system is resilient as it wants to return to a critical configuration and remain there. It’s is a feature of many naturally occurring processes, from earthquakes to immune systems.
Thank you for your readership and support.
© Dr. VK. All rights reserved, 2024
References
- Self-organized criticality: an explanation of 1/ƒ noise, Per Bak, Chao Tang, Kurt Wiesenfeld, 1987
- How Nature Works: The Science of Self-Organized Criticality, Per Bak, 1996
- Self-Organized Critical State of Sandpile Automation Models, Deepak Dhar, 1989
- ‘1/f noise’ in music and speech, Richard Voss, John Clarke, 1975
- Fractal Music, Hypercards, and More …, Martin Gardner, 1991
- 25 years of self-organized criticality: concepts and controversies, Nicholas Watkins, Gunnar Pruessner, Sandra Chapman, Norma Crosby, Henrik Jensen, 2015
- Dynamics of Sand, Per Bak, Michael Creutz, 1991
- Complexity, contingency, and criticality, Per Bak, Maya Paczuski, 1995
- What is … a sandpile? Lionel Levine, James Propp, 2010
- Exact integration of height probabilities in the Abelian Sandpile Model, Sergio Caracciolo, Andrea Sportiello, 2012
