Quantifying equilibrium emergence
How do we quantify emergence? How can we measure the strength of collective effects and forces that drive ordering? How can we test for the proposed intermediate-scale mechanisms that drive the change in large-scale behaviors? These questions are ubiquitous in the study of complex systems.
One of the first efforts to answer these questions was undertaken by the Soviet physicist Lev Landau. He was interested in a very particular problem of continuous phase transitions: some systems have an abrupt onset of spontaneous order below a critical temperature, with complicated singular behavior at the transition. This order is responsible for such effects as ferromagnetism and superconductivity that found both theoretical and applied importance. Landau published his theory while working in Moscow in 1937 (just as state terror was reaching its peak outside the window).
The central object of Landau's theory is the order parameter, an intermediate-scale metric of system organization. Landau writes down a phenomenological free energy functional that is analytic and obeys the symmetries of the order parameter. These constraints turn out to be strong enough to determine the system behavior close to the phase transition, when the order parameter switches from zero to nonzero. The Landau theory thus causally explains the appearance of large-scale order (e.g. permanent magnetization) by consideration of intermediate-scale order (order parameter).
In physics, we like to think of forces as the primary causal agent that leads to "objects" moving between "places". In first physics classes, we are taught the forces of gravity, string tension, friction. In later classes, we deal with metaphorical forces: electric current is driven by voltage difference, fluid flow is driven by pressure difference. For many forces (but not all), there is a simpler description: we can assign potential energy to "places", and the causal forces are just a gradient of that potential energy.
This next bit will not necessarily sit well with logic purists since it conflates causation with correlation. In equilibrium statistical mechanics, potential energy determines the probability of finding the system in different states, per Boltzmann's exponential formula. This probability can be added up, or coarse-grained, allowed by Kolmogorov's axioms. After coarse-graining, we get the Landau free energy (LFE) as an inverse, or logarithmic, map. If we take a gradient of LFE, it behaves as a force, or a causal agent on a coarser scale, moving larger "objects" between coarser "places". So what again is the origin of the logic chain: is it the elementary events or the coarser sets? These distinctions become very murky in multiscale emergent systems.
As I wrote before, the notion of intermediate scale analysis and coarse graining was in the zeitgeist for a while, but was not pedagogically well explained until Goldenfeld's textbook. The LFE formula has contributions, and indeed trade-offs, between energy, which we understood for a while, and entropy, which is much trickier. While there are different conceptual explanations for what entropy is, they all have to do with a lot of states. Coincidentally, counting a lot of states numerically has been challenging outside of cases where neat combinatorial tricks apply.
The trade-off between energy and entropy expires in systems where there is no energy. A great example is hard colloidal particles that are not allowed to overlap but have no attraction or repulsion otherwise. Such particles spend all 100% of their time in states of zero potential energy, but somehow at higher density those states tend to be ordered. What causes those ordered states? Since the causal agents in physics are forces, it had to be some sort of a force. It could not be a conventional, energy-based, force for the lack of energy, so it was inevitably a collective, emergent effect christened Directional Entropic Force.
Measuring the Directional Entropic Forces in a principled way took Monte Carlo simulations meeting a recovering string theorist. The first brought a systematic way to evaluate the large sum in what is essentially an LFE formula. The second brought a deep intuition about effective interactions by any name. Ultimately phrasings like "Landau free energy" or "effective action" were discarded from the publication as unreadable by the target audience, and the authors settled on coy "effective potential". The gradient of the effective potential is the flamboyant, causal Directional Entropic Force. While the direct interaction of hard particles is an all-or-nothing, abrupt, non-differentiable, singular function, the effective potential behaves much better. It is finite, continuous, smooth, and directly comparable with thermal fluctuations. The effective potential vanishes for dilute systems and shows considerable strength and specific shape for dense systems. When the effective potential gets strong enough, it causes ordering to emerge.
While the paper on Directional Entropic Forces shied away from the term "Landau free energy", the later papers, guided by the now-recovered string theorist and led by yours truly, did not. One of our quests was to map out the ultimate limits of entropy-driven order by stripping down the system entropy to a minimum. In this most barren system, we find that the entropic interaction persists since there is still substantial Landau free energy (or a minor variation of it, anyway). Another, larger quest was the study of design problems, previously thought of as optimization problems. In optimization, one aims to minimize some cost function, find the singular solution that is "best" of the huge design space. We show another way to navigate the design space, by coarse-graining it to a comprehensible intermediate scale. At that scale, solutions are driven not by the cost function alone, but also by the entropy, in a combined drive we call design stress.
Design stress is but the latest name for an emergent mechanism that is interpretable to a particular audience. Directional Entropic Forces, effective potentials, effective actions, Landau free energy all work similarly and rely on coarse-graining: picking some degrees of freedom to care about and letting others do what they want to do. We used to think that coarse-graining requires a strong separation of scales, for the relevant objects to be much "larger" than the irrelevant ones - but even that is not necessary anymore. The Directional Entropic Forces on particles can be caused by other particles, identical in shape and size, so long as there are many of them densely packed. The design stress analysis helps us understand emergent behaviors when we coarse-grain along two orthogonal directions: physical space, or network topology, or even both, or neither!
To exercise due caution, this discussion was somewhat limited to equilibrium phenomena, for which statistics don't depend on time. We also know the underlying basic interactions, we know about the Boltzmann weight, and have a physicsy intuition for how energies, forces, and probabilities convert into each other. But one by one, these boundaries can be pushed. If today we talk about emergent physical phenomena in design problems, what is to limit our understanding of emergence tomorrow?