Statistical mechanics is boring
Statistical mechanics (aka statmech) is a part of physics that accounts for large numbers of particles and explains their collective behaviors. These collective behaviors are some of the hottest things in science now, and statmech is one of the methodological pillars of their analysis. Statmech is used to analyze scaling of cities, transmission of diseases, formation of complex networks, and yields of pistachio trees. Statmech explains why deep learning works, what limits the inference of parameters from data, and why the values of parameters are sometimes irrelevant for the outcomes. In fact, all of my work published so far relies on statmech.
There is a problem with statmech: it is extremely boring. I realized this when writing my dissertation last summer: statmech done properly is mostly tedious accounting of combinatorially large state spaces. One professor even told me that you can't publish straight statmech in journals anymore since nobody wants to read that. Instead, you write a paper about applications, and hide your philosophizing deep in the supplement. There is of course also the issue of the discipline literally driving people crazy and obsessed: two early contributors tragically took their own lives (their science was ultimately proven correct), and a more recent one spent a lifetime building a theory so grand that only the first, 1957, paper is ever cited.
Here is a boring example: for over a hundred years, people have been arguing over so-called "Gibbs paradox" of whether particles of different types should be considered distinguishable. It is named after Josiah Willard Gibbs, one of the earliest famous American physicists. In late 19th century Gibbs codified statmech discussion and wrote one of the first textbooks. He did discuss the gedankenexperiment and found no contradiction, but the explanation was confused beyond repair by a very unfortunate turn of phrase (Gibbs wrote in English, so you can't even blame the translator). Yet if today you enter "Gibbs paradox" into Google Scholar, you will find dozens of papers obsessing over it. Ironically, Daan Frenkel opens his paper with "There should be no need to write this article about the Gibbs paradox, but I am afraid that there is."
So what went wrong? Why didn't science sort out the boring parts of statmech and settle them once and for all, as it did with aether and phlogiston? The answer, I believe, is in quantum mechanics: not its content, but its existence and meteoric rise as a research subject. Quantum mechanics purports to explain what the "microscopic particles" actually are deep down, in a reductionist way. This noble pursuit (that transformed our understanding of the world and enabled all of electronics and the modern way of life) distracted us from figuring out the basics of the classical statmech, assuming that everything quantum is automatically more legit. In mid-20th century we got a number of powerful combinatorial results, invented the Monte Carlo method, clarified the relationship between entropy and disorder, and discovered universality.
Yet, experimentally, we arrived in a situation where we need theories for collections of classical objects, but are interested in the particularities rather than universalities. Examples range from colloidal particles to biological macromolecules to even macroscopic objects such as bird flocks and traffic jams. In making these theories, seemingly boring accounting questions keep popping up: Can we treat these particles as distinguishable? Should I care about the bond entropy? Should I sum the unique particle configurations or include the copy numbers? Is it okay if my partition function diverges? In trying to answer these, I first thought that my memory just let me down and I should look up a general "textbook" answer. Yet I failed to find that answer in the first textbook, and the second, and the tenth.
Something about the classical statmech foundations is clearly missing from the textbooks, but there is barely a mechanism to fix it. Science today operates in papers every other month, not monographs every other year. Spending time and effort on these boring parts is not a great investment, especially for junior scientists - but there are a few great shining examples. Meng et al. show that colloidal clusters of same energy can self-assemble at vastly different yields primarily driven by the symmetry number, which often appears to be just an obscure accounting device. Cates and Manoharan highlight the fallacies of transferring quantum intuitions to classical systems in their paper that can only be described as a love letter to classical statmech. Sethna went even further and coined the term undistinguished not in a research paper but in his excellent textbook, which is the best thing that happened to statmech pedagogy in the past 20 years. In a bold move, Sethna relegates thermodynamics (the most boring part of boring statmech, yet the traditional first chapter of any textbook) to the sidelines in order to make space for more exciting and motivating things.
So what to make of this boringness? Is it a good or a bad thing? It is of course bad to require the practitioners, especially the students, to go through the rounds of accounting choices that seem inconsequential to the application at hand. On the other side, sometimes these subtle choices can lead to a change in prediction of an experimentally measurable quantity, and thus become testable, which is ultimately good for science. I would argue that the history of science in the 20th century made statmech boringness a maladaptive trait: originally useful and rigorous but by now a bit of a drag.
How do we turn the maladaptation around? The face of scientific work in the discipline is changing rapidly, as pen-and-paper analytics are supported and increasingly supplanted with numerical and symbolic calculations on everyone's computer. Computers don't make the modeling decisions for us, but can take a lot of tedium out of the derivations. I think the future of statmech then is in new pedagogy with emphasis of interdisciplinarity and model-building, as well as creation of new user-friendly tools that maintain rigor but require fewer pedantic exercises. Ideas like graphic models, tensor networks, and automatic differentiation allow mathematically precise expressions without spending pages upon pages in algebra. It is my hope to illuminate such ideas both in these pages and in peer-reviewed venues.