Does String Theory Actually Describe the World? AI May Be Able to Tell

The original version of this story appeared in Quanta Magazine.

String theory captured the hearts and minds of many physicists decades ago because of a beautiful simplicity. Zoom in far enough on a patch of space, the theory says, and you won’t see a menagerie of particles or jittery quantum fields. There will only be identical strands of energy, vibrating and merging and separating. By the late 1980s, physicists found that these “strings” can cavort in just a handful of ways, raising the tantalizing possibility that physicists could trace the path from dancing strings to the elementary particles of our world. The deepest rumblings of the strings would produce gravitons, hypothetical particles believed to form the gravitational fabric of spacetime. Other vibrations would give rise to electrons, quarks, and neutrinos. String theory was dubbed a “theory of everything.”

“People thought it was just a matter of time until you could compute everything there was to know,” said Anthony Ashmore, a string theorist at Sorbonne University in Paris.

But as physicists studied string theory, they uncovered a hideous complexity.

When they zoomed out from the austere world of strings, every step toward our rich world of particles and forces introduced an exploding number of possibilities. For mathematical consistency, strings need to wriggle through 10-dimensional spacetime. But our world has four dimensions (three of space and one of time), leading string theorists to conclude that the missing six dimensions are tiny—coiled into microscopic shapes resembling loofahs. These imperceptible 6D shapes come in trillions upon trillions of varieties. On those loofahs, strings merge into the familiar ripples of quantum fields, and the formation of these fields could also come about in multitudinous ways. Our universe, then, would consist of the aspects of the fields that spill out from the loofahs into our giant four-dimensional world.

String theorists sought to determine whether the loofahs and fields of string theory can underlie the portfolio of elementary particles found in the real universe. But not only are there an overwhelming number of possibilities to consider—10⁵⁰⁰ especially plausible microscopic configurations, according to one tally—no one could figure out how to zoom out from a specific configuration of dimensions and strings to see what macroworld of particles would emerge.

“Does string theory make unique predictions? Is it really physics? The jury is just still out,” said Lara Anderson, a physicist at Virginia Tech who has spent much of her career trying to link strings with particles.

Now, a fresh generation of researchers has brought a new tool to bear on the old problem: neural networks, the computer programs powering advances in artificial intelligence. In recent months, two teams of physicists and computer scientists have used neural networks to calculate precisely for the first time what sort of macroscopic world would emerge from a specific microscopic world of strings. This long-sought milestone reinvigorates a quest that largely stalled decades ago: the effort to determine whether string theory can actually describe our world.

“We aren’t at the point of saying these are the rules for our universe,” Anderson said. “But it’s a big step in the right direction.”

The Twisted World of Strings

The crucial feature that determines what macroworld emerges from string theory is the arrangement of the six small spatial dimensions.

The simplest such arrangements are intricate 6D shapes called Calabi-Yau manifolds—the objects that resemble loofahs. Named after the late Eugenio Calabi, the mathematician who conjectured their existence in the 1950s, and Shing-Tung Yau, who in the 1970s set out to prove Calabi wrong but ended up doing the opposite, Calabi-Yau manifolds are 6D spaces with two characteristics that make them attractive to physicists.

First, they can host quantum fields with a symmetry known as supersymmetry, and supersymmetric fields are much simpler to study than more irregular fields. Experiments at the Large Hadron Collider have shown that the macroscopic laws of physics are not supersymmetric. But the nature of the microworld beyond the standard model remains unknown. Most string theorists work under the assumption that the universe at that scale is supersymmetric, with some citing physical motivations for believing so while others do so out of mathematical necessity.

Second, Calabi-Yau manifolds are “Ricci-flat.” According to Albert Einstein’s general theory of relativity, the presence of matter or energy bends spacetime, causing so-called Ricci curvature. Calabi-Yau manifolds lack this kind of curvature, though they can (and do) curve in other ways unrelated to their matter and energy contents. To understand Ricci flatness, consider a doughnut, which is a low-dimensional Calabi-Yau manifold. You can unroll a doughnut and represent it on a flat screen on which moving off the right side teleports you to the left side and likewise with top and bottom.

The general game plan for string theory, then, boils down to searching for the specific manifold that would describe the microstructure of spacetime in our universe. One way to search is by picking a plausible 6D doughnut and working out whether it matches the particles we see.

The first step is to work out the right class of 6D doughnuts. Countable features of Calabi-Yau manifolds, such as the number of holes they have, determine the countable features of our world, such as how many distinct matter particles exist. (Our universe has 12.) So researchers start by searching for Calabi-Yau manifolds with the right assortment of countable features to explain the known particles.

Researchers have made steady progress on this step, and over the last couple of years a United Kingdom-based collaboration in particular has refined the art of doughnut selection to a science. Using insight gathered from an assortment of computational techniques in 2019 and 2020, the group identified a handful of formulas that spit out classes of Calabi-Yau manifolds producing what they call “broad brush” versions of the standard model containing the right number of matter particles. These theories tend to produce long-distance forces we don’t see. Still, with these tools, the UK physicists have mostly automated what were once daunting calculations.

“The efficacy of these methods is absolutely staggering,” said Andrei Constantin, a physicist at the University of Oxford who led the discovery of the formulas. These formulas “reduce the time needed for the analysis of string theory models from several months of computational efforts to a split second.”

The second step is harder. String theorists aim to narrow the search beyond the class of Calabi-Yaus and identify one particular manifold. They seek to specify exactly how big it is and the precise location of every curve and dimple. These geometric details are supposed to determine all the remaining features of the macroworld, including precisely how strongly particles interact and exactly what their masses are.

Completing this second step requires knowing the manifold’s metric—a function that can take in any two points on the shape and tell you the distance between them. A familiar metric is the Pythagorean theorem, which encodes the geometry of a 2D plane. But as you move to higher-dimensional, curvy spacetimes, metrics become richer and more complicated descriptions of the geometry. Physicists solved Einstein’s equations to get the metric for a single rotating black hole in our 4D world, but 6D spaces have been out of their league. “It’s one of the saddest things as a physicist that you come across,” said Toby Wiseman, a physicist at Imperial College London. “Mathematics, clever as it is, is quite limited when it comes to actually writing down solutions to equations.”

As a postdoc at Harvard University in the early 2000s, Wiseman heard whispers of the “mythical” metrics of Calabi-Yau manifolds. Yau’s proof that these functions exist helped win him the Fields Medal (the top prize in mathematics), but no one had ever calculated one. At the time, Wiseman was using computers to approximate the metric of spacetimes surrounding exotic black holes. Perhaps, he speculated, computers could also solve for the metrics of Calabi-Yau spacetimes.

“Everyone said, ‘Oh, no, you couldn’t possibly do that,’” Wiseman said. “So me and a brilliant guy, Matthew Headrick, a string theorist, we sat down and showed it could be done.”

Pixelated Manifolds

Wiseman and Headrick (who works at Brandeis University) knew that a Calabi-Yau metric had to solve Einstein’s equations for empty space. A metric satisfying this condition guaranteed that a spacetime was Ricci-flat. Wiseman and Headrick picked four dimensions as a proving ground. Leveraging a numerical technique sometimes taught in high school calculus classes, they showed in 2005 that a 4D Calabi-Yau metric could indeed be approximated. It might not be perfectly flat at every point, but it came extremely close, like a doughnut with a few imperceptible dents.

Around the same time, Simon Donaldson, a prominent mathematician also at Imperial, was also studying Calabi-Yau metrics for mathematical reasons, and he soon worked up another algorithm for approximating metrics. String theorists including Anderson started trying to calculate specific metrics in these ways, but the procedures took a long time and produced overly bumpy doughnuts, which would mess up attempts to make precise particle predictions.

Attempts to complete step 2 died out for nearly a decade. But as researchers focused on step 1 and on solving other problems in string theory, a powerful new technology for approximating functions swept computer science—neural networks, which adjust huge grids of numbers until their values can stand in for some unknown function.

Neural networks found functions that could identify objects in images, translate speech into other languages, and even master humanity’s most complicated board games. When researchers at the artificial intelligence company DeepMind created the AlphaGo algorithm, which in 2016 bested a top human Go player, the physicist Fabian Ruehle took notice.

“I thought, if this thing can outperform the world champion in Go, maybe it can outperform mathematicians, or at least physicists like me,” said Ruehle, who is now at Northeastern University.

Ruehle and collaborators took up the old problem of approximating Calabi-Yau metrics. Anderson and others also revitalized their earlier attempts to overcome step 2. The physicists found that neural networks provided the speed and flexibility that earlier techniques had lacked. The algorithms were able to guess a metric, check the curvature at many thousands of points in 6D space, and repeatedly adjust the guess until the curvature vanished all over the manifold. All the researchers had to do was tweak freely available machine learning packages; by 2020, multiple groups had released custom packages for computing Calabi-Yau metrics.

With the ability to obtain metrics, physicists could finally contemplate the finer features of the large-scale universes corresponding to each manifold. “The first thing I did after I had it, I calculated masses of particles,” Ruehle said.

From Strings to Quarks

In 2021, Ruehle, collaborating with Ashmore, cranked out the masses of exotic heavy particles that depend only on the curves of the Calabi-Yau. But these hypothetical particles would be far too massive to detect. To calculate the masses of familiar particles like electrons—a goal string theorists have chased for decades—the machine learners would have to do more.

Lightweight matter particles acquire their mass through interactions with the Higgs field, a field of energy that extends throughout space. The more a given particle takes notice of the Higgs field, the heavier it is. How strongly each particle interacts with the Higgs is labeled by a quantity called its Yukawa coupling. And in string theory, Yukawa couplings depend on two things. One is the metric of the Calabi-Yau manifold, which is like the shape of the doughnut. The other is the way quantum fields (arising as collections of strings) spread out over the manifold. These quantum fields are a bit like sprinkles; their arrangement is related to the doughnut’s shape but also somewhat independent.

Ruehle and other physicists had released software packages that could get the doughnut shape. The last step was to get the sprinkles—and neural networks proved capable of that task, too. Two teams put all the pieces together earlier this year.

An international collaboration led by Challenger Mishra of the University of Cambridge first used a homegrown neural network to calculate the metric—the geometry of the doughnut itself. Then they harnessed additional original algorithms to compute the way the quantum fields overlap as they curve around the manifold, like the doughnut’s sprinkles. Importantly, they worked in a context where the geometry of the fields and that of the manifold are tightly linked, a setup in which the Yukawa couplings could be calculated in an alternative way, although this had never been done before. When the group calculated the couplings in both manners, the results matched. Moreover, the couplings they found hinted at a separation between particle masses—a mysterious feature of the standard model.

“People have been wanting to do this since before I was born in the ’80s,” Mishra said.

A group led by string theory veterans Burt Ovrut of the University of Pennsylvania and Andre Lukas of Oxford went further. They too started with Ruehle’s metric-calculating software, which Lukas had helped develop. Building on that foundation, they added an array of 11 neural networks to handle the different types of sprinkles. These networks allowed them to calculate an assortment of fields that could take on a richer variety of shapes, creating a more realistic setting that can’t be studied with any other techniques. This army of machines learned the metric and the arrangement of the fields, calculated the Yukawa couplings, and spit out the masses of three types of quarks. It did all this for six differently shaped Calabi-Yau manifolds. “This is the first time anybody has been able to calculate them to that degree of accuracy,” Anderson said.

None of those Calabi-Yaus underlies our universe, because two of the quarks have identical masses, while the six varieties in our world come in three tiers of masses. Rather, the results represent a proof of principle that machine-learning algorithms can take physicists from a Calabi-Yau manifold all the way to specific particle masses.

“Until now, any such calculations would have been unthinkable,” said Constantin, a member of the group based at Oxford.

Numbers Game

The neural networks choke on doughnuts with more than a handful of holes, and researchers would eventually like to study manifolds with hundreds. And so far, the researchers have considered only rather simple quantum fields. To go all the way to the standard model, Ashmore said, “you might need a more sophisticated neural network.”

Bigger challenges loom on the horizon. Attempting to find our particle physics in the solutions of string theory—if it’s in there at all—is a numbers game. The more sprinkle-laden doughnuts you can check, the more likely you are to find a match. After decades of effort, string theorists can finally check doughnuts and compare them with reality: the masses and couplings of the elementary particles we observe. But even the most optimistic theorists recognize that the odds of finding a match by blind luck are cosmically low. The number of Calabi-Yau doughnuts alone may be infinite. “You need to learn how to game the system,” Ruehle said.

One approach is to check thousands of Calabi-Yau manifolds and try to suss out any patterns that could steer the search. By stretching and squeezing the manifolds in different ways, for instance, physicists might develop an intuitive sense of what shapes lead to what particles. “What you really hope is that you have some strong reasoning after looking at particular models,” Ashmore said, “and you stumble into the right model for our world.”

Lukas and colleagues at Oxford plan to start that exploration, prodding their most promising doughnuts and fiddling more with the sprinkles as they try to find a manifold that produces a realistic population of quarks. Constantin believes that they will find a manifold reproducing the masses of the rest of the known particles in a matter of years.

Other string theorists, however, think it’s premature to start scrutinizing individual manifolds. Thomas Van Riet of KU Leuven is a string theorist pursuing the “swampland” research program, which seeks to identify features shared by all mathematically consistent string theory solutions—such as the extreme weakness of gravity relative to the other forces. He and his colleagues aspire to rule out broad swaths of string solutions—that is, possible universes—before they even start to think about specific doughnuts and sprinkles.

“It’s good that people do this machine-learning business, because I’m sure we will need it at some point,” Van Riet said. But first “we need to think about the underlying principles, the patterns. What they’re asking about is the details.”

Plenty of physicists have moved on from string theory to pursue other theories of quantum gravity. And the recent machine-learning developments are unlikely to bring them back. Renate Loll, a physicist at Radboud University in the Netherlands, said that to truly impress, string theorists will need to predict—and confirm—new physical phenomena beyond the standard model. “It is a needle-in-a-haystack search, and I am not sure what we would learn from it even if there was convincing, quantitative evidence that it is possible” to reproduce the standard model, she said. “To make it interesting, there should be some new physical predictions.”

New predictions are indeed the ultimate goal of many of the machine learners. They hope that string theory will prove rather rigid, in the sense that doughnuts matching our universe will have commonalities. These doughnuts might, for instance, all contain a kind of novel particle that could serve as a target for experiments. For now, though, that’s purely aspirational, and it might not pan out.

“String theory is spectacular. Many string theorists are wonderful. But the track record for qualitatively correct statements about the universe is really garbage,” said Nima Arkani-Hamed, a theoretical physicist at the Institute for Advanced Study in Princeton, New Jersey.

Ultimately, the question of what string theory predicts remains open. Now that string theorists are leveraging the power of neural networks to connect the 6D microworlds of strings with the 4D macroworlds of particles, they stand a better chance of someday answering it.

“Without a doubt, there are loads of string theories that have nothing to do with nature,” Anderson said. “The question is: Are there any that do have something to do with it? The answer might be no, but I think it’s really interesting to try to push the theory to decide.”

Original story reprinted with permission from Quanta Magazine, an editorially independent publication of the Simons Foundation whose mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.