‘Gem’ of a Proof Breaks 80-Year-Old Record, Offers New Insights Into Prime Numbers

The original version of this story appeared in Quanta Magazine.

Sometimes mathematicians try to tackle a problem head on, and sometimes they come at it sideways. That’s especially true when the mathematical stakes are high, as with the Riemann hypothesis, whose solution comes with a $1 million reward from the Clay Mathematics Institute. Its proof would give mathematicians much deeper certainty about how prime numbers are distributed, while also implying a host of other consequences—making it arguably the most important open question in math.

Mathematicians have no idea how to prove the Riemann hypothesis. But they can still get useful results just by showing that the number of possible exceptions to it is limited. “In many cases, that can be as good as the Riemann hypothesis itself,” said James Maynard of the University of Oxford. “We can get similar results about prime numbers from this.”

In a breakthrough result posted online in May, Maynard and Larry Guth of the Massachusetts Institute of Technology established a new cap on the number of exceptions of a particular type, finally beating a record that had been set more than 80 years earlier. “It’s a sensational result,” said Henryk Iwaniec of Rutgers University. “It’s very, very, very hard. But it’s a gem.”

The new proof automatically leads to better approximations of how many primes exist in short intervals on the number line, and stands to offer many other insights into how primes behave.

A Careful Sidestep

The Riemann hypothesis is a statement about a central formula in number theory called the Riemann zeta function. The zeta (ζ) function is a generalization of a straightforward sum:

1 + 1/2 + 1/3 + 1/4 + 1/5 + ⋯.

This series will become arbitrarily large as more and more terms are added to it—mathematicians say that it diverges. But if instead you were to sum up

1 + 1/2² + 1/3² + 1/4² + 1/5² + ⋯ = 1 + 1/4 + 1/9+ 1/16 + 1/25 +⋯

you would get π²/6, or about 1.64. Riemann’s surprisingly powerful idea was to turn a series like this into a function, like so:

ζ(s) = 1 + 1/2^s + 1/3^s + 1/4^s + 1/5^s + ⋯.

So ζ(1) is infinite, but ζ(2) = π²/6.

Things get really interesting when you let s be a complex number, which has two parts: a “real” part, which is an everyday number, and an “imaginary” part, which is an everyday number multiplied by the square root of −1 (or i, as mathematicians write it). Complex numbers can be plotted on a plane, with the real part on the x-axis and the imaginary part on the y-axis. Here, for example, is 3 + 4i.

The zeta function takes points on the complex plane as inputs, and it produces other complex numbers as outputs. It turns out that for some complex numbers, the zeta function is equal to zero. Figuring out where those zeros are located on the complex plane is one of the most interesting questions in mathematics.

In 1859, Bernhard Riemann conjectured that all the zeros are concentrated on two lines. If you extend the zeta function so you can compute it for negative inputs, you’ll find that it equals zero for all negative even numbers: −2, −4, −6 and so on. This is relatively easy to show, so these are called trivial zeros. Riemann conjectured that all the other zeros of the function, called nontrivial zeros, have a real part of 1/2, and so are located on this vertical line.

This is the Riemann hypothesis, and proving it has been prohibitively difficult. Mathematicians know that every nontrivial zero must have a real part between zero and 1, but they can’t rule out that some zeros might have a real part of, say, 0.499.

What they can do is show that such zeros must be incredibly rare. In 1940, an English mathematician named Albert Ingham established an upper bound on the number of zeros whose real part is not equal to 1/2 that mathematicians continue to use as a point of reference today.

A few decades later, in the 1960s and ’70s, other mathematicians figured out how to translate Ingham’s result into statements about how clumped or spread out prime numbers are as you move further along the number line, and about other patterns they might form. Around the same time, mathematicians also introduced new techniques that improved Ingham’s bounds for zeros with a real part greater than 3/4.

But it turned out that the most important zeros to cap were those with a real part of exactly 3/4. “Lots of headline results about prime numbers were limited by our understanding of zeros with real part 3/4,” Maynard said.

About a decade ago, Maynard started thinking about how to improve Ingham’s estimate for those particular zeros. “It’s been one of my favorite problems in analytic number theory,” he said. “It always felt tempting that you just have to work a bit harder, and you’ll be able to get an improvement.” But year after year, no matter how many times he came back to it, he kept getting stuck. “It almost sucked you in, and it looked much more innocent than I think it was.”

Then, in early 2020, during a plane trip to a conference in Colorado, an idea came to him. Perhaps, Maynard thought, tools from another area of math called harmonic analysis might be useful.

Larry Guth, an expert in harmonic analysis who was at the same conference, just happened to already be thinking along similar lines. “But I didn’t know the analytic number theory at all well,” he said. Maynard explained the number theory side of the story to him over lunch and gave him a test case to work with. Guth studied it on and off for a few years, only to realize that his techniques from harmonic analysis wouldn’t work.

But he didn’t stop thinking about the problem, and he experimented with new approaches. He got back in touch with Maynard in February. The two started collaborating in earnest, combining their different perspectives. A few months later, they had their result.

A Mathematical Gambit

Guth and Maynard started out by converting the problem they wanted to solve into another one. If you have a zero that doesn’t have a real part of 1/2, then a related function, called a Dirichlet polynomial, must produce a very large output. As a result, proving that there are few exceptions to the Riemann hypothesis is equivalent to showing that the Dirichlet polynomial cannot get large too often.

The mathematicians then performed another act of translation. First, they used the Dirichlet polynomial to build a matrix, or a table of numbers. “Mathematicians love to see matrices, because matrices are one of the things that we understand really well,” Guth said. “You learn to keep your ears open and be ready to see that there are matrices all over the place.”

Matrices can “act on” a mathematical arrow called a vector, which is defined by a length and direction, to produce another vector. They generally change both the length and direction of the vector when they do so. Sometimes there are special vectors that, when run through a matrix, change only in length but not direction. These are called eigenvectors. Mathematicians measure the size of those changes using numbers called eigenvalues.

Guth and Maynard rewrote their problem so that it was now about the largest eigenvalue of their matrix. If they could show that the largest eigenvalue could not get too big, they’d be done. To do that, they used a formula that gave them a complicated sum, and searched for ways to make the positive and negative values in that sum cancel each other out as much as possible. “You have to rearrange the sequence, or look at it from the right angle, in order to see some symmetry that gives some cancellation,” Guth said.

That process involved several surprising steps, including “what’s in my mind the most important idea, which still seems a bit magical to me,” Maynard said. At one point, there was a seemingly obvious step they should have taken to simplify their sum. Instead, they left it in its longer and more complicated form. “We do something that at first sight looks completely stupid. We just refuse to do the standard simplification,” Maynard said. “And this gives up a lot. It means that now we can’t get any easy bound for this sum.”

But in the long run, this turned out to be an advantageous move. “In chess you call it a gambit, where you sacrifice a piece to get a better position on the board,” Maynard said. Guth likened it to playing with a Rubik’s Cube; sometimes you have to undo previous moves and make everything look worse before finding a way to get more colors in the right place.

“You have to be really brave to throw away an obvious improvement and hope that you can recover it later,” said Roger Heath-Brown, a mathematician at Oxford and Maynard’s former adviser. “That goes against everything that I thought you should be doing.”

In fact, he added of his own experiences working on this problem, “now that I think about it, that’s where I got stuck.”

Maynard said that Guth’s expertise as a harmonic analyst rather than a number theorist made this gambit possible. “He doesn’t inherently have these rules drilled into him, so he was more happy to consider things that go against the grain.”

Ultimately, they were able to get a good enough bound on the largest eigenvalue, which in turn translated to a better bound on the number of potential counterexamples to the Riemann hypothesis. Although their work began with the ideas from harmonic analysis that had inspired Guth, the mathematicians were ultimately able to cut those more complicated techniques out of the picture. “Now it looks exactly like the sort of thing I might have tried to do 40 years ago,” Heath-Brown said.

By giving a better bound on the number of zeros with a real part of 3/4, Guth and Maynard automatically proved results about how prime numbers are distributed. For example, estimates of how many primes are found in a given interval get less accurate for shorter intervals. The new work has allowed mathematicians to shorten the intervals in which they can get good estimates.

Mathematicians suspect that the proof will yield improvements to other statements about primes as well. There seems to be room to push Guth and Maynard’s techniques further. But “I feel that these aren’t the right techniques to solve the Riemann hypothesis itself,” Maynard said. “It’s going to need some big idea from somewhere else.”

Original story reprinted with permission from Quanta Magazine, an editorially independent publication of the Simons Foundation whose mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.