Gödel's Lost Letter and P=NP

Twin Primes Are Useful

Pip — Wed, 22 May 2013 01:10:07 +0000

Why the recent breakthrough is important

photo sources: UNH and Simons article

Yitang Zhang, of the University of New Hampshire, has apparently proved a finite approximation to the famous Twin Prime Conjecture. This is a result of the first order. After ten days of progressively more detailed news, including today’s article in the New York Times, Zhang’s 56-page preprint has just been released on the Annals of Mathematics journal website. This is the final accepted version of the original submission, which was said in a story in Nature last week to have needed only “a few minor tweaks.”

Today Ken and I want to explain important aspects of the Twin Prime Conjecture.

Recall that the Twin Prime Conjecture states that there are an infinite number of primes and that are as close as possible: . Well 3 and 2 are closer, but that can only happen once, so the best one can hope for is primes that are within two of each other.

Zhang’s beautiful result is that there are an infinite number of primes and so that and is bounded by an absolute constant. The constant is large—in the tens of millions—but it is a constant. Perhaps we should call these “Cousin Primes,” since they are not within two; I will leave the naming of them to the number theorists. Whatever you call them, his result is a huge improvement to what was previously proved, and is a singular advance.

The proof is long, which is not unexpected. Ken saw the news of the paper’s release earlier today on Terry Tao’s Google+ page about the work, which gives some idea of how the proof goes. There are many links and comments in a post by Peter Woit that also mentions a recently announced proof by Harald Helfgott of the “ternary Goldbach conjecture” that every odd number above 5 is the sum of three primes.

So what can we possibly add to the discussion about Zhang’s breakthrough? Nothing on the proof right now. Something, however, on why the Twin Prime Conjecture can be really useful. This is all from a personal point of view, but one that I hope you will enjoy. Let’s first take a quick look at what was known before his work, and then discuss what it may be useful for.

Before Zhang

One measure of the density of the primes is that the summation

does not converge. That is, the sum

tends to infinity as tends to infinity. The growth is slow, but the sum does diverge. In 1915, Viggo Brun used sieve methods to prove that twin primes were rarer in a precise sense: the summation over twin primes

converges. Indeed his result can be improved to show that the number of twin primes less that is bounded above by

Using heuristic arguments, Godfrey Hardy and John Littlewood guessed that not only are there an infinite number of twin primes, but that there density is close to what a “random” model would predict. Let be the number of twin primes less than —recall is used to denote the number of primes less than —then the Hardy-Littlewood Conjecture is that

for an explicit constant

Tao is on record as saying that certain approaches based on sieve theory cannot resolve the Twin Prime conjecture—see this for a short discussion. Mark Lewko, in a comment to a MathOverflow thread on Zhang’s paper, indicates that its mechanism alone cannot reduce the gap under , and it does not circumvent a more general obstacle to gaps below . However, even if Zhang’s new techniques do not overcome such general barriers, at least they push against them with a lot more oomph.

Another Problem

Years ago I worked on a problem that had nothing directly to do with the Twin Prime Conjecture. The question is a fundamental one about the complexity of Boolean functions. It is not classic, has not been open for a very long time, and could be trivial. But like many problems about Boolean functions it turned out to fight back hard, and the best we could do was to make a small dent in the problem.

The work was joint with Mihail Kolountzakis, Evangelos Markakis, Aranyak Mehta, and Nisheeth Vishnoi. See the paper for details. Indeed while I helped start the research with Markakis, Mehta, and Vishnoi, without Kolountzakis the work would have never been completed. Our result was then greatly improved by Amir Shpilka and Avishay Tal, as we covered here.

The problem is concretely about Boolean functions of variables, and seems not to involve prime numbers at all. For any subset of the coordinates, the corresponding Fourier coefficient is given by

where is if is odd, and otherwise. The problem is:

What is the smallest value such that for every symmetric 0-1 function on that is not affine linear—by symmetry, this means neither constant nor a parity function—some with does not vanish?

The Prime Gap Trick

A heuristic that I have used before is this: When trying to prove some theorem in number theory, assume any reasonable property that you need about primes. Either you will at least get a conditional theorem, or you might later be able to weaken the assumption you made. The latter is what happened to us. Or you might be luckier and get your theorem to follow both from the assumption and its negation, so that you don’t need it at all. I noted once how Littlewood did this with the Riemann Hypothesis—incidentally that post was about a theorem of Ravi Kannan, and I am attending a birthday workshop in his honor later this week.

In this case we reduced the problem to showing that a certain integer-valued polynomial is constant over the set . Then we expressed the connection in the paper in these terms:

First, we show that is constant over the union of two small intervals . This is obtained by looking at modulo carefully chosen prime numbers. One way to prove this (at least infinitely often) would be to assume the twin primes conjecture… We manage to replace [this] by choosing four different primes in a more involved manner…

Thus we actually did something else besides use twin primes, but this is how we got the idea. Moreover, Shpilka and Tal used gaps between consecutive primes in a different way, obtaining bounds in terms of the largest such gap in the interval , which is known to be . If we can now get our hands on enough cases where the gaps are small, maybe we can improve the estimates further. Why is it useful to have primes with small gaps?

We have covered the use of the Chinese Remainder Theorem for analytical results before. Usually for complexity-theoretic applications such as this one, we want the primes themselves to be small—and don’t mind having a bunch of primes. For logspace we can work with polynomially-many primes in a sequential manner, so long as each needs only bits to store.

When we don’t need size to be so constrained, however, it can be more useful to have the gap between primes in the representation be small. Then if we know in advance that values are below , we know that the values and have to be close in an absolute sense. In particular, they cannot be closer than to each other unless , making them equal—and in our case we would get them all to be zero. For higher ranges of one retains a lot of this control. The larger and the smaller , the bigger the effective range.

We will hence be further interested in how dense the pairs of “cousin” primes must be, and how efficiently they can be generated. Anytime there is a breakthrough, it is time to revisit old ideas and see whether they too can profit.

Open Problems

How far can the gap between consecutive primes, for infinitely many such pairs, be reduced? Do this and Helfgott’s result on Goldbach herald a more general breakthrough in number theory?

[updated link and info about paper in opening paragraph; sourced photo and linked to Simons Foundation article]

Graduate Student Traps

KWRegan — Thu, 16 May 2013 04:42:46 +0000

Can we help avoid parallel repetition of mistakes?

Irit Dinur has recently again shown a wonderful skill at re-conceptualizing an area that had seemingly been well worked out. A notable previous instance was her re-casting the proof of the PCP Theorem as a progressive amplification. Now she and David Steurer have posted a new paper whose title, “Analytical Approach to Parallel Repetition,” introduces a new framework. The subject of parallel repetition, which they call “a basic product operation for one-round two-player games,” is distinctive in having its genesis in a mistake made in a paper—a trap of automatic thinking. Lance Fortnow and Mike Sipser thought that executing multiple instances of a randomized protocol in parallel would have the same power-law reduction in the probability of overall failure as executing them independently in sequence, overlooking how the strategies in the parallel cases could be dependent and exploit this.

Today we talk about similar traps that people can fall into even at advanced graduate level. How might they be avoided?

Truth-in-blogging note: this post is really about a different case of products and independence, and most of it was written months ago. It was lacking an intro section with someone to feature according to our “blog invariant,” and we also wanted a few short examples of graduate-student traps in computational theory and mathematics before progressing to the main one. The parallel repetition example came not only first to mind, but also second and third and fourth… as Dick and I struggled to think of more good ones. Lance’s haute faute has already been mentioned a few times on this blog, and I thought it would make a tiresome and repetitious parallel to my own “trap.” It didn’t help that the neat example I saw online years ago which furnished the phrase “graduate-student trap”—but which I didn’t preserve and forgot—has evidently vanished into unsearchability.

I was trying to decide between leading with the late-medieval mathematician-theologian Nicholas de Cusa for his advice on not pretending to have completed knowledge, or alternately a colleague in Buffalo who has compiled a good graduate-student advice list. Lance has similar advice, and when looking for it I spotted the mention of Dinur in his Twitter feed—actually Lance’s re-tweet of one by my former student D. Sivakumar. Voilà—Lance’s example it was. Thanks, Irit.

Traps

What is a trap? Pitfalls and paradoxes abound in mathematics and the sciences, and surmounting them is just part of acquiring the literature. Sometimes it is a confounding of preconceived expectations, but it is hard to find a way of defining such expectations that works for everybody or even most people. What makes a trap, in my opinion, is there being concrete indications in the concepts, in their contextual use, in their notation, or in the literature itself that run counter to the truth. Here are what strike Dick and me as a few simple but significant examples:

Square root is not a function. It is written like a function, but isn’t. Here is an example of what you can “do” by conveniently forgetting this: , so take square roots of both sides. You get

This contradicts the definition .

Not all matrices are diagonalizable. Since even many singular matrices are diagonalizable, it is easy to forget this is not true in general. If it were true, then there would be a really quick proof that a matrix always satisfies its characteristic polynomial . Namely, let with the diagonal matrix of eigenvalues. Then on substituting the right-hand side into the formula , all the ‘s and ‘s cancel except for one in front and one in back. The rest is the component-wise evaluation on each eigenvalue in , which identically vanishes, leaving the all-zero matrix.

Well this is often not a bad error to make. Every matrix has arbitrarily close perturbed forms that are diagonalizable, indeed have distinct eigenvalues. The above proof gives where the characteristic polynomial is coefficient-wise close to . Continuity then implies .

is not the same as the field . The former is not a field, as it has zero divisors. The multiplicative subgroup formed by the elements that are not multiples of is not a field either. But this is again not always a bad error to make, even in crypto. A lot of properties and problems are similar between the structures.

These are really at high school or undergraduate level, before the research stage. What kind of traps matter at research level?

My Trap

My own strongest feeling of falling into a “graduate student trap” came in October 2006, as I began my work on statistical claims of cheating with computers at chess that had arisen during the world championship match that month and before. I modeled a human player and a computer as distributions and over the choices of available moves in game positions. Cheating would depend on how close the ensemble of played moves was to vis-à-vis , so I wanted a suitable distance measure between distributions. Modeling the computer as a distribution not only allows for different chess-engine program versions and parameter settings, but also for a steady amount of small variation caused by hash collisions—as I described first here and mainly here.

I decided to postulate that for two different (sets of) positions and , the player’s distributions would be independent, and similarly for the computer. This makes the joint distributions and on satisfy and . So that I could group game turns as I wished, I wanted the distance measure to be additive, namely

The first distance measure I considered, called Kullback-Leibler (K-L) divergence, is defined (on discrete domains ) by

Its Wikipedia page says straight out that is additive. Great, I thought.

Unfortunately, K-L is not symmetric, and more of concern to me, approaches whenever there are events for which is tiny but is not. In chess, such events would be moves the computer recognizes as bad but that players still fall into, or are tempted by. This was a concern because chess positions can have many bad moves, so that the “tail” of the move distribution could distort the value of . I could switch around and to avoid this, but then reasonable moves shunned by players would cause other distortion.

Is Jensen-Shannon Divergence Additive?

Applications employing distributional divergence measures were new to me, but it so happened that my department’s Distinguished Alumni Speaker that month knew something about them. After hearing my issues, he—I won’t name the “guilty party,” though I already did—suggested using Jensen-Shannon (J-S) divergence instead. J-S reduces this distortion by employing the interpolated distribution . Then it is defined from two invocations of K-L by

This always gives finite values, and is symmetric—hence the use of comma not . Analogous to how the sum of squared differences, which is obviously additive on product vectors, is the square of the Euclidean metric, is also the square of a metric. All this, plus the absence of contrary information, plus the frequent words “J-S is a symmetrized and smoothed version of K-L,” naturally made me assume that was additive. Grateful for the tip, I happily started on the machinery to apply it for chess.

A week later I started drafting a paper describing my concept and model, and decided it would be good to include a proof that J-S divergence is additive. Then, only then, is when I discovered with an electric shock:

It isn’t.

I’ll leave the task of actually constructing counterexamples to the reader, but here’s an intuition. It uses a generalizing trick that reminds me of one by Bob Vaughan that we covered last July. For any put , and then define

A little reflection may convince you that this cannot be additive for all . Hence its being additive for , which yields , would be an “accident.” Finally thinking how and themselves can give-and-go with gives the inkling that the accident doesn’t happen.

How Clear in the Literature?

I can put this in the form of a “blog-beg” (sometimes called a bleg):

Can you find an easily-accessed source that says clearly that the basic Jensen-Shannon divergence is not additive?

As of this writing, the Wikipedia page on J-S still does have such a statement. Adding one yourself would be cheating. In 2006 I did not find one elsewhere, even in a couple of texts. My one-hour trial by Google when I first drafted this post last summer found one paper in 2008 titled “Nonextensive Generalizations of the Jensen-Shannon Divergence.” This clued me in that nonextensive is the antonym of additive. So the authors’ generalizations are not additive, but what about the original ?

Another paper I found had the promising title “Properties of Classical and Quantum Jensen-Shannon Divergence,” and even more, its first author and I have Harry Burhman as a common coauthor. It defines J-S with a bang in the opening sentence, states some generalizations of J-S, and (still on page 1) says the magic word:

Shannon entropy is additive in the sense that the entropy of independent random variables, defined as the entropy of their joint distribution, is the sum of their individual entropies. (emphasis in original)

But the next sentence brings up the different topic of Rényi entropy, and after a mention of “non-extensive (i.e. nonadditive) generalizations” of J-S it goes into quantum.

Another paper picks up the thread in its title, “Nonextensive Generalizations of the Jensen-Shannon Divergence.” The point of the first two words must be that the original J-S is additive, yes? It’s the generalizations that are non-additive. Right? The paper’s abstract says it builds something called the Jensen-Tsallis -difference, “which is a nonextensive generalization of the JSD.” So the original J-S is extensive, then? After defining additivity, it brings up the generalizations in which

… the additivity property is abandoned, yielding the so-called nonextensive entropies.

The next paragraph introduces K-L and J-S, but doesn’t tell me whether the “abandoned” property was ever there. It seems that this simple knowledge is presumed, but how might a bright young graduate student—or an old one—find it in the first place?

Open Problems

Can you give some more examples of “graduate-student traps”? Ones that are helpful to know?

Is Jensen-Shannon divergence close to being additive, in some useful sense? This actually strikes me as a non-trappy, research-worthy question.

Advances on Group Isomorphism

Pip — Sat, 11 May 2013 21:45:43 +0000

Finally progress on this annoying problem

David Rosenbaum is right now the world expert on one of my favorite problems, group isomorphism. He is a third-year PhD student at the University of Washington in Seattle under Paul Beame, and has been visiting MIT this year to work with his other advisor, our familiar friend Aram Harrow. He presented a paper on this at SODA 2013, and recently posted a successor paper to the ArXiv. He grew up in the Portland area where he won prizes in scholastic chess tournaments.

Today I want to talk about his work, which not only advances our understanding of this problem, but also makes progress on other ones.

Group isomorphism is a great problem for many reasons. It is a special case of the graph isomorphism problem, which is another annoying problem. Very annoying. Of course a good idea when confronted with a immovable problem is to work on a special case. Add extra conditions. Perhaps the special case adds enough extra structure, giving you additional leverage, so the problem can be “moved.”

Group isomorphism is not a great problem because people are lining up to solve it. Nor are people lining up to solve graph isomorphism problem either, but that is another story. The new algorithms that David has found are all galactic as we coined the term here. Indeed they are way out there in space usage too. But galactic or not they are based on beautiful ideas that will have impact in other areas, in my opinion. This is why they are so important. Every advance in our understanding of how to create clever algorithms—for any problem—advances our general understanding of computation. And that is good.

Notes From Underground

You can skip this, since it is about how we work here at GLL. Our “staff” has various rules and procedures for the creation of a new piece. We have a complex process… Of course not. But I did consider writing about David’s work quite a while ago.

Something made me delay and delay and delay. Other pieces were written, others were put out, and a discussion of group isomorphism stayed on the stack. Other things have slowed us including “floodgates” on Ken’s side. Somehow I knew that David was onto something great. Initially he made progress on special groups, then on a larger class of groups, and finally on all finite groups. Even better he discovered a new algorithmic principle that may be of importance to other areas of computing. It is exciting and may be useful in both theory and in practice.

An aside: perhaps I should still wait some more. Could his next result be even stronger? Oh well, we will push forward and if he makes more progress we will discuss that another day. Just to prove my point here is a quote from my earlier draft that we never “approved” for release:

I wonder if he can extend to prove for for all groups. The reason is that 2-groups always seemed to be the roadblock to me for this improvement.

Now we can note that he can indeed. Let’s talk about group isomorphism and more.

Group Isomorphism

The group isomorphism problem is about multiplication tables. A finite group is described by giving the complete product table. If the group consists of the elements

then the table is in size and the entry is : the product of and . The isomorphism problem is: given two tables, do they describe the same group up to renaming. That is, are the groups isomorphic?

Long ago Bob Tarjan and Zeke Zalcstein and I made a simple observation: Group isomorphism could be done in time

This relies on the easy-to-prove fact that every group has at most generators. We have discussed this idea earlier here.

What is nice about this observation is it is a much better time bound than any known for graph isomorphism. But it also depends on group theory in a very weak way. It only uses Joseph Lagrange’s theorem: the order of a subgroup always divides the order of the group.

New Results

David made progress on group isomorphism by combining powerful insights from group theory with several from computing. Let’s first state some of his newest results, and then discuss at high level how he proves them.

Theorem 1 Group isomorphism can be solved in time

for solvable groups.

Solvable groups are special, but I always thought that they would be a hard class to handle. For starters their structure is immensely complex. Yet solvable groups do have many neat closure properties that have proved to be useful.

Note that the “extreme” opposite of solvable groups are simple groups. Thanks to the classification of simple groups, the isomorphism problem for them is trivially in polynomial time. The algorithm is based on the cool fact that every simple group has a generator set of at most size two. It is interesting to have a trivial algorithm whose correctness proof is immense, since the correctness depends on the classification theorem. Perhaps this is the worst ratio of algorithm size to correctness proof? Anyway it is a building block for the final result.

Theorem 2 Group isomorphism can be solved in time

for general groups.

Of course the “general groups” here are finite—group isomorphism as discussed here is always about finite groups. The improvement is the constant ; previously it was . Thus this result is a square root speedup on the previous work—the first improvement of any kind. Even for galactic algorithms a square root improvement in the running time is a huge improvement. A very impressive result.

A New Game

As usual the best way to understand what David did is to look at his paper. However there is a new principle that he isolated that I would like to discuss here. This is the principle that I believe can be used elsewhere, and is a fundamental advance.

Consider the following game between Alice and Bob—who else? They each have access to a finite tree with root . The tree is labelled in different ways for each player. But given any node , the players can get the children of the node via their own oracles. Alice’s job is to select a set of paths from the root to the leaves; Bob also does the same and selects a set of paths . The tree is not binary—in a family of such trees of sizes , the arity of each node may depend on . This game is co-operative: Alice and Bob win if they are able to find sets and so that they have a common path. No bits can be sent between them, and since their labels are different the game is interesting. Their joint goal is to keep as small as possible. Clearly one could pick all paths and the other could pick one path, but that would be very expensive. Call this the brute-force strategy.

The issue in the game is whether Alice and Bob can follow a pre-determined strategy that identifies structural features in the tree. For instance, suppose they agree that after strating from the root they never choose a node with an odd number of children. Then the sets and can be limited to paths through nodes of even arity (after the root). Alice and Bob’s strategy succeeds on all trees except those that have a “blocking set” of odd-arity nodes.

David, not acting as a player in this game, has discovered a wide range of trees where there is an economical winning strategy for Alice and Bob. This strategy involves counting, and yields a much smaller value of then the brute-force strategy. The trees involved have nodes corresponding to ordered sets of group elements, with full generating sets at the leaves. The operations and analysis are quite beautiful and can be used to speed up his isomorphism algorithms. This is the principle that I think could have applications elsewhere.

One cautionary aspect of this game, however, is that even though it is defined on a tree, it lacks the space-saving properties of depth-first search. The entire sets involved seem to need to be maintained in memory at once. David in fact shows that his game strategy forms a spectrum with a common time-space tradeoff, with his best time at one end and the simple generator-enumeration algorithm at the other end.

Open Problems

The main open problem is to show—really show—that group isomorphism is in polynomial time. Or failing that, show that it can be done in time . This still seems beyond David’s methods, yet we will see.

Another interesting open question is what can we say about his new principle? Can we use his search ideas on other problems? This seems like an accessible research line.

A Most Perplexing Mystery

Pip — Mon, 06 May 2013 22:56:26 +0000

The discrete log and the factoring problem

Antoine Joux is a crypto expert at Versailles Saint-Quentin-en-Yvelines University. He is also one of the crypto experts at CryptoExperts, having joined this startup company last November. His work is featured in all three of the company’s current top news items, though the asymptotic breakthrough on the exponent of finding discrete logs in small-characteristic fields which we covered last month is not among them. In its place are concrete results on two fields of medium characteristic (between a million and a billion) whose elements have bit-size 1,175 and 1,425. The news release on this concludes (emphasis in original):

[We] recommend to all cryptographic users to stop using medium prime fields.

Today I want to talk about a mystery, which I find the most puzzling problem in all of complexity theory, but which Ken thinks is “only” a sign of youth of the field.

Not the question, not what is the power of quantum computers, not graph isomorphism, nor any other number of great puzzles. The single strangest problem, in my opinion, is the relationship between the discrete log problem and the integer factoring problem.

History shows that every improvement to one of the these problems seems to yield rather quickly a corresponding improvement to the other. This works for quantum and classical algorithms alike—recall that Peter Shor’s famous paper solved both problems at one stroke. As related here:

Historically, it has been the case that an algorithmic advance in either problem, factoring or discrete logs, was then applied to the other. This suggests that the degrees of difficulty of both problems are closely linked, and that any breakthrough, either positive or negative, will affect both problems equally.

The Problems

Let me restate the two problems for comparison, where are primes.

The discrete log problem is: Given , find the value of .
The factoring problem is: Given , find the value of .

What possible relationship is there between these problems? One concerns the structure of the finite field modulo some large prime . The other is about the multiplicative structure of the integers.

There is no known reduction from one to the other, at least none known to me. This is the great mystery.

However, they have a common parent. Consider the finite ring of integers relatively prime to , with operations modulo . Every element has an order , meaning a least integer 0}' title='{x > 0}' class='latex' /> giving (mod ), and this is the period of the powers of . In essence, is the discrete logarithm of in base in this ring. Shor’s algorithm gives a high-enough-to-amplify probability that a randomly chosen has a findable period that helps produce a factor of .

That the same idea works for discrete logarithms in fields indicates that this common parent problem captures at least some techniques that work for both factoring and discrete log. This MathOverflow item points out the importance of this being in turn a case of the hidden-subgroup problem for Abelian groups. Ken feels that only recently is the area finding techniques that really separate the problems from their parent, an indication of the field shedding its youth. This paper by Joux is an example, as we explain next.

The Breakthrough

The new wrinkle—a sign of maturity—is that the improvement in the running time depends on the characteristic of the field, and is greatest when it is fixed. Recall that in a finite field , is the characteristic and is the degree of extension from the prime field . The prime field is the same as the field of integers modulo , but should not be confused with the integers mod , which do not form a field. The idea of characteristic is also a distinction from .

Running times of algorithms for factoring and discrete log have heretofore been expressed in terms of the “-function,” whose general form is defined by

Here we may have . The length of the input is essentially . The backbone of this formula is , while the factor acts as a counterweight. The main focus is the exponent . If taken only thus far, the characteristic seems subservient to the degree in the time, since the formula takes the logarithm of .

The relation between and partly depends on how two phases of the algorithms are balanced, a “sieving phase” in which information about specific field elements is compiled, and a “linear algebra phase” for the main computation. Once other parameters are chosen to effect the balance, the part is often bounded and ignorable, yielding the simpler designation for the formula. The game plan is to improve the axis of the -vs.- tradeoff, to get lower .

The breakthrough by Joux is to do this when is bounded. This is important while bootstrapping the algorithm through extension fields and , where itself is a power making . The result is:

Theorem: For bounded , discrete logarithms in fields of characteristic can be computed in time .

His paper itself emphasizes two new ideas, which we express in terms we’ve described earlier on this blog.

Ingredients

Many “sieving” algorithms for factoring and discrete log begin with what strikes us as the opposite idea to the famous Sieve of Eratosthenes. That sieve works to find primes by crossing off numbers with factors. The newer sieves have the opposite goal: they want to find integers with lots of small factors.

A big reason lots of small factors are useful is that they help build Chinese Remainder representations for large numbers while keeping their own arithmetic simple. More generally put, the small factors help generate a lot of congruences—and the large numbers with those factors serve as common zero-points for those congruences. As a general strategy point, the more small factors with known multiples, the better. Here is where we mention one idea from our old post that is already presumed by Joux:

Integers polynomials: As we wrote before, integers share many properties with polynomials over the integers, but polynomials are usually much easier to handle. For example, the analogue of the famous Goldbach conjecture is solved for polynomials but is still open for integers: Every nontrivial polynomial over the integers can be written as a sum of two irreducible polynomials. For another example, the deterministic primality algorithm of Manindra Agrawal, Neeraj Kayal, and Nitin Saxena begins by introducing polynomials over one variable.

In Joux’s case, polynomials are used to construct extensions of finite fields to begin with. Joux operates further on these extensing polynomials. Adding one or a few variables creates more ways to define small factors. Thus using polynomials already constitutes a way to amplify, but Joux found a new way to carry this further.
Amplifying Linearity: If is a polynomial known to split into many small factors, then for any field constant , the linearly transformed polynomial also has this property. This was previously known and exploited as far as it can go. Joux noticed that certain ostensibly-nonlinear transformations could wind up having the same effect. Well, a transformation of the form

is sometimes called a “fractional linear transformation,” and in complex analysis is known as a Möbius transformation. Now is not a polynomial, but it becomes a polynomial again upon multiplication by where is the degree of .

Joux proves that if you take a polynomial that splits into small factors over the base field , and take any in an extension field such that , then the resulting polynomial over likewise splits into small factors over (though irreducibility and degrees may not be preserved from the corresponding factors of ).

The amplifying advantage is that whereas the transformation gives at most the size of the field number of new polynomials, the transform gives approximately the cube of that size—after eliminating redundancies and trivialities that happen when are all in the base field , for example. He also employs transforms by ratios of two quadratic polynomials.
Seeding More than Sieving: The cubic advantage from the last step is so great that the algorithm can dispense with sieving steps needed to build a base of small factors via search. Instead we can “seed” the base with some polynomials known to have many small factors in advance. Joux starts with the simple case of splitting into linear factors over . A further fact of particular importance is:

Lemma: A polynomial splits (into linear factors) over if and only if

also splits.

The Future

When I saw Dan Boneh at the RSA conference, as I first related here, he felt very confident that Joux’s work would lead soon an improvement in factoring. We will see if these happens, but his feeling needs to be taken quite seriously since he is an expert in computational number theory. I mentioned this to Lance Fortnow the other day and he immediately offered to make a bet with me. We have yet to work out the details, but I believe we will firm up the bet shortly.

Open Problems

Would you like to take sides on the bet? Will factoring get improved too? What about the mystery? Is there even some heuristic that explains why these problems are related, in a more detailed way than their common parentage?

A Trillion Dollar Math Trick

rjlipton — Thu, 02 May 2013 19:15:16 +0000

How linear algebra can make databases go really fast

Mike Stonebraker is one of the world’s leading expert on database technology. He started in academe at Berkeley, is now again in academe at MIT, and has launched a raft of successful companies. He is currently the co-founder and chief technology officer of at least three startups in the database area. One is called “Data Tamer” and is a joint venture with researchers from QCRI—Qatar Computing Research Institute—see this release.

Today I would like to talk about his presentation at the TTI Vanguard meeting on “Ginormous Systems” in DC. His talk was on “Seven Big Data Megatrends.”

By the way the word “Ginormous” is a real word–see here for the formal definition. I initially thought the Vanguard organizers had made up the word, but it is real. It should be obvious that Ginormous means large, actually really Large. This Vanguard meeting was dedicated to Ginormous systems of all kinds: from huge data centers, to city-wide systems, to supercomputers, and much more.

In Mike’s wonderful talk he made seven points about the past, present, and the future of database technology. He has a great track record, so likely he is mostly right on his guesses. One of his predictions was about a way of re-organizing databases that has several remarkable properties:

It speeds up database operations 50x. That is to say, on typical queries—ones that companies actually do—it is fifty times faster than classical database implementations. As a theorist we like speedups, especially asymptotic ones. But 50x is pretty cool. That is enough to change a query from an hour to a minute.
It is not a new idea. But the time is finally right, and Mike predicts that future databases will use this method.
It is an idea that no one seems to know who invented it. I asked Mike, I asked other experts at the conference, and all shrugged and said effectively: “I have no idea.” Curious.

Let’s look quickly at the way databases work, and then consider the trick.

Some Background

Modern databases store records—lots of them, usually on disks. A record is a fixed-size vector of information. The vector is divided into fields, where a field stores a type of information. An example is:

Thus the first field is the person’s name, the next the address, and so on.

In a sense the data is really stored in a table—or an array if you wish to be mathematical—call it for data. The rows contain each record, and the columns store the fields.

The issue is how the array is stored on the disk. Each record is stored one after the other on the disk. The records are stored as

Here each is the row.

This is a reasonable method, it puts each record together, and allows fast access of all of the records. Thus, a query can scan over all the records by reading the disk one track at a time. This is not a bad way to use a disk-like device.

Mike points out that all the classic database systems—well at least most—store records in this manner. Their code, which also is huge (if not ginormous) is tuned to handle data that is stored in this manner. Let’s call it the “record ordered method” (ROM). As a mathematical idea it is just storing the array in row-major order. Not only is this a perfectly fine way to organize the data, and to store the array, it respects principles that go back to COBOL in the 1950′s: Each data object should be conceptually and physically together.

But there is a better way.

The Trick

The trick to the 50x speedup is based on the deep, advanced, complex operation that we in math call the transpose of a matrix. Just kidding. It is based on the simple idea that instead of storing the matrix we store the matrix . Recall is just the matrix defined by

Let’s call this the column ordered method: COM. Now the data on the disk contains

Here each is the column.

So why is this method so much faster than the ROM? The answer is how the data is accessed by the queries. The data is read much more than it is written, so the key is to speed up the ability to read the data. But the critical insight is this:

A query is likely to use only a few columns.

For example, suppose the query is:

Select all the records with age in the range [21,31] and cell phones with area code 404.

Then the query needs only to look at two columns. All the other fields are completely un-needed.

Now suppose the records have a hundred fields. Since the query only looks at two fields there is a huge speedup. Then the speedup is roughly. In the COM the database algorithm only reads the data that it needs to use to answer the query. In the ROM method it reads all the data and that tremendously slows down the query. Note, things can be even worse, since the size of fields can vary widely. So the true speedup depends on the ratio of

Clearly if a record has even one large field that is not used in the query, the speedup could be very large.

How did people not realize this simple idea: replace the table by its transpose ? Well they did not actually miss it, but its power was not realized until relatively recently.

Whose Trick?

As I stated earlier no one seems to be able to say who exactly discovered the COM. Maybe as a default we could call it the Gauss Database Method, since most things are named for him. I did track down a system called TAXIR that was essentially a COM storage system with focus on information-retrieval in biology in 1969. The paper describing it is by George Estabrook and Robert Brill. Maybe they invented it. Perhaps their focus on biology made it hard for those in databases to notice their work? Especially years ago before powerful on-line search engines. Perhaps.

Ken adds that in a textbook used years ago for Buffalo’s course on programming-language concepts, the COM idea was called “parallel arrays” and was frowned upon. The main reason given was that this structure was hard to maintain, as a single off-by-one indexing error in one array could damage the entire set of records. However, a high-level system can maintain the data in-sync, while modern machine architectures increase the reward for keeping just the data you need in caches.

Open Problems

Okay, maybe the trick is not worth a trillion dollars. But the total amount invested yearly in data systems suggests that the column idea could over the next few years be worth quite a few dollars.

A simple thought: Is the column method the best way to store records? Can we in theory prove it is best in some sense, or is there an even better method? So forget the million-dollar Clay prizes and go after the real money.

Happy Birthday, Kurt Gödel

Pip — Sun, 28 Apr 2013 21:13:09 +0000

Wang on Gödel, and Gödel on Wang

source, with more Wang quotes

Hao Wang was a logician who made many important contributions to
mathematics and especially logic. His creation of the now famous tiling problem was of great importance. He did seminal work on mechanical mathematics, getting in 1983 the first Milestone Prize for Automated Theorem-Proving. Perhaps one of his best “results” was being the PhD advisor to Stephen Cook, Shimon Even, Stål Aanderaa, Joyce Friedman, and a recent invention-prize co-winner whom we mention below.

Today Ken and I wish to point out that it is Kurt Gödel’s birthday.

Gödel was born on April 28th, in 1906. Wang talked with Gödel in the 1970′s and documented that in his book: A Logical Journey. The book is a source of many insights into Gödel’s life, thoughts, and ideas. We will quote from it and then explain Wang’s tiles.

From Wang’s Book

Gödel published about 300 pages in his whole life. These are some of the deepest and best pages of mathematics ever. But it is a relatively small amount. We have researchers today that routinely have hundreds of papers, not just pages. He did not like writing up his work—he stated several times that he liked thinking much more.

Gödel studied Gabelsberger shorthand extensively from 1919 to 1921 while he was attending grammar school. One of the reasons that much of his unpublished work is unavailable, adds Wang, is that he used the shorthand for his notes.

Gödel’s doctoral dissertation proved the completeness theorem of predicate calculus. This had been stated as an open problem by David Hilbert and Wilhelm Ackermann in their book, Grundzüge der theoretischen Logik. Gödel’s solution was found right after the book was published: the book was published in 1928 and the proof was found in 1929.

Wang became involved with Gödel when he was writing a piece about the completeness theorem. Wang noticed that all the pieces of the famous proof were already known before Gödel’s definitive work. Wang sent a letter to the great man and received an interesting answer: Gödel said the point was that he had the “required epistemological attitude” to draw the conclusion even though the step was mathematically, “almost trivial.”

Wang discusses at length a wager between George Pólya and Hermann Weyl. The bet was about foundational issues of set theory. Here is a summary from our friends at Wikipedia:

George Pólya and Weyl, during a mathematicians’ gathering in Zürich (9 February 1918), made a bet concerning the future direction of mathematics. Weyl predicted that in the subsequent 20 years, mathematicians would come to realize the total vagueness of notions such as real numbers, sets, and countability, and moreover, that asking about the truth or falsity of the least upper bound property of the real numbers was as meaningful as asking about truth of the basic assertions of Hegel on the philosophy of nature. Any answer to such a question would be unverifiable, unrelated to experience, and therefore senseless.

Later in 1940 when the bet was “called” everyone declared Pólya the winner except for Gödel.

Gödel lost his cool when he discussed Ludwig Wittgenstein’s position on a matter of logic: “Has Wittgenstein lost his mind? Does he mean it seriously? … For example, ‘you can’t derive everything from a contradiction.’ He should try and develop a system of logic in which that is true.”

Gödel was quite interested, especially later in his career, on questions of mind and machine. He had a quite neat conjecture that he felt would prove the superiority of mind over machine. As quoted by Wang (p189):

It would be a result of great interest to prove that the shortest
decision procedure requires a long time to decide comparatively short propositions. More specifically, it may be possible to prove: For every decidable system and every decision procedure for it, there exists some proposition of length less than 200 whose shortest proof is longer than . Such a result would actually mean that computers cannot replace the human mind, which can give short proofs by giving a new idea.

Very cool question, very impressive for the time frame which was in 1976. We still have no idea, I believe, whether this is true.

Paul Cohen invented—I think invented is better here than discovered—the notion of forcing. This allowed him to prove independence results in set theory. Gödel’s comment about it was:

Forcing is a method to make true statements about something of which we know nothing.

Gödel had some very interesting thoughts on intuition in mathematics. My favorite is:

Intuition is not so unreliable. Often a mathematician first has an intuition that a proposition is probably true, and the proves it. If all consequences of a proposition are contrary to intuition, then statistically it becomes very implausible.

To see more, much more, get a copy of Wang’s book. It is filled with interesting bits of information about Gödel.

Tiling, Compactness, and Existence

Wang’s original tiles are squares with colors associated to each of the four edges. In a tiling the colors on every two meeting edges must match. One can put notches in place of the colors, so that tiles must fit like jigsaw pieces. A set of tiles is really a multi-set: one can use as many copies of each tile as one likes to create a tiling. The natural question is:

Given a finite tile set , can one tile the entire infinite plane?

Wang proved a theorem and stated a conjecture, both to the effect that the answer depends on what happens in finite regions.

Theorem: A finite tile set fails to tile the whole plane, if and only if, there is a finite connected region that it fails to tile.

The proof is a wonderful application of compactness.

Conjecture: A finite tile set tiles the whole plane, if and only if, it can tile a finite region so that the entire plane can be tiled from copies of that region.

Put another way, the conjecture states that if can tile the plane, then it can create a tiling that is periodic in the sense of a wallpaper pattern. Formally, this means the tiling is invariant under translations by vectors in a 2-dimensional lattice. One could believe further that the only infinite tilings must be periodic, but the “can” clause sufficed for the consequence drawn by Wang:

Theorem: If the conjecture is true, then the tiling question is decidable.

Proof: Here is the algorithm, given a finite tile set : Consider areas of size in order. For each value of , consider the finitely many connected regions of size , and the finitely many possible assignments of tiles to them. If some region has no legal tiling, reject. If the algorithm never rejects, then by the theorem can tile the whole plane. Then by the conjecture there is a finite periodic region. The algorithm will eventually discover this and accept. Hence the algorithm halts for all inputs, so it is a decider.

However, the conjecture was reversed when another of Wang’s students, Robert Berger, showed in 1964 that the tiling problem is undecidable. That is, he showed how to translate any given Turing machine into a finite set of tiles that tile the plane if and only if (when run on empty input) does not halt. Already a surprising consequence tumbles out of this fact:

Corollary: There must exist a finite tile set that tiles the plane, but only aperiodically.

src

Existence, Nature, and Gödel Again

Note in the corollary that not only can tile with no repeating pattern, it cannot tile any other way. This is because the conjecture was refuted even with the weaker “can” clause in its statement. What may be more disturbing than this property of itself, however, is the way we proved it by negation, without needing to have the slightest idea what could look like. Mathematics had started proving many “Existence First” theorems in the 1800s, and this led some to feel the objects thus proved needed to be “confirmed” by actual constructions.

In the tiling case Berger’s proof already gave a construction, by applying it to a universal Turing machine. This yielded an aperiodic-only tile set of size

However, once a constructive breakthrough is made improvements often flow. The above 13-tile set and partial coloring diagram are noted by Wikipedia. What seems to be still the smallest set for square tiles was found and proved in 1976 by Raphael Robinson:

The wonderful source page by Steven Dutch of the University of Wisconsin at Green Bay goes on to give some intuition about how aperiodicity gets forced with these tiles.

Ultimately Roger Penrose discovered that two non-square shapes suffice. The greatest “confirmation” may be that Nature herself implements essentially the Penrose-tile scheme in quasicrystals, for which Dan Shechtman was awarded the 2011 Nobel Prize in Chemistry after discovering them in 1982.

It is a tempting feel-good story to jump from Gödel-Turing computability and undecidability to the discovery of new natural processes, but we must not forget that the possibility of simplicity is the ultimate arbiter. The flip side of simplicity is complexity, a field we know scarce little about.

Instead we will jump back to Gödel, starting with Dutch’s note that “most aperiodic tilings have some kind of hierarchical structure”—such as a finite region being amplified into successively larger versions of itself. If we codify this kind of structure, we obtain a proof system for tilings that is pretty powerful. However it cannot cover all cases. The above proof of Wang’s theorem extends to yield:

Theorem: For every effective formal theory —such as Peano Arithmetic or ZF set theory—we can construct a finite aperiodic tile set that cannot prove tiles the plane (if is consistent).

We wonder if anyone has made a “Gödel tile set” that can tile the plane if and only if set theory is consistent.

Open Problems

How small can a “Gödel tile set” for set theory be? Does Gödel’s conjecture about proof lengths enter the picture? Finally, Happy Birthday again.

Sex, Lies, And Quantum Computers

rjlipton — Sat, 27 Apr 2013 13:48:38 +0000

Okay, no sex, but a discussion about quantum computers.

Steven Soderbergh directed the famous movie: Sex, Lies, and Videotape. This 1989 movie won the Palme d’Or at the 1989 Cannes Film Festival, and is now in the United States Library of Congress’ National Film Registry for being “culturally, historically, or aesthetically significant.”

Today I want to explore claims about quantum computation.

With all due apologies to Soderbergh his title seems perfect for our discussion. There may be no sex, but there is plenty of sizzle surrounding quantum computation. We just finished a thorough and wonderful debate here on the feasibility of quantum computers–see this for part of the debate. It was ably moderated by Ken, and the two advocates were Gil Kalai, against, and Aram Harrow, for.

While there are still many interesting issues to add to the debate, our discussion is not about whether quantum computers are feasible or not. We will stipulate that they can and will be built, eventually in some future time. The issue is about the present:

What has been proved about them so far?

And sadly we will discuss: what is believed to be proved but has no proof.

Claims

There are many claims in the literature on quantum computers that I would like to address here. Some are right, some are wrong, and some are at best misleading. Let’s start.

Quantum Computers have been proved to be more powerful than classical. Wrong.

This has been repeated many times and is often claimed in the literature. But there is no mathematical proof that a quantum computer that runs in polynomial quantum time cannot be simulated in polynomial classic time. None. Let me repeat that. There is no such proof. It is an open problem whether

If this is true, then quantum polynomial time equals polynomial time. Okay, most do not believe that this is true, but we are talking about what is proved. Nor is there any speedup theorem about improving the exponent in a general -time algorithm when you have qubits vis-à-vis bits. Thus from a mathematical view point it is clear that there is no proof that quantum is better than classical. None. Zero.

Quantum Computers can harness exponential parallelism, trying every possible solution at once. At best half-true.

Quantum computers can create superpositions of -many basis vectors, each representing a string in that can be a trial solution. However, the best-known quantum algorithm for finding a single solution still has exponential running time, order-of , and this is tight in black-box models (see below). The allowed linear algebra operations restrict the way this parallelism can be exploited. Scott Aaronson in particular has expended much effort debunking claims that quantum computers are imminently effective at solving certain NP-hard problems.

Quantum Computers can factor integers faster than classical ones are known to. Right.

But misleading, especially when the “are known to” part is sloughed off. There is no proof that factoring cannot be done classically in polynomial time. None. The best factoring algorithms are indeed super-polynomial, but there is no mathematical proof that they are optimal. So tomorrow, or next week, or secretly already?, there could be a classical polynomial time factoring algorithm. Peter Sarnak, for example, is on the record as believing this. I am too. But beliefs aside, there certainly could be such an algorithm.

Quantum Computers have been proved to be more powerful than classical in the black box model. Right. But this is at best misleading; at worst There are proofs that quantum machines can out-perform classical in this model. But the model is unfair. The classic machine gets only oracle access to some black box function say . Its job is to find some so that has some property. As with oracle results in complexity theory, it is often easy to show that this requires a huge exponential search.

What about the quantum machines? They can zip along and solve the problem in polynomial time. So this proves they are more powerful—right? No. The difficulty is that the quantum machine needs to have the function encoded as part of its quantum circuit. So the quantum computation gets the circuit representation of the black box, or the box is not black. The box is a white box—we can see the wires and gates that implement it.

This seems unfair, and is at best misleading. The fair comparison is to allow the classic machine the same ability to see the circuit representation of the box. The trouble now is the proof disappears. Without the black box restriction there is no mathematical proof that a classic machine cannot unravel what the box does and cheat in some way. So this is at best misleading. We also tried to see what happens if we open the box for the classical algorithm, here and here.

Quantum Computers cannot be efficiently simulated by classical, even for fifty qubits.Wrong.

I heard this the other day, and also it is stated on the web. For instance, Neil Gershenfeld was quoted by Julian Brown in his 2002 book Minds, Machines, and the Multiverse: The Quest for the quantum computer as saying,

“Round about fifty qubits is where you begin to beat classical computers. What that means is that with custom hardware tuned for computation with spectroscopy, you could just begin to graze the point where classical computers run out.”

Yes this was over a decade ago, but petabyte-scale computing was on the horizon then. Note that the interest in this question is quite reasonable, even though fifty qubits are way too few to implement any interesting quantum algorithm, certainly with the overhead of current fault-tolerant coding. The thought goes that fifty qubits may, however, be sufficient to do something interesting. Perhaps they can solve some physical question about a real system? A very interesting question. Let’s turn to discuss this question and scale in more detail.

The Fifty Bit Problem

The challenge is to figure out how we can simulate on a classical computer a quantum computation that uses at most fifty qubits. This is not nearly enough qubits to do anything interesting for cryptography, but makes for a nice question. The obvious ways to simulate such a quantum computation is not impossible for a classical machine, but is still not a simple computation. Thus the challenge: can we do this classically? Can we do a fifty quantum qubit problem on a laptop? Or on a server? Or in the cloud?

The obvious solution is to write down the initial vector of size and start applying the quantum gates to the vector. This immediately hits a problem, since such a vector is really large. But it is not that large. The size is “just” a petabyte—or actually 1/8 of a petabyte. The MapReduce cloud framework has already been used to carry out some basic algorithms on petabytes of data.

Quantum operations are notionally sparse, each affecting only a small constant number of qubits, generally up to three. Under the standard encoding, each qubit splits the long vector into a measurement set for a value and a set for the value, each of size . However, under different encoding schemes the operations could be local. In particular, many important quantum states, before and after some common quantum circuit operations, are succinct. There is scope for doing simulations analogous to what happens with homomorphic encryption, whereby operators might work directly on the succinct functional representations.

It strikes us that simulating a fifty-qubit algorithm classically is a concrete large-data kind of problem, one that may be interesting for its own sake. This stands apart from recent work on general “de-quantization” of circuits and algorithms, for which Maarten van den Nest’s ArXiV page is a good place to start.

Open Problems

What is really going on with the information structures that are acted on by quantum computers? That they are -long bit-vectors in any sense we have come to doubt. Is there a better general way to represent them?

Even with the -vector representation, for qubits, can we make classical simulations concretely efficient?

Can We Prove Better Independence Theorems?

rjlipton — Wed, 24 Apr 2013 14:23:12 +0000

An approach to independence with more complexity dependence

Florian Pelupessy recently defended his PhD thesis at the University of Ghent in Belgium. In joint work with Harvey Friedman, he found new short proofs for two independence results from Peano Arithmetic. One result is the famous “natural” Ramsey-theoretic independence result proved by Jeff Paris and Leo Harrington in 1977, while the other is a different Ramsey-type result obtained in 2010 by Friedman. Pelupessy also maintains a page with links on “phase transitions” in proof theory—meaning cases where a slight change in values of parameters makes a statement go from being provable to independent.

Today I want to talk about whether we can prove that some of our open problems are independent of Peano Arithmetic or other theories.

When we cannot prove something several ideas come to mind. Perhaps it is false, or perhaps it is time to try and prove the opposite statement. Or perhaps it is time to move on and try to work on some other problem. Or perhaps the reason we have failed to solve the problem is that it is actually unprovable. Kurt Gödel’s famous Incompleteness Theorem shows, roughly, that any powerful enough theory must either be inconsistent or incomplete.

That is the theory must either prove everything—it is inconsistent—or it must miss proving some true statements. One idea that keeps coming up, one question that we hear often is: could our favorite open problems like be unproved, since there are no proofs? It’s hard to find a needle that is not in the haystack.

We have talked about independence before—here—but I wanted to add some new thoughts on this issue.

An Independence “General Nonsense”

Ken and I were both part of a bunch of people interested three decades ago in whether a general kind of independence result had larger significance. The general idea is that there are computable functions that outgrow every function that Peano Arithmetic can prove to be total. Moreover, one can majorize them in a sense by predicates that are very easy to compute:

Theorem: For every computable we can compute a function such that:

is computable in linear time and log space;

for every there are infinitely many such that ;

the function mapping to the least x}' title='{x' > x}' class='latex' /> such that makes f(x)}' title='{h(x) > f(x)}' class='latex' /> infinitely often; and

for all , f(x)}' title='{h(h(x)) > f(x)}' class='latex' />.

Indeed Ken’s 1996 paper with Heribert Vollmer showed how to compute in various notions of logarithmic time. Think of the values as being “colors”: red, green, blue… Then colors with infinitely many colors, but such that each one of the following false statements is consistent with Peano:

The range of is almost all red.
The range of is almost all green.
The range of is almost all blue.

To apply this, make a language that consistes of the red strings in , the green prime numbers, and the blue strings that belong to the Graph Isomorphism language. Then since all these languages belong to and is easy, belongs to . However, it is consistent with Peano to believe that is a finite difference of any one of these languages. If in fact , then it tumbles out that is neither in nor -complete, and not polynomial-time equivalent to Graph Isomorphism either. Hence all of these facts are independent. This applies regardless of what machine code is used to specify formally. One can churn out myriad similar instances.

This kind of theorem once seemed exciting, but has proved unsatisfying, because it doesn’t really say much about complexity. So here we want to think of ideas that are more concrete about computations.

The BPP Hierarchy Problem

Call a nondeterministic Turing Machine (NTM) decisive provided on an input either at least paths accept or at least of the paths reject. We assume in these machines that all paths have the same length with binary branching. That is, they have the same probability if the NTM is viewed as a randomized Turing machine.

The complexity class contains those languages accepted by a decisive NTM that runs in time . The classic open problem is: does more time help for these complexity classes? More precisely is

for any ? This is widely believed to be true, but is open even in the oracle world. See the last section of this post for a summary on this.

An Idea

I have often thought about trying to prove something like the following. There is a language so that is not in , and yet it is consistent with Peano Arithmetic (PA) to suppose that is in . This is plausible, and stays “safely” short of proving what we really want. Still even this seems difficult.

I would like to share an idea about how one might attack this problem. To make the explanation “fun,” let’s call an NTM program a Knight if it is decisive and runs in at most linear time; let’s call a Knave if it runs also in linear time but is only stipulated to be decisive for inputs of length . It can be decisive elsewhere, but must be decisive for these length inputs. Here is the computable function mentioned above that Peano Arithmetic cannot prove to be recursive, or even bounded by a recursive function. So a Knave can look like a Knight for many inputs.

Let be a NTM that runs always in time . It operates as follows. On input it does the following: It checks that 2^{f(N)}}' title='{w > 2^{f(N)}}' class='latex' />. If not then it rejects. If it is true, then we check that is decisive on all inputs of length at most . If it fails, then again we reject. But if it works, then it simulates on the input for steps and flips the answer.

Let be the language accepted. We claim that is not in . This follows since we diagonalize over both Knights and Knaves.

What can we say about being in ? The time bound is not a problem. The problem is whether there is a decisive machine that accepts . In order to study this consider the formal sentence:

What this says is that there are an infinite total recursive number of inputs that make indecisive. Note that if this is false, then is decisive for all but a finite set of inputs. Since , like most complexity classes, is closed under finite differences of its languages, falseness implies that is in .

The critical idea is that I believe, but cannot prove, that this sentence is consistent with PA. The intuition is that provided grows very fast, if PA could find an infinite number of bad inputs, then this would contradict the growth of .

I must admit this seems close, but it does not yet work. Perhaps someone can fix the above argument.

Open Problems

Can the above argument be made to work, and yield more interesting independent statements?

Ken suggests a problem that may be a stepping-stone. Define to be the class of languages accepted by polynomial-time NTMs such that for some poly-time NTM , Peano can prove that and are complements. Then , and by enumerating proofs, we can construct a language in to which every language in reduces.

This is cool: if every language in is provably so, then has a complete set. The question is, does itself have a complete set, regardless? The hitch with is that proving a nontrivial accepts the complement of seems to involve proving that Peano is consistent, which Peano itself cannot do. But perhaps a coding trick can construct a , and importantly a formal sentence representing it, which can be proved while avoiding a fatal self-reference.

Subset Powers of Graphs

Pip — Thu, 18 Apr 2013 22:03:59 +0000

A possible way to extend the idea of matchings?

source—interesting genealogy

Johann Pfaff was a German mathematician of the late 18th and early 19th century. He is best known for the fact that the determinant of a skew-symmetric matrix (that is, where the transpose equals ) is the square of a polynomial in its entries. Actually it is thought that he did not discover this result. Arthur Cayley named the polynomial after Pfaff posthumously in 1852, having published the result in 1847, while was re-discovered and published by Thomas Muir in 1882. At least the polynomial was not named for Carl Gauss. Perhaps this was because Gauss was Pfaff’s student.

Today Ken and I want to talk about a kind of graph powering that may be new, and that may relate in some cases to Pfaffians.

The Pfaffian of a skew-symmetric matrix of odd dimension is always zero. For even dimension it is defined by the formula

Note that for any matrix , is also skew symmetric, so it has a Pfaffian given by the identity

Hence one can do “Pfaffian elimination” where represents doing an elementary operation to the rows, so long as you do the same operation to the columns—which is represented by . This simplifies the computation like with Gaussian elimination, though of course one can also get a polynomial time algorithm for by doing and taking the square root.

The main use of Pfaffians in complexity theory is that the task of counting perfect matchings in a graph sometimes reduces to computing a Pfaffian. When is the adjacency matrix of an undirected -vertex graph, this is already hinted by the above formula: a term of the product is non-zero if and only if the entries involved define a matching. If one can negate half the entries in so that all the terms have the same sign, then the Pfaffian computes the number of perfect matchings. Michael Fisher, Pieter Kasteleyn, and Harold Temperley proved that this was both possible and efficiently computable for planar graphs. My Tech colleague Robin Thomas has a nice survey on graphs and Pfaffians.

Products and Powers

One of the central ideas in mathematics is taking two mathematical objects and and creating a new object that is the “product” of the two. There are more types of products in math than almost any other construct. The root of all products is probably the simple Cartesian product of two sets: . Recall this is the set of all ordered pairs where is in and is in .

Another central idea is taking just one mathematical object and creating a new object that is a “power” of . Usually this is defined in terms of a product by making , or , and so on. The “so on” can include negative and even fractional powers, but always the notion of “power” seems to depend on a notion of “product.” The question is, must it always?

Cycle Graphs

I got into this by considering finite directed graphs that consist only of directed cycles: we will call them cycle graphs. Of course cycle graphs are essentially the graphs that correspond to permutations, and indeed this connection is essential. For now let’s think of them as just graphs. We use to denote that there is an edge from the vertex to vertex , and will drop the subscript when it is clear which graph is. Ken and I started thinking about cycle graphs a long time ago, see this.

Subset Power

For a set let be the sets with each in and the elements all distinct. The notation suggests the number of such subsets, but suppresses “” which is just the size of .

Definition 1 Suppose that is a directed graph and is an integer. Define as the graph with vertices and with an edge from to if and only if for each , there is an edge from to . Call it the subset power.

A first point is that since the nodes of are sets, we are free to permute the elements any way we wish. Consider . There is an edge from to in provided and or and .

A second point is that not all of need be distinct—in the former case we can have or , in the latter or . That is, if is a path in , then in .

If is bipartite, with edges , then for any , has a fairly simple description. If and are subsets of the same size , then is an edge in if and only if has a perfect matching as a subgraph of . But also we can get edges in where and “cross the partition.” That is, if you take any perfect matching of and swap the vertices on the ends of any edge between and , you get such a . You can define exactly such edges in by making such swaps, and when the perfect matching of is unique, these are the only edges one can make from subsets of . However, when it is not unique this process may lead to some double-counting of edges of .

Thus counting is tricky in even when is bipartite, and tricky even in again because of possible double-counting of pairs of edges when are all distinct. But it may be possible in certain cases, and tell us things about counting perfect matchings of subgraphs. We consider simple cases, and find that things are already complex, however.

Why Are Things Complex?

The basic issue is that the subset power of a single cycle can be complex. For example consider the cycle , which has length by our convention. What is —the square of by the direct product? It is easy to see that it consists of vertices and has six cycles all of the same length six. Easy.

Now switch to ask, what is —the square of by the subset product? A natural conjecture seems to be that the subset power also has all cycles of the same length. This is not true. By definition contains

vertices. Right away we see that is not a multiple of so they cannot all have length six. Let be

Then the cycles of are:

Thus has two cycles of length and one of length .

This may be surprising to you. Things can get messier when the subset power is higher. So the analysis of the structure of subset powers can be a challenging task.

Squaring the Cycle

We need to understand the structure of when is a single cycle and . We prove that when equals the issue we saw for 6-cycles is the only one that arises, but it is already tricky.

Proposition 2 Let where is a cycle of length . Then

If is odd then has cycles all of length .

If is even then has cycles all of length and one cycle of length .

Proof: The case when is odd is eay so let’s suppose that the length is even. It is convenient to assume that has vertices and edges are modulo . Then a cycle of length will look like

where . Also note that has vertices. It is not hard to prove that

This implies that is a multiple of : thus, is either or is . We will call former the regular case and the later case the short case.

We claim that there is exactly one short cycle of length and all the rest are length . This will imply (2) since

So it remains to show that there is only one short cycle. Let be a vertex on a short cycle. Then which implies that and . Thus the vertex is

Let and be on short cycles. Clearly for any is also on the same cycle: that is

is on the same cycle. Set . Then is on the same cycle: so there is only one short cycle.

Let’s check the example from the last section. Recall we showed that had two cycles of length and one of length . The above Proposition~2 shows that there are cycles of length and one of length : this is exactly what we showed.

Open Problems

There are two main open questions. Consider : the subset power of a single cycle of even length . We wish to count the number of even cycles of . What is a formula for this number? Also even without an explicit formula can we prove that the number for fixed is periodic in the length ? The latter if the period was explicit would allow us to calculate answers. Both these conjectures are non-trivial—I believe—because of the shortcut issue that arises.

Might the concept help with matchings, or ideas of integer-valued flows in parts of a graph? What do you think?

Measures Are Better

rjlipton — Thu, 11 Apr 2013 21:35:05 +0000

The Littlewood conjecture—another drive you crazy conjecture

John Littlewood is the latter half of famous duo of Hardy-Littlewood. I have discussed him before here and only wish to point out that he was, on his own, one of the great analyst of the last century.

Today I will discuss the Littlewood Conjecture, which has been open now for over eighty years.

There are conjectures that seem to approachable, and there are those that seem unapproachable. The Littlewood conjecture falls into the former category. Yet it has resisted efforts to solve it now for years. Akshay Venkatesh of NYU has a lovely survey about the problem. Also Tim Gowers has some interesting ideas about it—see this post.

The Conjecture

Define to be the fractional part of the real number . Thus, . Littlewood was interested in how well two real numbers could be approximated by rational numbers that had the same denominator. A reasonable question, no? He conjectured that for any two real numbers and ,

could be made as small as one wished by choosing the right natural number . Another fancier way to say this is that

Note, that it is not hard to prove that is bounded for arbitrarily large . Thus the critical point of the conjecture is for these larger and larger , why would not at least be smaller and smaller?

I got interested in this question for several reasons, but perhaps the most compelling reason is there is some strong evidence that it is true. In particular, it is now known that the and for which the conjecture fails are very “sparse”: they have Hausdorff dimension zero. This is even stronger than measure zero. Yet it is open whether the conjecture is true even for and . This reminds me of the situation that we face in many parts of complexity theory. Often we can prove that there exists some object, but finding a concrete example is another whole story.

Let’s take a quite look at why Littlewood was possibly led to his conjecture, and discuss some simple issues around the conjecture.

Dirichlet Approximation Theorem

Lejeune Dirichlet’s famous theorem of 1842 says:

Theorem: For any real number there are an infinite number of natural numbers and so that

To be sure, the theorem is only interesting when is irrational, and in that case one can add that and have no common factor.

Note that the theorem implies that . Thus to prove Littlewood’s conjecture—since this is bounded—we need “only” show that can be made small for some of the ‘s.

The Proof

We can assume that is irrational. Divide the interval into intervals: the is where is from the range . Define the function as the interval that

is in. Recall that means the fractional part of : it is where is an integer and . Note that since is irrational, the function is well defined. Now consider ranging over the values

where is less than but large. Then by the pigeonhole principle there must be a so that for two values: denote them by and and assume t}' title='{s>t}' class='latex' />.

The key is that than there are integers and so that

where both and are less than . It follows that

where is an integer. Let , and so

An alternate way to view this is given as “Theorem 1” in Gowers’ post. For any and , if we choose uniformly, the expected number of members of the progression (wrapping modulo ) that give is 1}' title='{1 + \frac{1}{n} > 1}' class='latex' />. Hence by the probabilistic method, for some there exist two such members, call them and with . These automatically give .

One Trick

Again look at Venkatesh’s paper for more information on the Littlewood Conjecture. One “trick” that is key to progress is this: measures sometimes behave better than sets. Consider what a set is and what a measure is:

A set can be viewed as a map from a universe to the boolean values .
A measure can be viewed as a map from subsets of the universe to the real interval .

The idea used in the study of the Littlewood Conjecture is that measures can often behave better than sets.

Here is a concrete version from the paper. Take a closed subset of the unit square . Let be a projection from the square to the interval . Define as the set . There is no reason that and near each other should have and “near” in any sense. Sets behave badly.

But measures can save the day. Change the set to a measure . Then the corresponding idea almost works: on a set of measure the measures will be near each other. That is, let be the projected measure on the interval of the -axis, and for each , let be the measure restricted to the fiber . Then we obtain

where importantly, the function is continuous on all but a set of tiny measure.

I am intrigued by this shift from sets to measures. In complexity theory can we use anything like this?

Littlewood Conjecture And An Old Friend

A classic pair of results are that and are both irrational; actually even better, they are transcendental numbers. That is they are not just not rational, but are not the roots of any integer polynomial. What about and ? One must be irrational: if both were rational then their sum would be too, which is impossible. But which one is irrational? I believe that this is still wide open.

There is a simple connection to the Littlewood Conjecture. Call and a bad pair of irrational numbers if they are counterexamples to the conjecture. That is

0. ' title='\displaystyle \liminf_{n \rightarrow \infty} n||n\alpha|| \cdot ||n\beta|| > 0. ' class='latex' />

Can and be a bad pair?

Let’s look at this. Suppose that for some rational number . Then by Dirichlet’s Theorem there are with as large as we like so that

But this implies that

We get, therefore, that

and that

This shows for large enough that can be made as small as one wishes. Thus and are not a bad pair, provided we know that is rational.

What does this mean? It shows that there is a simple connection—which I believe to be a really interesting connection—between two old hard problems. Can we exploit it to say something more about either the Littlewood Conjecture or whether or not numbers like are irrational?

Open Problems

It would help greatly if we had some control over the value of in the Dirichlet Theorem. Suppose that we place into bins. Let be the set of so that y}' title='{x>y}' class='latex' /> and and go into the same bin. What can we say about ? Can we say anything useful?