Gödel's Lost Letter and P=NP

Routing Forever on an Expander Graph

rjlipton — Sun, 06 Dec 2009 15:04:49 +0000

Can one route requests forever on an expander graph?

Noga Alon is one of greatest combinatorial experts in the world. He has won many awards and has made countless contributions to almost all aspects of combinatorics and theory.

Today I want to talk about a problem that occurred to me after listening to his talk at our recent theory day. It concerns a neat result that he has on routing on expander graphs.

On-line Routing on Expanders

Alon and Michael Capalbo have a pretty paper on routing on sufficiently strong expanders. Let’s call them AC—not to be confused with the complexity class. Suppose that is an vertex graph that is a strong expander—for now just think an expander. Then, they consider the following natural routing game. There are two players, the requester and the router. The requester is allowed to make a request of the form where and are vertices of the graph . The router must then find a path from to in the graph. We will say that all the edges on this path are in use. Think of edges as network links that can only carry one “conversation’ at a time. The game allows the requester to continue to make one request after another. The router must satisfy each request as it is made. Of course the router can only use edges that are not in use: thus, once an edge is used for a request it cannot be part of another path in the future.

Further the router has no knowledge of the future requests that are going to be made—it is solving an on-line game. This is what makes the game both realistic and interesting. The goal of the router, of course, is to never fail—to always be able to handle the requests.

There are two simple restrictions on the requester. First, the requester is restricted to make requests such that each vertex appears as a source or sink of a request at most once.

Note, AC, remember that’s Alon and Capalbo, are able to relax this quite a bit, but some restriction is needed. Otherwise, just make requests using the same vertex of degree and the router must fail. Second, the requester is limited to a total of some requests. This is also natural. Otherwise, since edges can only be used once the router would fail for the simple reason that all the edges are gone. The question is how many requests can the router handle? Their beautiful result is:

Theorem: On any strong expander, there is a deterministic strategy that can successfully route any number of requests.

This is essentially optimal, since on average the paths with be of length , and so any more requests would potentially use up all the edges. See their paper for the exact definition of strong expander.

The proof of the result is quite neat and involves some clever maintenance of a core partial expander with a special structure. Note, prior to this result there were weaker bounds. For example, Alan Frieze has an off-line randomized algorithm with the same bound on the number of requests. AC get the optimal bound, their algorithm is deterministic, and also it is on-line: the router is able to make the decisions as they come—a very realistic assumption.

Routing Forever

After theory day I soon thought about how to improve AC’s theorem. Why not see if there could be a router that could handle requests forever? Of course, this is impossible, since the router must run out of edges eventually.

But wait. Let’s change the game as follows. The requester cannot expect the router to handle more than order requests. Thus, let’s have the requester have a new ability: the requester is allowed to ask the router to release a previous request. This will free up the edges that the router used to fill that request.

The question now is simple: Suppose that at no time are there more than requests that have not been released. Can the router go on forever? Or must it eventually get stuck?

I have thought about this with my graduate students, especially Atish Das Sarma, Subrahmanyam Kalyanasundaram, and Danupon Nanongkai. We cannot prove that there is a forever router, nor can we prove that none exists. The difficulty is that even though the there are no more than active requests, the requests and releases can be asked for in an adversarial manner. The requester could perhaps force the router to make a “bad” choice, then release some requests, and by repeating this over and over eventually get the router into a bad state.

We were able to prove the following special case:

Theorem: On any sufficiently strong expander, there is a deterministic strategy that can successfully route forever, provided the requests are released in a FIFO order, where the FIFO is length order .

Thus, as the requests come in they are placed in a FIFO of length . Once the FIFO is full the next request forces the oldest request to be released. This is a pretty realistic model—it seems reasonable that old connections are eventually released.

The proof of this theorem uses AC’s result. We tried to redo their proof for this theorem, but finally realized that a very simple strategy would work, and we only needed to use their router’s strategy as a black-box.

We imagine that our expander is strong enough to contain two strong expanders on the same vertices. Each of the expanders is strong enough in the sense of AC’s theorem. Call the first expander the “top” and the other the “bottom” expander. Also, we assume that the router knows which edges are from each expander.

The routing strategy is simple: Initially both expanders have no paths. On the first requests use the top expander to handle all the requests. Follow AC’s strategy here. Then switch to the bottom expander. Again use AC’s strategy here for the next requests. Then, switch to the top expander. The key point is that by now all the requests there have been released. Thus, the expander is “clean” and we can again use AC’s strategy there. This goes on forever switching between the two expanders.

Its a pretty simple idea. But, it shows that there is some hope that routing can be done forever. Actually at first we thought the FIFO order might even be the worst case. Note, even if one request is allowed to stay for a very long time, then when we switch to the top expander it will not be “clean” and AC’s strategy does not apply.

A final comment. Instead of requiring the expander to have two copies of a strong expander, the same result would follow if we allowed each edge to be used at most twice. Thus, in network terms if each edge/link had the capacity to carry two conversations, then our router can route forever.

Finally, we have several ideas and other partial results, some of which we hope could help us generalize the above result substantially. More on this in the future.

Open Problems

Does there exist a forever router that can handle non-FIFO releases?

Is There a Test for Consciousness?

rjlipton — Thu, 03 Dec 2009 16:51:34 +0000

Blum discussed a theory of consciousness recently and this is a follow-up

Manny Blum is one of the great leaders of the theory community. He is a Turing Award winner; was the Ph.D. advisor to some of the top leaders in the field; is the creator of whole parts of theory—from Blum complexity, to algorithms of all kinds, to the theory of pseudo-randomness, to much more; and is one of the true visionaries of the field. There is only one Manny Blum.

Today I would like to talk about his recent presentation before FOCS 2009 on “what is consciousness?” I have no answer, but I have thought about his provocative talk and have a question that I would like to share.

When I was at Berkeley, in the late 1970’s as a faculty member, Manny was then the chair of the Computer Science Department. In those days we used those little pink square sheets of paper to keep track of calls. You may remember them; each had a space for the obvious information: who called?, when? any message? This was before email was common, and we still used the phone as the primary way to reach each other.

Manny had a spike in his office. On the spike were all the pink sheets, in order, of all the calls that he had ever gotten as the chair. I once asked him why he kept them all, and he just smiled. I never knew if he ever went back and looked at old ones, or if he just liked the physical image of all the messages he had handled as chair. I wish we had something as cool looking today with our email. I always like the image of the messages piled up on a spike.

Let’s turn to the discuss the issue of consciousness.

My Question

Manny discussed the: What is consciousness problem. (WICP). He is quite excited about the theory called Global Workspace Theory, which was created by Bernie Baars. As I said before, Manny believes that this theory has the ability for the first time ever to start to explain the WICP.

Since his talk, I have spent some time reflecting on what he said, and I have a simple thought. Manny talked about trying to figure out what is consciousness—trying to solve the WICP. He spoke at length about a variety of ideas that could lead to some computational like theory of how our brain creates consciousness.

What about a Turing test for consciousness? Is there a way to interact with something and see if it is conscious or not? I have no idea how such a test would go, but the idea would be that it would consist of some kind of interactive scheme. Suppose that X is some entity—it could be my friend Fred, or my dog, or a rock. Is there a series of interactions with X that have the following properties:

If X passes the interactions, then it is reasonable to conclude that X is conscious.
If X fails the interactions, then it is reasonable to conclude that X is not conscious.

The test does not have to be perfect, but having such a test would seem to be “easier” perhaps than understanding the WICP. Let’s call such a test a Blum test. Does it exist?

I checked and there has been some work on using the famous Turing test to argue about the WICP. I quote:

The Turing test has generated a great deal of research and philosophical debate. For example, Daniel Dennett and Douglas Hofstadter argue that anything capable of passing the Turing test is necessarily conscious, while David Chalmers, argues that a philosophical zombie could pass the test, yet fail to be conscious.

The test for conscious would not in my opinion be the same as the Turing test. I think that a consciousness test could be quite different. For starters, my dog fails the Turing test by definition since she cannot talk. But, that seems a bit narrow—maybe my dog is conscious?

There also is the so called mirror test, invented by Gordon Gallup. The test essentially checks whether or not, in our terms, X can recognize X in a mirror. The test seems to not be what I am looking for. If nothing else, it says that any X that cannot “see” cannot be conscious.

Finally, Atish Das Sarma points out that there could be some connection to the recent work of Brendan Juba and Madhu Sudan on Universal semantic communication. I do not know.

Open Problems

Is there a Blum test for consciousness? What would such a test look like? What do you think?

[minor edits]

The Curious History of the Schwartz-Zippel Lemma

rjlipton — Mon, 30 Nov 2009 16:17:46 +0000

On the discovery that randomness helps exponentially for identity testing

Jack Schwartz was one of most famous mathematicians who worked in the theory of linear operators, his three volume series with Nelson Dunford is a classic. Yet he found the time, the creative energy, and the problem solving ability to make seminal contributions to many diverse areas of science—not just mathematics or theory—but science in general. He worked on, for example: parallel computers, the design of new programming languages, fundamental questions of computational theory, and many more wonderful problems from a wide range of areas. He is greatly missed by our community as well as all of science.

Today I want to talk about the Schwartz-Zippel Lemma. Or the Schwartz-Zippel-DeMillo-Lipton Lemma. Or the Schwartz Lemma. The whole point is to discuss the curious history of this simple, but very useful result.

First, I thought that I would give one short story that shows somethings about the character of Jack. When he was working on a problem, that problem was the most important thing in the world. He had tremendous ability to focus. He also was often ahead of his time with the problems he worked on; his important work on parallel computers was way before it was the critical problem it is today.

Once at a POPL meeting Jack Schwartz was scheduled to talk right before lunch. His talk was on his favorite topic at the time: the programming language SETL. He loved to talk about SETL, I think it was one of his favorite topics of all times—but that is just my opinion.

In those days there were no parallel sessions, and the speaker right before Jack gave a very short talk. That left almost a double period before lunch. Right after being introduced, Schwartz said with a smile,

“well I guess I have almost an hour then ”

There was a slight sigh in the room—it was clear that most people would have preferred a short break before lunch.

And he took it. But Jack’s talk was wonderful, and we soon forgot about lunch, and learned the cool ideas that had gone into his latest creation. I never used SETL, but I could see the importance of the ideas. Like his work on parallel computations it was perhaps ahead of the field. SETL was a very high level language compared with the popular ones of then, and seemed almost too big a jump.

SETL was, as you might have guessed, a set based language that had an important influence over later languages. Guido van Rossum, considered the principal author of Python, said of SETL:

Python’s predecessor, ABC, was inspired by SETL — Lambert Meertens spent a year with the SETL group at NYU before coming up with the final ABC design!

Thus, SETL is the grand parent of Python, which shows the importance of the ideas that Jack had years earlier. By the way many others worked on SETL and I do not wish to minimize their contributions—perhaps I will discuss that another day.

Let’s now turn to the history of probably one of simplest results Schwartz ever published—but perhaps in the top few of his most cited papers.

Identity Testing

The fundamental question of identity testing is: given a polynomial of degree how hard is it to tell whether or not the polynomial is identically equal to zero? Note that we can only evaluate the polynomial at points of our choice, and do not have access to the coefficients of the polynomial. It is not hard to see that a deterministic algorithm that can only evaluate the polynomial, could need as many as

total points.

About thirty years ago, the idea arose to a number of people that perhaps a random algorithm could do much better. Indeed randomness helps exponentially here.

The intuition is really simple: Suppose that you envision as a surface in dimensional space. How many of the points of the dimensional cube, can lie on this surface? My geometric intuition is not great, but it seems reasonable that no many how the surface bends and folds, few points of the cube will be on the surface. If by “few” we mean: what is the measure of the points, then the measure is . So if we pick a random point from the cube, the probability it lies on the surface is .

The issue, as computer scientists, is of course that we cannot pick points from the cube; we can only pick points from a discrete set, and the cube is continuous. Said another way: we cannot pick points of infinite precision, we must pick points of finite precision. Now the intuition is still correct, but one needs to prove it. This is a classic theme that recurs throughout theory: often pure mathematical results must be “improved” to be of use to our community.

The key lemmas that were proved were all of the following form:

Lemma: Suppose that is a non-zero polynomial of degree over a field and is a non-empty subset of the field. Then,

where are random elements from and is a function.

The question is what makes the lemma true? We will see that the following two bounds were proved:

The Strong Bound is, ;
The Weak Bound is, .

Schwartz’s Paper

Schwartz published his work in JACM in 1980. His actual paper is quite long and covers many other aspects of identity testing. His main result is the famous test for identity, which in our terms is the Strong Bound for Identity Testing.

He was also very interested in applications: plane geometry for example. He also discussed real variable versions and other extensions. An earlier conference version is in the Proceedings of the International Symposium on Symbolic and Algebraic Computation in 1979.

Zippel’s Paper

Zippel proved a more complex bound than Schwartz in his paper. That his result is at least as strong as the Weak Bound, is immediate. Part of the difficulty is that Zippel had a different model: he used the maximum degree of each variable, rather than the total degree. Thus, it is a bit difficult to compare Zippel’s result with the other two, since he has a somewhat different model.

His paper was published at the same conference as Schwartz in 1979, where he spoke right after Jack—according to the program. Zippel was driven by the work of actual symbolic systems and he gave timings of a real implementation of his identity testing algorithm. It is nice to see that the methods really work.

DeMillo-Lipton’s Paper

We published our paper in Information Processing Letters in 1978: it was a technical report in 1977.

Our paper was driven by an application to program testing, which was something we worked on at the time. We actually proved what we are calling the Weak Bound. Our paper was strange: we did not actually state a lemma, but gave our result as a free flowing statement. The reason we did this was because our audience was software engineers, and we felt that a very formal looking paper would not be one that they would look at carefully.

The reason we got the Weak Bound is instructive. We tried to bound the probability that the polynomial would be non-zero at a random point. This yields

and not

It is interesting that all three papers had the same two ideas:

A polynomial of one variable of degree can vanish at only points without being identically zero;
A polynomial of variables can be written as a polynomial in one variable whose coefficients are polynomials in variables.

The respective proofs then follow by an induction of some type.

Like Zippel, our audience was not theorists. For us the audience was, as I just stated, software engineers—especially those working on program testing. The main point of our paper was that identities could be tested in far fewer evaluations than

We did not implement our algorithm, but did include an entire half page of data to show how efficient the random algorithm would be for various parameters.

Comparison

I am part of this so it is impossible to be unbiased, but here are some of the key points that I think are clear:

Who was first? This is unclear, if by first you mean something that is not written down in some manner. Schwartz’s Technical report is from 1978, ours is from 1977. But who knows.

Who was first published in a journal? We were the first here: 1978 in Information Processing Letters.

Who was last? Schwartz was the last to discover this great result. His publication in the JACM in 1980 ended all the discoveries. One point from the history of science that is that the credit for a result often, not always, goes to the last person who discovers it.

Who had the weak result? All the players knew that only a polynomial, not an exponential number of evaluations, were enough to check any identity.

Who had the Strong result? Schwartz clearly, I believe, is the only one who had the Strong Bound. I cannot believe that we missed the bound—it is so simple. But Zippel also missed the Strong Bound. I think part of the reason is that Rich, Zippel, and I were thinking of practical applications; thus, the the exact polynomial bound was unimportant to us.

Why Schwartz-Zippel?

The result is usually called the Schwartz-Zippel lemma. Sometimes we are cited too. Mostly not. Oh well.

I can see calling it just the Schwartz Lemma: he was the only one with the Strong Bound. But since most applications need only the weaker results, I can see why Zippel is added. I do not get why we are usually left out. I think there could be two reasons: First, Schwartz and Zippel gave their respective papers at the same conference on symbolic computation in 1979—perhaps that is why they get cited together. Perhaps.

Second, DeMillo and I never pointed out our earlier paper to Schwartz. When Schwartz’s JACM paper came out, I quickly saw there was no reference to us. But I did nothing. Perhaps the lesson today is to immediately say something. Perhaps another lesson is that of Gian-Carlo Rota:

Publish the Same Result Several Times.

Open Problems

What should this result be called? What do you think? Some researchers have told me that they usually refer to all three papers, but they point out that the general idea that identity checking is easy with random points is an “old” idea in the coding theory community. Oh well.

Perhaps, more important is the real open problem that I will discuss another day: can we derandomize identity testing for arithmetic circuits? There have been some exciting partial results and I will discuss those soon.

Finally, I would like to thank Ken Regan for his tremendous help, especially in explaining to me the differences among the three papers. As usual all, errors are mine. Ken promises to add a detailed comment on his thoughts. Again, thanks Ken.

What Are Proofs For Anyway?

rjlipton — Wed, 25 Nov 2009 15:39:23 +0000

How to make a polygon convex and how not to prove it

Paul Cohen was one of the great logicians of the last century, who won the Fields Medal in 1966 for this brilliant work. He, of course, revolutionized set theory when he proved that the Axiom of Choice and the Continuum Hypothesis were both unprovable in the standard formal system called Zermelo–Fraenkel Set Theory. The theory’s history is a bit complex—but the main creators were Ernst Zermelo and Abraham Fraenkel.

Today I thought, with our Thanksgiving Holiday coming up, I would have a short discussion of “why we prove theorems?” It is related to Cohen, and contains a “tasty” result about polygons that I thought might be a good pre-holiday story.

I only met Paul Cohen once, when we both were at a Royal Society workshop on the nature of proof. The main questions discussed were: what is a proof? why do we prove things? and what is the role of machine proofs? It was a wonderful experience. I still do not know the answers to these questions—perhaps I will discuss them in more detail in the future.

I noticed two things about the workshop. First, the senior mathematicians present—including four Fields Medalists—all agreed that proofs were for explanation. They said: The goal of a proof is not to “check” that something is true, but rather to give insight into why it is true. Also they all disliked machine proofs. Cohen was probably the most extreme in his displeasure at the notion of a machine proof.

Second, I noticed how the presentation technologies changed over the two days of the workshop. Computer scientists spoke first, and we all used powerpoint. I even had a short video in my presentation, which included slides with elaborate pictures and diagrams. Then, the relatively young mathematicians spoke. They used powerpoint too, but no videos. Just simple slides, some with a small diagram. Finally, the more senior mathematicians gave their talks, using just plain overhead slides. I will not say anything about my talk, but all the other talks were great, even though the styles were so different.

Then, Paul Cohen spoke. He walked up to the overhead projector with one slide and one pen. He then gave a great lecture: he spoke and now and then wrote a formula on his single slide. It was a brilliant talk and performance. So much for powerpoint.

Convex Flips

The tasty result today is due to Paul Erdös who first conjectured it in 1935. I just stumbled across it recently, while doing some research on another problem. I think it may be connected to an old problem—a disease—that I have had for years. More on that in the future.

The problem Erdös raised, in the Math Monthly, was the following conjecture: Start with any simple polygon in the plane: that is a polygon that does not cross itself. Define a pocket as a maximal connected region interior to the convex hull of the polygon. A flip is the operation of reflecting the pocket across its line of support. The goal is to reach from any simple polygon, a convex polygon, by using just the flips across various pockets. Is that possible?

The following is a figure from a paper on the subject by Branko Grünbaum and Joseph Zaks:

Note, is the result of “flipping” the path from the vertex to .

Erdös’ conjecture was actually too strong, there was a trivial counter-example. Béla Szőkefalvi-Nagy made the statement weaker, proved it, and ever since it has been called the Erdös-Nagy Theorem:

Theorem 1 Starting with any simple polygon, any sequence of flips eventually stops, and the resulting polygon is always convex.

A pretty neat result—no?

Proof or Insight

The result has been rediscovered many times, and also re-proved many times. But these proofs are not proofs. Many, if not all, have subtle errors; it is also interesting to note that the errors made are not even all the same. When I read this, I immediately thought of the workshop on proof—where is the fundamental insight why this process stops at a convex polygon? Somehow, a large number of authors, some quite famous, had missed it.

There is a neat paper published in 2006, by Erik Demaine, Blaise Gassend, Joseph O’Rourke, and Godfried Toussaint on the Erdös-Nagy Theorem. They discuss the history of the problem, the gaps in all the previous proofs, and supply a correct proof. So the theorem is true: flipping a polygon stops at a convex polygon—always. Their paper is extremely well written and its title says it all: “Polygons Flip Finitely: Flaws and a Fix.”

There are many different problems with the existing proofs—see the paper by Demaine, Gassend, O’Rourke, and Toussaint for the details. One general approach is to look at the sequence of polygons that occur,

as the flips are performed. Most argue via a compactness argument that this sequence has a limit polygon . One difficulty is proving that this limit is reached in a finite number of steps. This reminds me of a similar problem I discussed before on the distributive law.

Another difficulty is proving that the limit is indeed a convex polygon: in some of the early proofs it is just stated without any justification, while others gave incorrect arguments.

Open Problems

Given any simple polygon there is a best sequence, and a worst sequence of flips that make it convex. I believe that getting the tight bounds on these is still open.

Have a safe and happy holiday.

New Streaming Algorithms for Old Problems

rjlipton — Sun, 22 Nov 2009 14:19:36 +0000

A streaming algorithm for the classic Dyck language

Claire Mathieu, previously Claire Kenyon, is an expert in the design and analysis of algorithms, which should be no surprise since she worked as a graduate student with two of the world’s best—Philippe Flajolet and Jeffrey Vitter. Claire has done and continues to do some very pretty work on all aspects of algorithms.

Today I plan to talk about a recent paper of Frédéric Magniez, Claire Mathieu, and Ashwin Nayak on streaming algorithms, and how it relates to work that was in vogue years ago.

I recall one day at Princeton when Bob Sedgewick and I had, what we thought, was a great idea. At the time we were working on a front end to TeX, which we called “notech.”

Notech was a preprocessor to TeX that extended some of the ideas that Don Knuth put into TeX: except we carried them to the extreme. Let me explain. Before TeX there were other languages that were typesetting languages. One was called Troff, and was developed at Bell Labs as part of the UNIX project. Troff was not bad—for example, many textbooks were written with it—but it had serious limitations. My first typeset paper was done with it, but Troff’s limits helped drive Knuth to invent TeX.

One of the features of Troff that I hated was that paragraphs were separated from each other by not a blank line as in TeX, but by the symbols .PP. This made the raw text very hard to read, in my opinion. In Troff this is how I would have written this paragraph:

.PP
One of the limits of Troff that I hated was that paragraphs were separated from each other by not a blank line as in TeX, but by the symbols
.ft I
.PP.
.ft R
This made the raw text very hard to read, in my opinion. In Troff this is how I would have written this paragraph:

I will stop here—the danger of recursion scares me.

I loved that TeX used the obvious rule that a blank line meant a new paragraph—what else could it mean? Bob and I worked out a series of similar rules each of the form:

if you typed something in a “reasonable way,” notech would try, via its rules, to guess what you meant, and typeset the material appropriately.

Notech was a pretty good guesser, and for years I used it as my own front-end to TeX. A notech file was mostly plain text—thus the name—except for math formulas. We could never figure out anything that was better than what Knuth invented for math formulas. Although we did use matching and as math delimiters, we left math basically alone. Here is a small part of a paper I wrote in notech:

A key lemma for the lower bound is the following:

Main Lemma: There exists a family of bounded degree directed acyclic graphs {G_n} on {n} vertices and an {\epsilon > 0} so that {G_n} has no {\epsilon n}-segregator with cardinality less than {\epsilon n}.

You might ask, what does this have to do with Claire? She is French and is of course fluent in both English and French. One day at a Princeton tea, Bob and I asked Claire a question: are there two words in French that have the same letters, but have different accents? If there were none, or if such words were rare, then we could program notech to automatically add accents to French words. This seemed to us to be a cool possibility, since accents were a pain to do in notech or TeX.

Her initial response was favorable. She said that she could not, immediately, think of any French words with the same letters, that had different accents. Bob and I were excited, and we explained to her that this would allow us to write a program that would supply the missing accents, since they were redundant.

Redundant did not go over well. Claire walked off and after a few minutes returned with a long list of words that changed with different accents. Our plan was ruined, and she had defended the French language. It would have been a great feature, in my opinion, if accents could be automatically added to a word. I still struggle to get them right—especially on names. Oh well.

Let’s turn to discuss her neat paper on streaming and Dyck Languages, and leave accents behind.

Streaming for Dyck Languages

Frédéric Magniez, Claire Mathieu, and Ashwin Nayak have written a pretty paper titled Recognizing well-parenthesized expressions in the streaming model.

The language is the language of all properly nested parenthesis expressions over two different types of paired parenthesis.

The parentheses ( and ) must be paired, and the parentheses [ and ] must be paired. Thus,

is in the language and

is not. More precisely, the empty string is in the Dyck language; and if is in the language, then so is and . Yes, there is a language , which has three types of paired parenthesis, and has four, and so on. A simple insight is that is already the “general case” for the streaming problem.

They prove:

Theorem: There is a one-pass randomized streaming algorithm for with space and time . If the stream belongs to then the algorithm accepts it with probability ; otherwise it rejects it with probability at least where 0}' title='{c>0}' class='latex' /> is a constant.

They also prove a lower bound that almost matches their algorithm’s space bound. I really like their result. First, it uses a non-trivial method to manage a potentially large stack, which, I believe, could be used in other streaming algorithms. Second, their result is related to an old language theory result that I will discuss in the next section. I like the connection between the current hot topic of streaming and an old topic.

Here is a short overview of how they prove their theorem. As usual please look at their paper for the full proof.

It may help to explain the naïve—note the accent—algorithm for determining whether or not a string is in . The algorithm uses a pushdown, which is initially empty. I will abuse the definitions a bit and call the pushdown a stack: recall a stack, in language theory, allows more operations that just push and pop.

The algorithm processes each input symbol in turn; thus, there are four cases based on the current input symbol:

The input is (. In this case just push the symbol onto the stack.
The input is [. In this case also just push the symbol onto the stack.
The input is ). In this case check that the top most symbol is a matching (. If it is not, then reject; otherwise, pop off the top of the stack.
The input is ]. In this case check that the top most symbol is a matching [. If it is not, then reject; otherwise, pop off the top of the stack.

When there is no more input, the algorithm accepts if and only if the stack is empty. It is not hard to see that this algorithm works correctly with only one pass over the input. Note, the string

will be rejected because at the third input symbol the stack will be

and the input is . Since the top does not pair off with the input, rule (4) forces the algorithm to reject.

Their clever insight is that it is possible to simulate the stack with much less space, than used by the naïve algorithm. Note, the stack can get as large as order in size for a string that is in the Dyck language: just have an input that starts with many ( and [ symbols.

The key idea is that the contents of the stack can be hashed, and in this manner the space requirements can be reduced from order to almost . The hashing is not completely straightforward---they need to carefully look at the types of motions of the stack. They must analyze how the stack can move up and down during the operation of the naïve algorithm. See their paper for the details.

The Connection to Language Theory

The Dyck language, named for Walther von Dyck, is fundamental to classic language theory. It is a special type of context-free language, called linear, but it is ``universal'' in a sense made precise by the beautiful Chomsky-Schützenberger Theorem:

Theorem: A language over is context-free if and only if for some finite alphabet , there is a homomorphism such that

where , the Dyck language over and is a regular language over .

The theorem is, of course, due to Noam Chomsky and Marcel-Paul Schützenberger.

This connection raises some interesting questions between their new streaming result and classic language theory. Can they use their method, for example, to give a streaming bound on all context-free languages?

I will end this discussion by pointing out that I have already discussed a close relative to the Dyck language. Consider the free group on letters. Given a word from the free group, some words, like would trivially equal . Let us call the language defined by such words . is like except that parentheses can cancel in either order: for example,

is now allowed. Sometimes the language is called, for obvious reasons, the two-sided Dyck language.

Zeke Zalcstein and I showed years ago that the following is true:

Theorem: There is an one-pass randomized streaming algorithm for the language with space and time . If the stream belongs to the free group then the algorithm accepts it with probability ; otherwise it rejects it with probability at least where 0}' title='{c>0}' class='latex' /> is a constant.

Well not exactly. As Frédéric Magniez, Claire Mathieu, and Ashwin Nayak generously point out, we proved a theorem that is equivalent to this, but we proved our theorem before there was a even a streaming model. I discussed our result and its strange history here.

Open Problems

Is their upper bound tight? More fun might be to try and see what is the streaming complexity of other languages. In particular, what happens for all context-free languages? What about linear ones?

Why is the Dyck language so much harder in the streaming model than the free group, the two-sided Dyck language?

Also their stack idea is quite ingenious---can similar ideas be used in other algorithms?

[Some minor updates.]

Nash Equilibrium for Sparse Games: Part Deux

rjlipton — Tue, 17 Nov 2009 16:07:01 +0000

How to find approximate Nash equilibrium for sparse games

Constantinos Daskalakis is one of the experts in modern game theory, especially the structure of Nash Equilibrium for non-zero sum games. He has written a wonderful paper with Christos Papadimitriou On oblivious PTAS’s for nash equilibrium. Also see his nice survey for more information.

Today I want to talk about Costis and Christos’ paper as it relates to sparse games. This is another example of the Iceberg Effect: their paper has a beautiful result on game theory, but I missed another of their results.

Constantinos, Costis, was kind enough to point out two things about sparse games after my earlier discussion on games. First, that in his paper with Christos at STOC 2009, they stated a theorem that solves the sparse case of symmetric games. He went on to sketch the proof, which they did not include in their paper: I will give their proof in a moment.

Second, he points out that Shanghua Teng has observed that the uniform distribution is trivially a -Nash equilibrium of a -sparse game—the expected payoff from any row is at most for the row player and similarly for the column player.

His second point is one that I would like to discuss, since it raises an issue by what we mean to approximate a Nash Equilibrium (NE).

Let’s turn to discuss first NE and then their theorem.

Nash Equilibrium: Exact and Approximate

What does it mean to find an approximate NE? I think it will be easier to first explain the main issue involved by an analogy. Anyway I love analogies.

Suppose that you have a polynomial and want to find an so that

of course, we call a root. Usually is not a rational number, so we cannot write it down exactly. Thus, what we do is try to find a so that

where 0}' title='{\epsilon>0}' class='latex' /> is the required precision and are integers.

There is a fundamental problem: simply because is small does that mean that it is near an actual root of ? It does not, in general. What we want are the stronger statements:

The value of is near ;
There is a so that and is also small.

In this case we can say that is an approximate root of the polynomial. If there is no such , then is an interesting point, but it is not correct to call it an approximate zero.

This behavior is analogous to what happens with “approximate NE’s.” There are two types of interpretations of what an approximate NE is:

An -near NE: This is a pair of strategies that are within an of an exact NE.

An -NE: This is a pair of strategies such that a player who defects from their strategy can gain at most an .

Costis, in his survey, points out that the first notion is reasonable, but the latter is the one that we usually use: Note also that it is easy for a player to check the approximate optimality of the actions used by her mixed strategy. So the notion of an approximate Nash equilibrium is much more appealing and, in fact, this is the notion most commonly used.

Note, the analogy to the root problem: in the first we are near a root, and in the second we are at a small value. As Costis points out the latter can be checked easily, while checking that a point is near an exact root is not as easy.

Sparse Games

I will now state their theorem:

Theorem: Suppose that is an matrix with entries in , such that each row and column in has at most ’s. Then, for the symmetric game defined by , we can find an -NE in time bounded by

Here is a sketch of their proof:

Proof:

The key insight is that it is enough to consider strategies for the row and column player where the probabilities are integer multiples of . The number of such strategies for either player is at most

This follows since the number of strategies are bounded by the number of ways to place objects into “bins.”

We now have a “small” set of strategies, the following argument will show that one of these strategies contains a pair that is an -near NE. Consider an exact NE say . Notice that there exist strategies such that the following are satisfied:

For all : and .
For all : if then , and if then .

Consider . If , then set to . Otherwise, set to the nearest multiple of . This almost works: the small issue is that if is not a exact multiple of , then do we round up or down? Either rule will make satisfy the needed conditions above. However, we also need to be a probability vector: it is easy to see that by rounding some up or down we can adjust to satisfy

which will make it a probability vector. The same method works for . Now we need to prove that this strategy pair is an -near NE. The payoff for the row player is and for the column player is . Let denote a unit vector with at -th coordinate. For notational convenience, we omit the transform symbols on the vectors. The usage should be clear from the context.

Notice that the pair of strategies and the NE satisfy:

, for all .
, for all .

The first claim follows from the fact that

The second claim is analogous. This is where we use that has at most ’s in each row/column. Therefore, the total error from the NE is at most

It follows from the above that is an -near NE of the game.

The above proof shows the existence of an -near NE in the search space formed by the strategies that are multiples of . This strategy pair is near the exact NE . However, because of the difficulty of searching for an -near NE, an exhaustive search in this space can only guarantee that we would find an -NE, and not an -near NE.

Open Problems

An interesting open problem is to try to found an -near NE. The above proof shows that it is in a relatively “small” list of strategies. Is it possible to discriminate and determine which are near exact NE’s?

The above proof gives us a PTAS for games with a bounded number of 1’s in each row and column. However, for matrices with 1’s in all, the problem of finding a PTAS is still, I believe, open.

More on Mathematical Diseases

rjlipton — Thu, 12 Nov 2009 20:47:48 +0000

A summary of some your ideas on mathematical diseases

John Conway is a world renowned mathematician, who defies a simple description. He has worked on countless games, puzzles, and easy to state, but often hard—if not impossible—to solve problems. These range from his classic game of Life, to his work on Surreal numbers; from his work on polyhedra, to his special notation for huge numbers. At the same time he has made deep contributions to many, if not most areas, of modern mathematics: from group theory, to number theory; from algebra, to geometric theory. There is only one John Horton Conway.

Today I want to talk about some of the mathematical diseases that were raised by those who were kind enough to comment on my previous discussion.

The response was so strong that I thought I would collect some of your comments in one place. I hope that this either helps someone to make progress on one of these diseases or to help spread them to others.

Conway was the source many of the popular MD’s, which is probably not too surprising given his wide range of results, that includes many unusual problems.

He once gave the keynote address at SODA. This conference, the Symposium on Discrete Algorithms, is theoretical, but a bit more down to earth than FOCS, for example. Conway’s presentation was a strange and wonderful one—at many levels. The main part of the talk was an impressive demonstration of his Doomsday algorithm. This algorithm allows one to calculate the day of the week from any date by a “simple” rule—Conway himself can do this in realtime. Thus,

The funniest part of his demonstration was that people would ask dates and get back days of the week, but for many of them we had no idea if Conway was really correct or not. Oh well.

Personally, of his many theorems, his 15-Theorem is one of the neatest:

Theorem: If a positive definite integral quadratic form represents all positive integers up to 15, then it represents all positive integers.

Conway proved this with William Schneeberger in 1993: see this for a overview of the result. Forget how one proves such a theorem, my question is more basic: where do you get the intuition that such a theorem might even be true?

Let’s turn now to the previous comments, with some extra annotation, here and there.

Some Mathematical Diseases (MD)

Conway’s Thrackle Conjecture: Joseph O’Rourke suggests this one: which he says “ bites me about once every two years”

I did not know this conjecture. See this for a description of this amazing simple sounding problem. László Lovász has worked on the problem so this disease can affect even one of the best mathematicians in the world.

Conway’s Game of Life: John Sidles suggest this one and, adds “What a great subject!” He says, To lead off, a wholly benign, utterly useless, and wonderfully enjoyable MD is the study of self-replicating structures in Conway’s Game of Life. The accomplishments of the Life community over the last 39 years are so amazing, that all one can say is Golly!

The Game of Life is played on an infinite grid of squares. It is not really a game, it is a zero player game, one starts off by providing the seed — by marking some finite set of squares in the grid. Then the game is entirely deterministic, at each time instance, it evolves using 4 simple rules. Conway’s original question was : Is there a seed which can grow indefinitely? For the answer to this, and a beautiful exposition on the game itself, please see the this article on the game.

Collatz Problem: Akash Kumar suggests the famous conjecture, which is also called the Collatz Conjecture. He says, “This `seemingly’ toy problem has much to offer as shown by its resistance to attempts at solving it.”

I was introduced to this famous problem, while I was a graduate student, by Albert Meyer. I tried to show that the mapping had no cycles by a modular argument. It failed. Somehow the problem never has appealed to me again. Definitely, a simple to state, but very hard problem.

Palindromic Number Conjecture: Ted Carroll suggests this one. He says, “I suck non-mathematicians in with that one all the time—especially computer people because there’s a proof that the conjecture holds for binary.”

Start with a number and reverse it, then add the two together. Repeat until a palindrome is reached. Does this always happen? See this for an example and more:

which is a palindrome.

Specifically, the number has attracted lots of attention. Starting with , after being iterated to million digits a palindrome is yet to be found. Curiously, in the binary case, there is a very short counterexample that can be shown to never reach a palindrome.

Goldbach Conjecture: Akash Kumar and Tom both suggest this one. Akash says, “I mean, I find it really amazing that this problem is accessible to people like me with a modest mathematical background and is so profound that it has avoided attacks by best brains.”

Tom says that “Everyone has dreamed about solving these at least a little bit at some stage in their life. There’s even a (fictional) book written about this problem about a man obsessed with solving it.”

The book titled, “Uncle Petros and Goldbach’s Conjecture” by Apostolos Doxiadis is well written and fun. I would definitely recommend it to you. See this for other examples of Goldbach’s Conjecture appearing in popular culture.

The best known results about the Goldbach Conjecture are:

Every sufficiently large odd number is the sum of three primes. This result is due to Matveevich Vinogradov.
Every sufficiently large even number is the sum of a prime and the product of at most two primes. This result is due to Chen Jingrun. He proved it during the “culture revolution” in 1966, and at first the western mathematicians did not believe that he had achieved this great result. He had.

Splay Conjecture: Mihai Pătraşcu suggests this one. This conjecture concerns the behavior of certain tree data structures. See this for an introduction to the question. A real computer science question.

Graceful Tree Conjecture Ibrahim Cahit and Shiva Kintali suggested this one. Cahit explains, Let me give short explanation why GTC is a mathematical disease. Alexander Rosa has identified essentially three reasons why a graph fails to be graceful: (1) has “too many vertices and not enough edges,” (2) “has too many edges,” and (3) “has the wrong parity.” If for any graceful tree an arbitrary vertex can be assigned the label 0 (rotatable tree) then the proof of the GTC would be piece of cake. Unfortunately not all trees are rotatable. Similarly if all trees are alpha-valuable (a kind of balanced labeling stronger than graceful (beta-valuable) labeling) then the proof of GTC follows easily. What remains is an algorithmic proof attempt based on the induction on the diameter (the length of the longest path in a tree) of a tree. Unfortunately most of the trees with diameter greater than have no alpha-labeling. In the past I have attempted twice (once in 1975, settled a class of symmetric trees and again 1980, settled all trees of diameter ). I didn’t give up, it is a disease after all.

Despite of huge efforts very little known for about graceful trees e.g., any tree with vertices has a graceful labeling and all trees of diameter up to are graceful.

Riemann Hypothesis: Subrahmanyam Kalyanasundaram suggests this one. He says, “Although not so easy to state, I think Riemann Hypothesis has attracted quite a lot of attention. See here for a list of attempted proofs.”

Two mathematicians from Purdue recently made independent claims, incorrectly, that they have proved the famous theorem. The Riemann has been claimed many times previously by both amateurs and professionals. I asked some experts the other day what is up with this great problem. The answer was there seems to be no progress; the conjecture is still as unreachable as ever.

4-Color Theorem: Gilbert Bernstein and Gil Kalai both suggest this one. Gilbert says that “633 configurations is still too many!”

Kalai adds, Once I had some idea about 4CT (which asserts that every planar cubic graph is 3-edge colorable) and relating it to Tverberg’s theorem, and I remember Laci Lovász asked me: “Can you use your approach to prove that a bipartite cubic graph is 3-edge colorable?” (Which is an easy graph theory result.) Dealing with bipartite cubic or even with bipartite planar cubic graphs looks like a good test-case for various hypothetical approaches.

I have outlined an approach that we have suggested for a “human” proof in a previous discussion. Roughly, we show that even proving that every planar cubic graph is approximately -edge colorable, is enough to prove 4CT. Still we are unable to prove that this is true.

Reconstruction Conjecture: Harrison and Ryan Williams suggest this one. Harrison explains why the conjecture is related to the GI conjecture:

As someone who’s been bitten by the reconstruction bug, I’ll tell you that it is indeed related to graph isomorphism on a fundamental level. Here’s a brief sketch of why:

If we label the vertices of a graph, then reconstruction is trivial—we just glue the induced subgraphs together so that the labels match up, and in fact we only need three induced subgraphs to reconstruct our original graph. The reconstruction conjecture is that, if we forget the labels, then this “gluing” process is still unique (up to graph isomorphism, of course). This feels like it should be intuitively true, but there’s a crucial problem: namely, if the graph has a large automorphism group, then its induced subgraphs can get “put into” the original graph in many different ways, and it’s not clear that the global properties of the graph don’t change.

So much of the work on graph reconstruction centers around understanding just how automorphism groups of graphs behave, and how they relate to combinatorial properties. (From the other direction, by the way, it’s an old result of Béla Bollobás that almost all graphs are reconstructible, since random graphs don’t allow us to embed large subgraphs into them in more than one way.)

Ryan Williams adds, Allow me to get a little bit infected If we assume the graph reconstruction conjecture, does this imply anything interesting about the complexity of graph isomorphism? If you want to tell whether two -node graphs are isomorphic, and you know that this “reduces” to checking whether there is an “isomorphism matching” between two sets of graphs on nodes each, can you get a recursive algorithm?

I have to say that I love this conjecture, but have some immunity—I have never thought about it all.

The Status of : This was suggested by Ninguem. He says:

Roger Apéry was a professor at a small French university. He was past the age most mathematicians prove big theorems, he had a history of bad proofs, not a big research output and, I was told, also an alcoholic. But he was a professional mathematician and not a crank. He showed that is not rational. We still don’t know whether is transcendental.

An anonymous commenter adds, The reason people were initially skeptical was that Apéry gave a very weird talk presenting the proof. In the talk, he stated some implausible-looking identities and recurrences, with no hint of how to prove them, and he showed that they implied the irrationality of . He apparently didn’t respond well to questions, and this led people to wonder whether he actually had a proof of the identities. Maybe he had just conjectured them based on numerical experiments, and perhaps they weren’t even true. He eventually came out with a paper that proved everything, and he probably had the proofs at the time he gave the talk, but he certainly didn’t explain it at all clearly at that time. I think it wasn’t until Don Zagier came up with his own proofs of Apéry’s assertions that everyone became convinced it definitely worked (although everyone got very excited even before that, once they checked everything numerically and found that it all seemed to work).

I heard that when Apéry wrote on the board the key identity he needed,

he gave a very strange answer to “where did this identity come from?” He is alleged to have answered, “they grow in my garden.” Obviously, this did not help make people feel comfortable. The identity is wonderful, the proof is correct, and the values of the zeta function at other odd integers is still a mystery.

Finding New Crypto-Systems: Chris Peikert suggests this one. He says: As one of the many who search for new cryptosystems, allow me to defend the affliction (before someone develops a vaccine)! Sure, factoring- and discrete-log-based systems are great for Alice and Bob’s everyday secret messages, but

What if you don’t believe that these problems are truly hard?
What if someone builds a quantum computer? (What if it happens sooner rather than later?)
What if your device is constrained and can’t handle 2048-bit exponentiations?
What if you need “extra features,” like delegation, revocation, or homomorphisms?
What if you want to be guaranteed security even if some/most of your secret key leaks out via a side channel?

Your standard-issue cryptosystems don’t admit very satisfactory answers to these questions. That’s why we need to look for more

I agree with all he says, but I still think the search for new systems has a bit of an MD flavor, however.

Rudrata Problem: Proaonuiq suggested this one. He says, In the Harary paper they cite the Rudrata disease, an old and widespread one. I like specially the RCD-variant (Rudrata problem for Cayley Digraphs), also old and widespread. Be aware: the later infection is harder to cure than the more general Rudrata disease, since the problem seems simpler!

I was puzzled by the reference to the “Rudrata disease,” since I had not heard of it before. I found out that it is another name for the Hamilton cycle problem, which is of course NP-complete. A special case of the Hamilton cycle problem is the famous knights tour problem on a chessboard. The pattern of moving a knight on a half-chessboard was presented back in the century by the Kashmiri poet Rudrata in a Sanskrit poem. Hence the name “Rudrata disease.”

Thus, this open question is about understanding the structure of Hamiltonian cycles on special graphs that arise from groups—Cayley Digraphs. I can see why this could be an MD.

I have to add a comment on a related but completely different topic: how good are you at chess? Moving knights on a chessboard can be used to rate your chess ability. There is a simple test: One places four pawns on the chessboard at certain locations. Your job is to move the knight from one corner of the board to the other as fast as possible. You must visit all squares in a certain order, and can never visit the squares that are occupied by the pawns. You can have the knight visit a square more than once. The time that you take to do this task apparently correlates very well with your chess ability. I was given the test by a friend, and did not do very well.

Long List of Problems: These were all suggested by Gil Kalai. He says the following: (I have made some minor edits—I hope that I have not changed any content by mistake.)

Let me adopt the MD term under a slight protest and mention some of my favorites MD’s that I spent most time studying.

The rate of error-correcting binary codes (and spherical codes).

(Infected by Nati Linial) A very easy to describe problem. You want an error correcting binary code on bits with minimal distance . What is the largest possible rate as a function of . (A closely related problem is about the densest sphere packing in high dimensional spaces.)

Is the Gilbert-Varshamov lower bound the correct one? Can the MRRW upper bound (the best known one) be (even slightly) improved? A (somewhat) related (easier) problem: Can you find an elementary construction (not based on algebraic geometry) for large-alphabets codes with better rate than Gilbert-Varshamov. (The zig-zag success for expanders give some little hope.)

The -conjecture for spheres:
This is probably the first problem on my list. I started working on it the earliest and probably spent more time on it than on any other. It is a little hard to explain and motivate. But there are quite a few people who thought about the problem. (There are a few posts about it on my blog).

The Hirsch conjecture (and strongly polynomial LP)
I wrote about it amply on my blog.

The Erdös-Rado Delta system-conjecture
This is on Gowers’s possible future polymath projects so let me not elaborate further here.

The Cap set conjecture
It is about the largest size of a subset of not having three elements that sum to zero. Closely related to Roth’s and Szemerédi’s theorems.

Borsuk’s conjecture.
I don’t think Jeff Kahn and I were “obsessed” about the problem while working on it for quite a few years. It is not clear if an obsession mode is a good sign.

Our approach to the problem is somewhat related to a famous open problem which is still open and is on Alexander Rosa’s list and was always high on Jeff’s list: The Erdös-Faber-Lovász conjecture. (Jeff settled the EFL conjecture skepticism and Jeff and Paul Seymour solved it fractionally.)

Bible codes
This represents an applied topic that I intensively (and obsessively) spent much time in the late 90’s. At the end I was a coauthor of a 4-author paper containing a thorough refutation of the scientific evidence for the existence of bible codes. It was a good (while strange) introduction for me on various issues regarding statistics, science, Learnability, even philosophy of science.

Learnability vs rationality
One tempting “cure” for various diseases, especially of conceptual nature, is “learnability” via VC-dimension. I was very optimistic at some time about the usefulness of replacing “rational” by “learnable” in the foundations of theoretical economics.

Infeasibility of quantum computers
This represents a current main research interest.

PNP and related issues
It is probably a good instinct whenever you study some new notion about Boolean functions (or simplicial complexes which are just monotone Boolean functions) to spend a little (let me repeat: a little) time on thinking: does this new notion has bearing on PNP or other questions in computational complexity? Most often you can easily realize that the answer is no, and sometimes you realize that the answer is no after some more effort.

A little flirt with Poincaré
I was interested in triangulation of manifolds for which the links of vertices are of the simplest possible kind: stacked spheres. (They are the boundaries of a set of simplices glued together along facets.)

For dimension greater than three, I proved that such a simply connected manifold is a sphere. For dimension three, I could not prove it and it is a very very very special case of the Poincaré conjecture. (I still cannot prove this special case directly.) If you drop the assumption that the manifolds are simply connected then for 3}' title='{d>3}' class='latex' /> you are left with very simple handle body manifolds. I do not know (and am curious to know) which -manifolds have a triangulation where are links are stacked spheres.

Open Problems

Solve some of these MD’s. Or suggest some others. One of my favorites that is missing is the power of polynomials over composite moduli.

Rumors and Playing Games

rjlipton — Mon, 09 Nov 2009 04:27:12 +0000

A rumor from FOCS on approximate Nash Equilibrium is partially true

Paul Spirakis is a senior researcher who has made many important contributions to theory. He has hundreds of publications that cover many areas of theory. What is so impressive about Paul is that he has been able to blend theory and practice in a very fruitful way. This is a pretty unique skill that few have.

Today I want to talk about a new result of his on finding approximate Nash Equilibrium for non-zero sum games.

The coolest rumor I heard at this year’s FOCS was that “someone” had proved a sub-exponential bound on the running time of an algorithm for approximate Nash Equilibrium. This was quite exciting and it is what I refer to in my discussion of Theory Day. I eventually tracked the rumor down and discovered the source was the pretty paper that I plan to discuss today. The paper does not quite have such a result, but definitely makes an important step in our understanding of games.

The rumor that I heard at FOCS turned out to be wrong. Oh well. That’s the nature of rumors: by definition a rumor is not always true; otherwise, it would not be a rumor. At least this rumor was partially true.

Years ago, in 1969, there was a great rumor that started right after Steve Cook proved his famous theorem that characterizes polynomial time. He proved:

Theorem: The following three conditions are equivalent for any language :

is accepted by some deterministic auxiliary pushdown machine with logspace storage.
is accepted by some non-deterministic auxiliary pushdown machine with logspace storage.
is in P.

This is a beautiful result, which in my opinion was not expected. An auxiliary pushdown automata with auxiliary logspace storage is a machine that can read the input two-ways, and store information on a regular Turing tape of length at most . In addition, the machine has a pushdown, which it can use also to store information.

Note, a pushdown only allows operations on the top: the top symbol can be read, a symbol can be pushed onto the pushdown, or a symbol can be removed from the top. That’s all. Symbols below the top symbol cannot be read without popping off the top symbol.

Almost immediately someone well known, I will call them X, claimed that Steve’s result could be extended “slightly.” In particular, X claimed that the same theorem could be proved with pushdown replaced by stack. A stack is not the same as a pushdown. It has one important extra property: a stack allows the reading of the symbols in the pushdown without popping off any symbols. A stack only allows this in a read-only mode. That is when inside the pushdown, symbols can only be read. Symbols can be changed at the top as usual.

We could not figure out how the new proof went, and worse X went on a camping vacation. So he was unreachable—this is well before cell phones. The theory community was all abuzz, since it was believed that an auxiliary stack machine could accept more than P. It would have been a great result. Clearly, X did not see this, or he would have been more circumspect about his claim.

Finally, X returned and was asked how his proof went. He explained the proof, and it was immediately clear that the proof did not work. He had forgotten one small point: a stack machine could run for more than exponential time unlike a pushdown machine. [I fixed an error here.] He had to retract his claim. Sometimes rumors are false.

Now let’s turn to non-zero sum games and a new approach to getting approximate Nash Equilibrium.

Win-Lose Games

The paper of Haralampos Tsaknakis, and Paul Spirakis is mostly a new approach to finding approximate Nash Equilibrium (NE) for general two player game through a reduction to a restricted class of games.

This class is special, but still quite interesting. It is the class of symmetric win-lose games. Such a game has one by - matrix : the matrix is the payoff for the row player and is the payoff for the column player.

The restriction to - values is why it is called a “win-lose” game, and the fact that one player uses the transpose of the other player’s matrix is why it is called symmetric. Finding a Nash Equilibrium in even win-lose games is as hard as the general case thanks to the beautiful result of Tim Abbott, Daniel Kane, and Paul Valiant.

Their proof is quite clever. As someone who has worked a bit on games, I was impressed that they could prove this result. In many complexity situations the - case is universal; however, proving that for games is not so easy. My intuition was that this, while probably true, would be very hard to prove.

Approximate Solutions

Since the problem of finding exact Nash Equilibrium for even win-lose games is as hard as solving the general case, it is natural to look for approximate NE’s. This is exactly what Haralampos and Paul do in their paper. They compare their new result with a result of Evangelos Markakis, Aranyak Mehta, and myself, which we proved in a earlier paper:

Theorem: For any 0}' title='{\epsilon>0}' class='latex' /> there is an algorithm that finds an approximate Nash equilibrium for a symmetric win-lose game of size , in running time bounded by where .

The main result of Paul’s paper is:

Theorem: For any 0}' title='{\epsilon>0}' class='latex' /> there is an algorithm that finds an approximate Nash equilibrium for a symmetric win-lose game of size , in running time bounded by where . The value of is equal to:
0} \lambda_{k} ' title='\displaystyle \sum_{\lambda_{k} > 0} \lambda_{k} ' class='latex' />

where are the eigenvalues of the graph induced by the game matrix.

This result is neat, in my opinion, since it addresses the barrier discovered by Constantinos Daskalakis and Christos Papadimitriou in their paper titled “On oblivious PTAS’s for Nash Equilibrium.” Constantinos and Christos show essentially that to get good approximations for NE’s, one must look at the structure of the payoff matrices: it is not enough to just examine the strategies as we did in our result.

Sparse Games

I am currently running an open problem seminar and we discussed Paul’s paper in a recent class. We noticed first that the value of in his theorem could be changed from

0} \lambda_{k} ' title='\displaystyle \sum_{\lambda_{k} > 0} \lambda_{k} ' class='latex' />

This follows since the sum of eigenvalues is , so the new measure is exactly twice the old, and this will have no effect on the results—all can be swept under the constant in the big-O notation.

Then, we considered sparse games: games where the induced graph has edges. For such games the value of the critical parameter is bounded by . The proof of this is quite simple:

1} | \lambda_{k} | \\ &\le& n + \sum_{\lambda > 1} \lambda_{k}^{2} \\ &\le& n + \sum_{k=1}^{n} d_{k} \end{array} ' title='\displaystyle \begin{array}{rcl} m &=& \sum_{\lambda \le 1} | \lambda_{k} | + \sum_{\lambda > 1} | \lambda_{k} | \\ &\le& n + \sum_{\lambda > 1} \lambda_{k}^{2} \\ &\le& n + \sum_{k=1}^{n} d_{k} \end{array} ' class='latex' />

where is the degree of the vertex. This last quantity is easily seen to be at most if the graph is sparse.

Based on these simple insights my students Atish Das Sarma, Subrahmanyam Kalyanasundaram, and Shiva Kintali believe that they can prove the following theorem:

“Theorem”: There is a PTAS for finding approximate NE’s for the class of symmetric win-lose games whose induced graph has bounded degree.

Unfortunately, the proof of this, while it looks like it will work, does not seem to be able handle sparse graphs. The reason is that a sparse graph could fail to have some technical properties that Paul’s theorem requires. We are hopeful that this can be fixed.

For random sparse symmetric win-lose games they do believe that it should be possible to prove the following theorem:

“Theorem”: There is a PTAS for finding approximate NE’s for the class of random sparse win-lose games that works with high probability.

There has already been some nice work on random games by Imre Bárány, Santosh Vempala and Adrian Vetta. Their result works for dense games where the values are Gaussian, while we are looking at sparse games with - values.

Open Problems

The major open problem is: is there an algorithm that runs in time for any non-zero sum game and finds an approximate Nash equilibrium? Can one even prove this for a special class of games?

On Mathematical Diseases

rjlipton — Wed, 04 Nov 2009 23:29:49 +0000

Mathematical diseases: symptoms and examples

Underwood Dudley is a number theorist, who is perhaps best known for his popular books on mathematics. The most famous one is A Budget of Trisections, which studies the many failed attempts at the ancient problem of trisecting an angle with only a ruler and a compass. This problem is impossible, yet that has not stopped some people from working day and night looking for a solution. Trying to find such a solution is an obsession for some; it’s almost like they have a malady that forces them to work on the problem.

Today I plan on talking about other mathematical obsessions. They are like diseases that affect some, and make them feel they have to work on certain mathematical problems. Perhaps P=NP is one?

Dudley’s book is quite funny, in my opinion, although it does border on being a little bit unkind. As the title suggests, in “A Budget of Trisections,” he presents one attempt after another at a general method for trisecting any angle. For most he points out that when the angle is equal to some value what the exact error is. For others he adds a comment like:

This construction almost worked, if only the points and and had really been co-linear it would have worked. Perhaps the author could move

His book is about the kind of mathematical problems that I will discuss today: problems that act almost like a real disease.

I cannot resist a quote from Underwood that attacks bloggers. Note he uses “he” to refer to himself in this quote:

He has done quite a bit of editing in his time–the College Mathematics Journal for five years, the Pi Mu Epsilon Journal for three, the Dolciani Mathematical Expositions book series (six years), and the New Mathematical Library book series (three years). As a result he has a complete grasp of the distinction between “that” and “which” (very rare) and the conviction that no writing, including this, should appear before the public before passing through the hands, eyes, and brain of an editor. Take that, bloggers!

(Bold added by me.)

Oh well.

What Is a Mathematical Disease?

This is the flu season in Atlanta, and many are getting it. I hope you either miss the bug, or if you are unfortunate enough to get it, get a mild case.

There is another type of “bug” that affects mathematicians—the attempt to solve certain problems. These problems have been called “diseases,” which is a term coined by the great graph theorist Frank Harary. They include many famous problems from graph theory, some from algebra, some from number theory, some from complexity theory, and so on.

The symptoms of the flu are well known—I hope again you stay away from fever, chills, and the aches—but the symptoms for a mathematical disease (MD) are less well established. There are some signs however that a problem is a MD.

A problem must be easy to state to be a MD. This is not sufficient, but is required. Thus, the Hodge-Conjecture will never be a disease. I have no clue what it is about.
A problem must seem to be accessible, even to an amateur. This is a key requirement. When you first hear the problem your reaction should be: that is open? The problem must seem to be easy.
A problem must also have been repeatedly “solved” to be a true MD. A good MD usually has been “proved” many times—often by the same person. If you see a paper in arXiv.org with many “updates” that’s a good sign that the problem is a MD.

Unlike real diseases, MD’s have no known cure. Even the solution of the problem will not stop attempts by some to continue working on it. If the proof shows that something is impossible—like the situation with angle trisection—those with the MD will often still work hard on trying to get around the proof. Even when there is a fine proof, those with the disease may continue trying to find a simple proof. For example, Andrew Wiles’ proof of Fermat’s Last Theorem has not stopped some from trying to find Pierre de Fermat’s “the truly marvellous proof.”

Some Mathematical Diseases

Here are some of the best known MD’s along with a couple of lesser known ones. I would like to hear from you with additional suggestions. As I stated earlier Harary was probably the first to call certain problems MD’s. His original list was restricted to graph problems, however.

Graph Isomorphism: This is the classic question of whether or not there is a polynomial time algorithm that can tell if two graphs are isomorphic. The problem seems so easy, but it has resisted all attempts so far. I admit to being mildly infected by this MD: in the 1970’s I worked on GI for special classes of graphs using a method I called the beacon set method.

There are some beautiful partial results: for example, the work of László Babai, Yu Grigoryev, and David Mount on the case where the graphs have bounded multiplicity of eigenvalues is one of my favorites. Also the solution by Eugene Luks of the bounded degree case is one of the major milestones.

I would like to raise one question that I believe is open: Is there a polynomial time algorithm for the GI problem for expander graphs? I asked several people at the recent Theory Day and no one seem to know the answer. Perhaps you do.

Group Isomorphism: This problem is not as well known as the GI problem. The question is given two finite groups of size are they isomorphic? The key is that the groups are presented by their multiplication tables. The best known result is that isomorphism can be done in time . This result is due to Zeke Zalcstein and myself and independently Bob Tarjan. It is quite a simple observation based on the fact that groups always have generator sets of cardinality at most .

I have been affected with this MD for decades. Like some kind of real diseases I get “bouts” where I think that I have a new idea, and I then work hard on the problem. It seems so easy, but is also like GI—very elusive. I would be personally excited by any improvement over the above bound. Note, the hard case seems to be the problem of deciding isomorphism for -groups. If you can make progress on such groups, I believe that the general case might yield. In any event -groups seem to be quite hard.

Graph Reconstruction: This is a famous problem due to Stanislaw Ulam. The conjecture is that the vertex deleted subgraphs of a graph determine the graph up to isomorphism, provided it has at least vertices. It is one of the best known problems in graph theory, and is one of the original diseases that Harary discussed.

I somehow have been immune to this disease—I have never thought about it at all. The problem does seem to be solvable; how can all the subgraphs not determine a graph? My only thought has been that this problem somehow seems to be related to GI. But, I have no idea why I believe that is true.

Jacobian Conjecture: This is a famous problem about when a polynomial map has an inverse. Suppose that we consider the map that sends a pair of complex numbers to where and are both integer polynomials. The conjecture is that the mapping is 1-1 if and only if the mapping is locally 1-1. The reason it is called the Jacobian Conjecture is that the map is locally 1-1 if and only if the determinant of the matrix

is a non-zero constant. Note, is the partial derivative of the polynomial with respect to . The above matrix is called the Jacobian of the map.

This is a perfect example of a MD. I have worked some on it with one of the experts in the area—we proved a small result about the problem. During the time we started to work together, within a few months the full result was claimed twice. One of the claims was by a faculty member of a well known mathematics department. They even went as far to schedule a series of “special” talks to present the great proof. Another expert in the area had looked at their proof and announced that it was “correct.” Eventually, the talks were cancelled, since the proof fell apart.

Crypto-Systems: This is the quest to create new public key crypto-systems. While factoring, discrete logarithm, and elliptic curves seem to be fine existing public key systems, there is a constant interest in creating new ones that are based on other assumptions.

Some of this work is quite technical, but it seems a bit like an MD to me. There are amateurs and professionals who both seem to always want to create a new system. Many times these systems are broken quite quickly—it is really hard to design a crypto-system.

A recent example of this was the work of Sarah Flannery and David Flannery in creating a new system detailed in their book In Code. The book gives the story of her discovery of her system, and its eventual collapse.

P=NP: You all know this problem. See this for a nice list of attempts over the years to resolve the problem. Thanks to Gerhard Woeginger for maintaining the list.

Open Problems

What are other MD’s? What is your favorite? Why do some problems become diseases? While others do not?

I would love to see some progress made on group isomorphism—I guess I have a bad case of this disease. I promise that if you solve it I will stop thinking about it. Really.

The Iceberg Effect in Theory Research

rjlipton — Sat, 31 Oct 2009 15:06:36 +0000

The iceberg effect in research: how theorems can be lost

Fred is my favorite name, when I need a “random” name. You might have also noticed that the picture is not even a picture of a person—it’s an iceberg. I will explain why in a moment.

Usually I talk about real people, but today I thought I would use phony names to protect the innocent—or at least protect my friends.

I plan to talk about an issue that comes up in research:

Is the fact X a new result? Or is it a known result?

At FOCS, a great researcher, who I will call “Fred,” asked me if a certain theorem, he had just proved, was new. He did this in the hall right outside one of the conference rooms, which did not allow us much time or any blackboard for drawing pictures. His result concerns a clever algorithm that yields a neat approximation algorithm to a well studied problem. He asked me and others if it was “new.” My immediate response was that I knew a related theorem, but that Fred’s theorem seemed new to me.

Moreover, Fred’s theorem seemed to generalize the known theorem, which made me excited so I began to write an entire post on his result. In the post I talked about his theorem, its proof, and how it differed from the old known result. In order to do the latter I had to Google until I found the old paper. This was not a completely trivial task, since the paper was so old. I try to be careful in how I cite papers and wanted to see the original. That is where I hit a snag: his “new” theorem was about years old. It was “known.” Or was it?

The Iceberg Effect

The confusion about whether or not Fred’s theorem was new could be traced to what I will call the iceberg effect. Often an author may write up a paper that becomes famous for some theorem T. But also in the paper there are other theorems, which are not nearly as exciting as the main theorem T. Over time everyone knows T, we teach T to our students, we use T in our papers, and T becomes part of the fabric of theory.

The problem is that T is like the visible part of an iceberg. We see the tip, T, that is above the water, but the part below, the other theorems, are soon forgotten. They may be quite clever or hard to prove, but people just know about T. This is where the iceberg effect hits. You prove a “new” theorem, which is not new—it’s part of the invisible theorems that are below the water line.

This is exactly what happened to Fred. He lost a theorem, and I lost a post. Oh well.

Other Effects

There are many other effects that make it hard to know for sure that something is new. I think that this problem is less of an issue in “hot” areas, since the results are fairly recent.

For results that are older the iceberg effect and other effects can be a major problem. There are so many on-line papers, workshops, conferences, journals, and other sources of information that it is easy to overlook something. This is not unique to theory, but it is an issue that we need to work on.

The Theorem In Question

Dick Karp proved a theorem about TSP for random points in the unit square. His theorem is based on a famous result of Jillian Beardwood, John Halton and John Hammersley who prove that the length of the optimal TSP tour for uniform independent random points in the unit square is .

Karp proved his famous result in this paper:

Theorem: Let random points be in the unit square, and let the length of their optimal tour be . Then, there is a polynomial-time algorithm that given the points finds a tour of length so that

Since the optimal tour is , this result shows that the tour found is close to the optimal length: the error term is low order.

This is the T result. The “new” theorem was:

Theorem: Let arbitrary points be in the unit square, and let the length of their optimal tour be . Then, there is a polynomial-time algorithm that given the points finds a tour of length so that

What Karp proved in his original paper is much stronger than his theorem on random points, which is the T result that we all remember. He actually proved the new result in one of his below-the-waterline theorems. He proved exactly the above theorem, but it was stated in slightly different language:

For a more recent treatment of Karp’s result check out this.

The Proof

I will include a sketch of Karp’s theorem, with a weaker bound. Actually, this is how Fred sketched his “new” result, when he spoke to me in the hall. Check out Karp’s paper or the notes for the better bound. The key ideas are all here, however.

Suppose that points are given that all lie in the unit square. The points need not be random.

The algorithm has three steps.

Step 1: Divide the square into small square cells of side where is a slow growing function. Then, if a small cell has more than point in it, connect all the points together at their respective centroid. Now each small cell can be thought of as having at most point—call these points the leaders. The error introduced by this step is at most , which is .

Step 2: Next divide the square into large square cells of side . Each large cell contains at most leaders, since a large cell contains at most small cells. Use a brute force dynamic program to solve the TSP for each large cell exactly. This causes no error, but takes time at most,

If , then this runs in polynomial time.

Step 3: Finally, connect all the large cells together by a simple path: the path snakes across each row of large cells. The total length of this path is linear in the number of rows of the large cells, . This path can be all error so the total error from each step is at most:

Since is , the total error is bounded by , which implies the theorem.

Open Problems

I may be alone, but I often run into this question of what is known. The iceberg effect is one barrier that I have hit. But there are other issues. Can we do a better job of making sure that we do not re-invent what is known?

Or was it really known if top researchers did not recall Karp’s “other” result? Perhaps the outcome here is good—I had forgotten that Karp’s ideas worked just fine for any collection of points. So perhaps the whole affair has worked out just fine.