Savitch’s Theorem: Opportunities Missed and Found
Walter Savitch has done seminal work in computational complexity theory and has made many important contributions. For all his wonderful work, he is undoubtedly best known for his beautiful result on the relationship between deterministic space and non-deterministic space. Probably everyone knows his famous theorem:
Theorem: For any ,
There are countless descriptions of the proof of his famous theorem: in books, on the net, and on blogs, So, if somehow you do not know the proof check one of them out. My goal today is to talk about a missed opportunity, seeing the opportunity, and about the future.
My understanding of how Savitch came to prove his theorem is based on talking with the main players over the years, and looking carefully at the literature. I could have some details wrong, but I think the story is too good to not be true.
Savitch was a graduate student working for Steve Cook, when he proved his theorem; actually the theorem was his thesis. Where did the idea for his theorem come from? Did he create it out of thin air? Looking back, the argument is so simple, so elegant, so clearly from “the book” that it seems hard to believe that it was not always known. But it was not known before 1970 when he proved it.
He did not create it out of thin air. In the 1960’s one of the main focuses of people working on, what we call complexity theory today, was language theory. This was the study of regular languages, context-free languages, and context-sensitive languages. These correspond, of course, to finite automata, to non-deterministic pushdown automata, and to linear bounded non-deterministic machines. The rationale for this interest was driven by Noam Chomsky and Marcel-Paul Schützenberger, especially Chomsky who was interested in a theory of “natural langauges”. At the same time programming languages, such Algol 60, were beginning to appear that needed a theory to support compiler construction.
Language theory was an important source of questions for theorists. One the so called LBA problem, I have already discussed in an earlier post was raised in the 1960’s and took almost thirty years to get solved. Many other problems from language theory played an important role in shaping theory’s direction at the time.
In 1965 at IFIP Congress in New York City an important paper was presented: Memory bounds for recognition of context-free and context-sensitive languages by Philip Lewis, Juris Hartmanis and Richard Stearns (LHS).
IFIP stands for “International Federation for Information Processing” and once was one of the top international conferences. Today there are so many other international conferences that it is probably fair to say that it is less important than it once was. In my post on Karp I pointed out that I first met Dick at the 1974 IFIP Congress, so for me IFIP will always bring back good memories. When, Lewis, Hartmanis, and Stearns presented their paper, IFIP was one of the top conferences.
They sketched the proof of a number of theorems in their paper, but the one that we are concerned with was the following:
Theorem: If is a context-free language, then
This is a hard theorem–I knew the proof once, but could not explain it today with looking it up. The “miracle” is that they must show how to simulate a pushdown that can hold as many as symbols, with only space. This is not easy.
Cook Makes a Phone Call
Savitch realized that the LHS paper had a fundamental idea, an idea that was very clever, yet an idea that even the authors did not completely understand. In proving that every context-free language was accepted by only , LHS had used a clever method for keeping track of the pushdown states. Essentially, Savitch saw that when separated from context-free languages, what they were doing was exactly what he need to prove his theorem. Exactly. The paper LHS had the key idea, but by applying it to the less interesting question of space complexity for context-free languages they missed a huge opportunity. They came very close in 1965 to proving Savitch’s theorem. We should not feel too bad: two of them, Hartmanis and Stearns, went on to win the Turing award for other work. But they were close.
Apparently, once Savitch and his advisor Cook realized that they could use the method of the LHS paper–not the theorem itself–they phoned Hartmanis. They told Juris that they thought his 1965 result was terrific–Juris later told me that he was always happy to hear that someone liked one of his papers. Who does not? They then asked some technical questions, and then rang off. In a short while Hartmanis got to see their paper. He then realized what they were doing, and realized the great result that he almost got. Oh well. We cannot get them all.
In the 1972 volume 3 issue of the Journal of Symbolic Logic there is a telling review on the LHS paper by Walter Savitch. In the same issue Alonzo Church also has a review of another paper–I point this out to show how long ago this was. Here is the review: (It is available via JSTOR if your library has access.)
This paper summarizes the known results on the tape complexity of context-free and context-sensitive languages. It also presents a number of new results in the area. The notions of tape complexity used are those introduced in the article reviewed above. The principal new results deal with context-free languages. A number of specific context-free languages are presented and their exact location in the tape complexity hierarchy is determined. It is shown that all context-free languages are -tape recognizable. The proof of the -tape bound is quite intricate, and this article gives only a sketch of the proof. The proof is, therefore, hard to read; however, the techniques are interesting and useful. The reader with perseverance will be rewarded for his effort. Walter Savitch
I love the end of the review: The proof is, therefore, hard to read; however, the techniques are interesting and useful. The reader with perseverance will be rewarded for his effort.
Savitch certainly was rewarded for his effort, a thesis, and a great theorem.
There are two points to make. First, sometimes the proof methods of a theorem are more important than the theorem. That means that we must read the proof itself to see if the method of proof can be used elsewhere. Many times the method of proof is standard and the actual theorem is the key advance. However, as Savitch’s theorem shows, sometimes the method is what we must understand if we are to make further progress.
Second, one of the biggest embarrassments of complexity theory, in my view, is the fact that Savitch’s theorem has not been improved in almost years. Nor has anyone proved that it is “tight”. This is one of the great open questions of complexity theory.
I have though how to improve his theorem. I find it hard to believe that it is the best result possible. But I have no good ideas to suggest, I have no approach, I am lost.