Oblivious Turing Machines and a “Crock”
The Fischer-Pippenger Theorem, oblivious Turing Machines and a strange FOCS talk
Michael Fischer has made a huge impact on at least two quite separate areas of complexity theory. In the first part of his career he made important and seminal contributions to the basic understanding of Turing Machines (TM). This work is quite technically challenging and still stands as some of the deepest work done on the behavior of TM’s. While today we are more interested, often, in the behavior of general algorithms, the care and feeding of TM’s is important. In my opinion, there remain many nuggets to be mined regarding their behavior.
In the second part of his career, he switched to the area of distributed computing. He has made many important contributions to this area: both upper and lower bounds. Much of the work—especially in the beginning–was joint work with Nancy Lynch. It is interesting to note that Nancy started her career in recursion theory, before also switching to help create the theory of distributed systems. She was awarded the prestigious Donald Knuth Prize in 2007 for her work on distributed computing. I plan on talking more about that in the future.
Today I plan to talk about a wonderful theorem that is due to Mike Fischer and Nicholas Pippenger. Their result showed a tight relationship between time complexity of a TM and boolean complexity. The bound they proved three decades ago is still the best known. Amazing.
At the FOCS 1975, which was held that year at Berkeley, I had two great memories. One concerns discovering on checking into my room that I was sharing with Rich DeMillo, that there was another occupant sharing the room. There was a mouse racing around our room. A mouse. The meeting was being held in the old Claremont Hotel, and when we told the desk that we had cleverly trapped the mouse under a wastebasket, their response was, “yes, and what would you like us to do?” We, Rich and I, changed rooms—I believe the mouse stayed.
The other memory, from that FOCS, is one about Mike and the word “crock”. See this for the meaning if you are unsure what “crock” means, but I prefer not to directly explain it. That way we can keep our G rating for this blog.
Here is what happened. I was walking the hall when a talk entitled “Economy of Descriptions by Parsers, DPDA’s, and PDA’s” was about to start. I had no intention of going to this talk; it was great work no doubt, but it was not something I knew about, nor cared to know about. As I was trying to find a quiet place to think I ran into Professor Dr. Fischer. I was junior and he was senior. Mike told me that the next talk was on an important topic—one that he had worked on—and I should come hear it. So off we went and sat down in the middle of the hotel ballroom.
The talk was given by Matt Geller and his co-authors were: Harry Hunt, Thomas Szymanski, and Jeffrey Ullman. As soon as the talk started Mike proceeded to fall asleep. I figured that I could not leave in the middle of the talk, and since I had a blank pad and a pen, I started to try to solve some problem I was working on at the time. Talks can be a quite relaxing place to think, all the more true if you are not really listening to the speaker. I knew Matt well, but as I said the talk’s topic was not one I followed.
The talk ended and Matt proceeded to get the usual technical questions. He answered them in turn, and all seemed fine to the naïve observer—me. Somehow Matt then said something that hit a raw nerve with his co-authors. Ullman, recall he was one of the co-authors, raised his hand, and was called on by the session chair. Jeff said, “Matt did you just say X?” Matt immediately answered yes he had. With that Jeff said,
“That is a crock, and I want to completely disassociate myself from that.”
As soon as “crock” was uttered, Mike Fischer’s head popped up and he was fully awake. Is there some subconscious mechanism we have for certain words? Anyway, Mike quickly began to whisper to me what was going on? I started to try to explain the context the best I could while the session chair quickly pointed out that were on a coffee break.
The next thing that happened was unique. Matt was soon surrounded by a circle of experts and all his co-authors. They had a heated argument about was X true. The more Matt said the further, I noticed, that his co-authors moved away—the circle around him keep growing in size. I never did find out what the issue was.
By the way, Geller moved out of theory a few years later and announced that he was moving to Hollywood to become a screenwriter. He actually had some success with TV shows, and there on the screen from time to time was, screenplay by Matt Geller.
Let’s turn now to a wonderful theorem on TM’s.
Oblivious Turing Machines
A TM operates on an input of length . The TM can move its heads based on the values of the input. However, we call a TM oblivious (OTM) if the motions of the heads are independent of the input . Thus, the head motion can only depend on the length of the input, but not on the individual bits of the input.
Oblivious TM’s are important for several reasons. First, there is a strong connection between the time an OTM takes and the size of the boolean circuit that the machine computes. Second, there have been applications to cryptography over the years of OTM’s, the fact that the heads move the same way independent of the input bits is useful in certain information hiding schemes.
Suppose a TM accepts some language . In a natural way there is a boolean function , for each , so that for all ,
By the boolean complexity of we mean the size of the smallest general circuit that computes .
Theorem: Suppose that is a deterministic oblivious TM that runs in time . Then, the language it accepts has boolean complexity at most .
The famous Fischer-Pippenger Theorem is:
Theorem: Suppose that is a deterministic TM that runs in time . Then, there is a deterministic TM that accepts the same language as and is oblivious. More, it runs in time .
See this for a modern description of these results. Earlier in 1972 John Savage proved essentially the first theorem and the second with the weaker bound of in his paper “Computational work and time on finite machines.” John’s paper is a very important early paper that had many cool ideas in it. I think it does not get as much attention as it deserves because John tried to make connections between “real” complexity measures and more theoretical ones. Yet it is an early important piece of work.
I will not give a detailed proof of their theorem. The reference does a great job, but I do want to say something about how it works in general.
The central idea of simulating a TM and making the tapes oblivious is really a neat kind of amortized complexity argument. This was before the concept of amortized complexity was invented. Mike and Nick had to cleverly move the Turing heads in a fixed manner, but yet do an arbitrary computation.
I think the following may help explain what they needed to do, without getting into the details of Turing tapes. Consider the problem of sorting numbers
There are many algorithms for sorting numbers in comparisons. However, the data motion is quite adaptive, that is what is stored where and when depends on the value of the data.
At a cost of only an extra one can make the data motion independent of the data values. The trick is to use a sorting network, such as: Batcher odd-even mergesort, bitonic sort, or Shell sort. In a sorting network the basic primitive is swap: compare two data items and move the larger to the “bottom” position and the smaller to the “top” position. The order of which items to perform the swap operation on is fixed independent of the values of the items. Here is an example of a sorting network and a trace of how it works on a given input.
See this for details on sorting networks.
In a natural sense a sorting network is oblivious, since which data items are compared at each step are determined independent of the data values. Of course, it is possible to do even better with a more complex sorting network, but I will leave that for another day.
Improve the Fischer-Pippenger Theorem, or prove that it is optimal. A separate question is to improve the circuit simulation result. There is no reason that a TM that runs in time cannot always be computed by a linear size circuit. I have commented earlier that the famous Andrey Kolmogorov may have believed this. I have tried over the years to improve both these theorems with no success. I hope you are more clever and can see a new approach.