Natural, Natural, and Natural
A great recent talk on natural algorithms
Bernard Chazelle is one of the world experts in computational geometry, lower bounds on data structures, upper bounds on data structures, and discrepancy theory—among many other aspects of theory. His book on the discrepancy is a classic. If you need to know about placing points so they are “well distributed,” you should read his book.
Today I plan to discuss a recent presentation he gave at Georgia Tech, actually at the ARC center, on Natural Algorithms (NA).
I have had the pleasure of knowing Bernard since he was a graduate student at Yale, where his advisor was David Dobkin. After I got to Princeton in 1980, I quickly recruited David, who then quickly recruited Bernard. I cannot imagine Princeton without either of them.
I have one story I would like to relate about Bernard. It concerns the PhD. examination process we used at Princeton years ago. One year I was in charge of the graduate examination process. We gave the students written exams, and then got the faculty together to discuss each student’s case. Our pass/fail decision was based on the exam and other information—for example, input from their advisor.
The written part of the exam was given over three days: one day was on systems, one on theory, and one on applications. There was a question on the systems exam that most of the theory students failed. This led to a pretty heated discussion about whether or not the question was fair. It was my question, so I thought it was okay, but many did not. Curiously, the systems faculty liked the question, but argued it was a theory question.
Bernard liked the question, thought it was fair, but still agreed it was a systems question—one theory students might not need to understand. Finally, cooler heads prevailed and we made decisions about exam grades all the faculty could support.
After the meeting Bernard was still convinced my question was not something a theory student should know. So I asked him to come to my office, where I pulled out an old paper on Turing Machines, and pointed to a lemma—there was my question. He finally agreed the question was definitely a theory one. I always thought the question was just a basic fact about data structures. Here is the question, you judge:
Using two stacks, show you can simulate a queue so the cost of operations of the queue is bounded by .
What do you think? Theory or systems?
The word “natural” is a wonderful word, full of positive feelings—I think good thoughts when I hear it. This is why the word is used so often: there are natural resources, there are natural remedies, there is natural selection, there is natural history, there is the movie “The Natural,” and on and on. Not surprisingly, “natural” is also popular in theory and in mathematics. Here are a few uses of “natural” in our area of theory.
Natural Proof Systems Gerhard Gentzen created natural deduction in 1935. He created a set of inference rules for predicate calculus: rules with a simple clarity, rules with a certain naturalness. For example,
is a simple natural deduction. You should read as meaning: if the formulas in are true, the formula follows. The brilliance of Gentzen was creating a logic consisting only of natural rules that is still complete: this uses his famous cut elimination method.
I have never used category theory in my research, and only know the very basics. If you are not familiar with category theory look here. With apologies to the experts, I view category theory as a way to study functions rather than objects as the primary concern. This shift in focus seems simple, but has remarkable consequences. For example, category theory is indispensable in explaining why some maps are “natural” and others are not. Category theory can do much more, but this is certainly one of the elementary but very powerful uses of it. Saunders Mac Lane once said:
“I didn’t invent categories to study functors; I invented them to study natural transformations.”
An important example comes from the theory of vector spaces. Consider, to be concrete, finite dimension vector spaces over the reals. Every such space has a dual space of the same dimension: the dual is nothing more than all the linear functionals defined on the vector space. A linear functional is just a linear map from to .
Since any two such vector spaces are isomorphic if they have the same dimension, the following two statements are true for any :
- The space is isomorphic to .
- The space is isomorphic to .
Yet, intuitively these statements are different: in (1) the map seems to depends on the basis used to define the spaces; in (2) the map is canonical or natural. The latter follows since there is a “canonical map” from to : Define by,
where is in the vector space and is in the dual space . This map clearly does not depend on any basis of the vector spaces—it is natural. Category theory gives a precise way to define natural maps, and unravels why these two isomorphisms are different.
Natural Selection John Holland in the early 1970’s created a notion of natural selection for the creation of algorithms. I will have to discuss this in more detail another time.
Natural Algorithms This is an area started by Chazelle in the early 2000’s and was the topic of his talk. NA is an attempt to apply complexity theory to the study of natural phenomena like bird flocking.
Form of Bernard’s Talk
Bernard gave a beautiful talk on his recent work on NA. It was terrific on two levels: a great presentation and great results. The presentation was so special I thought I would discuss how he did it. It was unique.
His slides were very sparse. Some slides had a single phrase or a single picture or a single equation. Usually we see talks with slides full of bullets and multiple pieces of information. Bernard’s was the opposite—a minimalist approach to the use of powerpoint. It flowed easily from one idea to the next, and had a beautiful rhythm. I was very impressed with the form of his talk. If you can have him visit and give a talk, you will be very happy you did.
Content of the Talk
In Bernard’s talk were some quite striking results and some intriguing open problems. One of the most surprising was an “old” result he has proved before on the theory of how birds flock. This is an intriguing question, with no compelling answer. They do flock, but we do not understand why they do, or how.
There are formal mathematical models, however, that let us try to study flocking. One of the models Bernard studies is a variant of the model of Felipe Cucker and Steve Smale. All models try to capture the following ideas:
- Birds will try to avoid local crowding.
- Birds tend to fly towards the average direction of their neighbors.
- Birds tend to fly towards the average position of their neighbors.
What is surprising is Bernard proves an upper bound and a lower bound on how long it will take for the birds to achieve a steady state. The times are immense—both the upper and the lower bound.
- A group of birds reach steady state in at most time steps.
- There are initial configurations of birds requiring more than time steps before steady state.
The notation with the arrows is defined by: and for ,
The proofs and the results are beautiful, but they suggest something is missing in the model, since the bounds are immense. Birds do actually flock in reasonable amounts of time, so there must a more realistic model. His results still are amazing.
Birds flocking over Rome:
One open problem is to find a model of flocking, for instance, where the bounds do not need exponential notation. This should be possible. The current work is beautiful, but birds do flock together faster than Bernard’s lower bound. He and I spoke about this after his talk and he agrees it is one of the key open problems in the area.