Maybe it’s the case that for any particular SAT solver, you can construct a SAT instance that it will choke on…but for any distribution of SAT instances you can construct a particular SAT solver that will, on average, do well on them. Sort of the NP version of the halting problem.

-John

This question becomes better-posed and considerably easier to answer if we transpose it to the quantum domain: “The QIT/QIS theory community is sure that P{\neq}BQP, and so quantum simulation is hard (as Nielsen and Chuang’s textbook asserts, for example). The quantum simulation community is sure that they can solve most, if not all, of the problems that they get from real-world applications, and so simulating quantum systems is easy. What is going on here?”

What is going on in the quantum domain is that most real-world applications concern systems that are (1) measured-and-controlled, and/or (2) in contact with a thermal reservoirs, and/or (3) are near their ground-state of energy.

Now, in the quantum world our analysis is greatly simplified because all three of these conditions are (physically and mathematically) the same condition … because it is well-known that thermal reservoirs are informatically equivalent to (covert) measurement-and-control processes … and as for systems near the ground-state of energy … well heck, they arrive at low temperature by contact with a thermal reservoir. So we can ask Lipton’s question “What is going on” with reasonable hope that a single answer will cover a broad class of physical systems.

For computational purposes, a very convenient way to answer Lipton’s questions is to describe the system dynamics as a symplectic/stochastic process that is determined by equations of Lindblad type. A Choi-type gauge can always be found in which the drift of these equations has the powerful property of concentrating the trajectory flow onto low-dimension manifolds; and these manifolds turn out to have a tensor network structure.

Essentially all large-scale quantum simulation codes exploit this concentration property of the Lindblad-Choi flow … and that’s how they generally solve (very efficiently) quantum simulation problems that formally are in BQP.

Obviously, modern quantum simulation codes can’t simulate quantum computation processes (which are intolerant of measurement and noise), any more than modern SAT solvers can solve hard NP problems.

The advantage of studying quantum simulation as an example of a class of problems that is formally hard, but practically easy, is that the symplectic drift equations have a simple mathematical form that makes it feasible to prove rigorous concentration theorems … theorems that provide solid foundations for Lipton’s ideas in the context of quantum simulation.

I’ll be giving a tutorial on large-scale quantum spin simulations, based on this concentration-theoretic approach to efficient computational simulation, at next month’s Cornell/Kavli Conference: Molecular Imaging 2009: Routes to Three-Dimensional Imaging of Single Molecules.

Obviously we Kavli attendees have a very practical application in mind … atomic-resolution biomicroscopy … but this won’t preclude us from having fun at the Conference, exploring these concentration-theoretic mathematical ideas.

If we were to translate these quantum simulation ideas back into SAT-world, they might be something like the common-sense notion “it is usually true that mistaken inputs and wiring errors do not generate interesting output” … but it is much harder (for me) to see how this might be stated rigorously.

The bottom line is that Dick Lipton’s SAT-related ideas seem to be broadly applicable within the discipline of large-scale quantum simulation, where concentration theory provides a well-posed avenue for turning these ideas into theorems, and the prospect of transformational applications like molecular spin-imaging helps keep everyone excited and focused.

Could one then say that the observation that most realistic problems are easy is a reflection that the percent of hard instances per interval of complexity is slowly decreasing? or that it is flat vs increasing?

Is that kind of reasoning what you are thinking of as a better way to look at complexity? A statistics of toughness of solution over an ordered problem space?

Survey propagation uses the following “typical” properties of random 3-CNF formulas below the threshold: The bipartite clause-vertex graph
(1) has large girth and (2) behaves like a constant-degree expander. (Though one could take a large clause and convert it to 3CNF using auxiliary variables, that conversion does not change the behavior of the algorithm.) These properties are not “typical” in any reasonable sense considering the kinds of problems that people actually want to solve. In fact, many of the ways that interesting SAT formulas are generated in practice are via reduction techniques that yield a lot of highly specific gadgets that have variables of large degree and some variables that appear in very large numbers of clauses.

What if SAT is easy on average?

We certainly have that kind of property for some NP-hard problems such as Hamiltonian circuit where we can almost surely find them easily in random graphs at any edge density for which they exist. Until the Survey propagation algorithm came along, most people working on SAT (among them the statistical physicis community from which Survey propagation arose) would surely have reasoned that, together with problems like coloring and clique, the threshold instances under the uniform random distribution will be hard. It still is not out of the question for random k-CNF for sufficiently large k.

A particular case of (2) for a given (heuristic) solver A is, for any given length n, the string x_n = “the lex-first string of length n on which A achieves its worst-case running time.” Conditioned on n, that’s a constant-size description of x_n. Hence under Solomonoff-Levin distribution M, time on M-average is Theta-of worst-case running time [Li-Vitanyi], and likewise for other stats such as approximation performance in place of time. I was interested in whether low-KC strings would show such “malign” behavior in practice, but back in the early 1990s I didn’t appreciate the difference between (2) and (3), and the graph instances I and my first PhD student generated weren’t very deep—hence our mixed results.

I suppose if the interivewing committee ever felt that you consider SAT is hard, so certainly this implies you are not into their business at all.

This is similar to, but perhaps more exact than, Michael Mitzenmacher’s idea on entropy mentioned in the post.

If this were the case, then under a reasonable hardness assumption (which I’ll discuss below), the ease of solving all SAT instances arising in practice could be due to the possibility that the instances arising in practice have low computational depth. By computational depth, here, I mean the difference between the poly-time bounded Kolmogorov complexity and the usual Kolmogorov complexity of the string.

Antunes and Fortnow “Worst-Case Running Times for Average-Case Algorithms” show the following result: if EXP is not infinitely-often contained in subexponential space (the reasonable assumption mentioned above), then the following are equivalent for any algorithm A:

1. For any P-sampleable distribution M, A runs in polynomial time on M-average.

2. For all polynomials p, the worst-case running time of A is bounded by 2^{K^p(x) – K(x) + log|x|}, for all inputs x.

(The term K^p(x) – K(x) is exactly the computational depth mentioned above.)

Of course this is a bit of a digression: I think what is meant here by most is most of the instances arising in practice. The meaning of that indeed varies from application domain to application domain.

The property hidden in SAT – though it is well-known to all of us – is that SAT is a logical problem. How did SAT become NP-complete, only for purely logical reasons. Those logical reasons themselves can easily be applied in such a way to deprive SAT from its position as the quintessential language, the most important language in computational complexity theory.

Define the language L={w in Sigma* such that w = NOT w}, you can have L in NP which is a counter-example irreducible to SAT. Hence, SAT is (NOT) NP-complete. There can be many logical languages with the property of L, like the Kleene-Rosser paradox for example.

Not only this, but also SAT became NP-complete due to the properties of the hardware (physical/mechanical) of the Turing machine. So if you derive a satisfiable [SAT] formula from the Voltage-Current characterestics of the emitter-base-collector of the transistors in the CPU, would this imply any specific property of the languages that this CPU is processing?

If you have 3 vehicles A, B and C which none of them can be driven in 3 different directions simultaneously, you get:

[A_x or A_y or A_z] and [B_x or B_y or B_z] and [C_x or C_y or C_z]

Would the satisfiability of this formula imply any property to the symbols/strings/words/language one writes on his vehicle? If you assume this vehicle is Turing-empowered and can process languages.

The language processed by a Turing machine is aceepted/not accepted due to the meaning of its strings and not due to the physical/mechanical properties of the machine which was used to prove SAT is NP-complete.