Some fun rejection comments
Joshua Gans and George Shepherd were doctoral students in economics at Stanford University back in the 1990s. They wrote an interesting paper that I just came across titled, “How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists.” It grew into a 1995 book edited by Shepherd: Rejected: Leading Economists Ponder the Publication Process.
Today I want to discuss the same issue in our area of theory.
Ken and I have not had a chance to do a formal survey of papers that were rejected in our area. We also would not do exactly the same as Gans and Shepherd since it’s not what happens to the “mighty” that matters most but rather to the great band of those doing productive and creative and sporadically uneven work. Our point is rather that all of us who write articles for conferences and journals are subject to sporadically uneven reviews.
So we will today just offer a few things from personal experience to season the grill. We are mostly interested in negative comments from bad reviews. We could also touch on the opposite, heroic reviews that found subtle mistakes—or maybe mistakes missed by everyone including the referees.
I once got the following comment back from a top theory conference in the rejection e-mail:
The authors assume incorrectly that the graph has an even number of vertices in Lemma
The graph in question was a cubic graph. By what is sometimes called the First Theorem of graph theory, all cubic graphs have this property. Just double-count edge contributions and one gets that
where is the number of vertices and the number of edges. I assume the referee was overwhelmed with work, but
I once submitted a short paper, joint with Andrea LaPaugh and Jon Sandberg, to the Hawaii International Conference on System Sciences and it was accepted. Well, sort-of accepted. The head of the conference asked me to make the paper “longer.” I asked back:
“What was missing? Was the problem not motivated? Was the proof unclear?” And so on.
The head simply replied: “we like longer papers.” I pushed and said I thought making a paper longer for no reason seemed wrong. He responded that the paper was now unaccepted.
I could not believe it. We quickly sent it off to an IEEE journal. Don Knuth handled it, and it soon was accepted with minor changes only. By the way the paper solved a simple question: what is the best way to store a triangular array in memory?
At the presentation of the Knuth Prize to Leonid Levin the following story was told about reviews:
Leonid once submitted a paper to a journal and got back a negative review: It said that the paper was too short and also terms were used before they were defined. Leonid responded by taking two identical copies of his paper, stapling them together, and resubmitting the “new” paper. It was now twice as long, which answered the first issue, and clearly all terms were defined before they were used.
It is unclear what happened to the paper.
Then there is the folklore rejection letter:
What is correct in your paper is known, and what is new is wrong.
I hope to never get this one.
We’d love to hear from you with your own examples of strange reviews.
I Get Rejection
I submitted the above post last week to my blog editor but didn’t hear back—I assumed he was overwhelmed with work. Then he replied and asked me to make the post “longer.” I asked back:
“What was missing? Was the issue not motivated? Was the evidence unclear?” And so on.
The editor simply replied, “it’s a bit thin.” I pushed and said I thought making a post longer for no reason seemed wrong. This editor at least gave some concrete suggestions:
Use something from the featured paper.
Gans and Shepherd give one interesting kind of example where the delay caused by rejection enabled others with similar ideas to get ahead—not on purpose by the rejecter but just-so. They also give some self-revealing quotes including this one by the economist Paul Krugman:
The self-serving answer [to the “why me?” question] is that my stuff is so incredibly innovative that people don’t get the point. More likely, I somehow rub referees and editors the wrong way, maybe by claiming more originality than I really have. Whatever the cause, I still open return letters from journals with fear and trembling, and more often than not get bad news. I am having a terrible time with my current work on economic geography: referees tell me that it’s obvious, it’s wrong, and anyway they said it years ago.
Use others’ personal examples or famous ones.
Ken recently heard a true giant admit he gets rejections, “often because the referees don’t believe this work is really new.” Perhaps Krugman’s last clause means the same?
A famous case in our field was the number of times the first interactive proofs paper by Shafi Goldwasser and Silvio Micali was rejected from FOCS and STOC before finally appearing at STOC 1985 with Charles Rackoff as third author. Ken recalls people excitedly telling him and everyone about the work at the FCT conference in Sweden in August 1983. This could have become an example like in the Gans-Shepherd paper of others pipping ahead, but happily didn’t.
Try to source the quotation at the end.
A version of it was used by Christopher Chabris and Joshua Hart at the end of their negative review in the New York Times last month of the book The Triple Package:
Our conclusion is expressed by the saying, “What is new is not correct, and what is correct is not new.”
In March, Ken took part in an online discussion with Chabris about sourcing it. Ken recalled hearing it in the early 1980s in this snarkier form:
“This paper has content that is novel and correct. However, the parts that are novel are not correct, and the parts that are correct are not novel.”
It was already then a widely-known math cliché. Someone else in the discussion sourced it to the slamming of John Keynes’ book, The General Theory of Employment, Interest, and Money, by Henry Hazlitt in the introduction to his 1960 book, The Critics of Keynsian Economics:
In spite of the incredible reputation of the book, I could not find in it a single important doctrine that was both true and original. What is original in the book is not true, and what is true is not original. In fact, even most of the major errors in the book are not original, but can be found in a score of previous writers.
We wonder if any of our readers can find an earlier source? When did “not true” become “not new”? In any event it was one tough review.
Was my referee right about lengthening the post?
[photo at top; format fixes]