Uriel Feige answers some frequently asked questions to STOC submitters, but they apply almost verbatim to any theory conference.
Q: Who wrote the reviews?
A: Individual PC members and external reviewers. The reviews do NOT represent the collective opinion of the program committee, and need not include the reasons for accepting or rejecting the given submission.Q: What other information was available to the program committee in making accept/reject decisions?
A: The submitted papers themselves. Other written information includes numerical scores and confidence levels (at least three per paper), and comments to the PC only. In addition, there were oral discussions during the PC meetings.Q: Some of the reviews forwarded to me say wrong things about the paper.
A: We know and ignored these reviews.Q: All of the reviews forwarded to me are by reviewers who did not understand the paper.
A: Try to write your paper more clearly. Alternatively, it may be the case that STOC attendees (as reflected by the composition of the STOC committee) are not the right audience for your paper.Q: All reviewers liked my paper. How come it was not accepted?
A: Given many good papers, some difficult choices had to be made.Q: No reviewer liked my paper. How come it was accepted?
A: Probably some committee members are asking the same question. But note again that the reviews forwarded to the authors do not reflect the full discussion. Several committee members must have liked the paper.Q: I think that the program committee made a wrong decision in my case. How can I get more information about the reasons for this decision?
A: The only information released to the authors is the information that you already received: the accept/reject decision and those reviews of reviewers who explicitly allowed the review to be forwarded to the authors. Any additional information is confidential.Q: What is the procedure for appealing?
A: There is no such procedure. The decisions of the program committee about rejected papers are final. The decisions about accepted papers are also final, unless subsequently a major error is discovered in the paper, in which case the authors are expected to withdraw their paper.
I never buy this "hard decisions" bit. The PC is usually sick & tired by the end of the discussion and would not have accepted more papers even if it was possible.
If your paper got rejected than your paper was not liked by the PC. The PC might be wrong, but it is probably a very good indication that you should go back and work on it. It is quite OK to be really upset about the PC decision and fume about it (especially when it is clearly wrong), but one should use this energy constructively by improving the paper, and not just sulking.
Posted by: Fake | February 20, 2007 at 11:18 AM
Talking about reviews: any opinions about including the reviewer scores in the author feedback ? Personally, I think this would be a great way to communicate to the author(s) *quantitatively* what the reviewers thought about the paper, especially in case of rejection (e.g., does it make any sense to submit it to the same conference, or are the scores so low that it probably does not make sense; or, was the paper controversial, or were the scores more or less uniform, etc). This would also further increase the transparency of the system (which has already been greatly enhanced by comments such as the ones above).
Most conferences outside of theory (at least as far as I know), as well as the NSF panel system, do return the scores back to the authors.
Any opinions ?
Posted by: Piotr | February 20, 2007 at 06:31 PM
Talking about reviews: any opinions about including the reviewer scores in the author feedback ? Personally, I think this would be a great way to communicate to the author(s) *quantitatively* what the reviewers thought about the paper, especially in case of rejection (e.g., does it make any sense to submit it to the same conference, or are the scores so low that it probably does not make sense; or, was the paper controversial, or were the scores more or less uniform, etc). This would also further increase the transparency of the system (which has already been greatly enhanced by comments such as the ones above).
Most conferences outside of theory (at least as far as I know), as well as the NSF panel system, do return the scores back to the authors.
Any opinions ?
Posted by: Piotr | February 20, 2007 at 06:32 PM
Quote from above:
> Q: Some of the reviews forwarded to me say
> wrong things about the paper.
> A: We know and ignored these reviews.
Why would I believe this if the PC doesn't actually say so?
Posted by: Herman | February 21, 2007 at 06:44 PM
You wouldn't. Because in many cases, the PC doesn't ignore the wrong things the reviewers say about your paper. I know of some...
Posted by: | February 21, 2007 at 09:46 PM
To clarify the above a little: recently I have usually been ok with the reviews I got: even when some reviewers failed to recognize that my contribution was non-trivial and worthwile, there was usually also at least one review that I found fair. But I am rather unhappy with the standard disclaimers that program committees use when they forward reviews to authors. In my opinion, by forwarding a review a program committee gives at least the impression that it took the review seriously, and if it forwards a lousy review, this sheds doubt on the quality of the decisions. If it is actually true that program committees ignore some reviews, then how does the review end up being sent to the authors of the paper?
Posted by: Herman | February 22, 2007 at 10:25 AM
For those unhappy with reviews do please check out:
http://www.math.rutgers.edu/~zeilberg/Opinion77.html
Posted by: | February 22, 2007 at 03:23 PM
The basic idea is that even a FALSE review has useful information to the authros - it tells them where their presentation is bad enough that the reviewer got really confused.
Posted by: fogi | March 03, 2007 at 01:50 PM