« Lies, Damn Lies, and Soda | Main | More lies, some official »

June 21, 2005

Comments

It was observed here that FOCS seemed to be waning in influence. It is interesting to see that this coincides with a drop in the number of accepted papers.

Be as it may, all theory conferences are approaching 1/4 acceptance rate, and FOCS is approaching 1/5 acceptance rate. One has to wonder if this is healthy for anything except for people tenure case, and ego.

---It was observed here that FOCS seemed to be waning in influence. It is interesting to see that this coincides with a drop in the number of accepted papers.

I once submitted a paper to FOCS. It was rejected without a single comment. Fuck them. One day, I'll be a program officer, and if anyone ever was on the PC of FOCS, I'll reject their proposal outright!


--- I once submitted a paper to FOCS. It was rejected without a single comment. Fuck them. One day, I'll be a program officer, and if anyone ever was on the PC of FOCS, I'll reject their proposal outright!


In Networks conferences each referee report is over a page long, containing numerous suggestions about what needs to be done to make the rejected paper publishable.

Sadly in TCS this is not the common practice. One of the problems is the large reviewing volume given to each PC member. My first referee report, still as a grad student, went on for five pages of detailed corrections, suggestions and typos. Now as a professor, they are often less than half a page long.

Just as a point of comparison, if you look at http://www.cs.ucsb.edu/~almeroth/conf/stats/
you can see that the major networking conferences have similarly low acceptance rates (some are even lower!).

--- major networking conferences have similarly low acceptance rates (some are even lower!).

Looking at the acceptance rates is rather pointless, since issues such as self-selection play such a big role. The important parameters are: (a) are the decent papers getting in and (b) are numbers growing in proportion to the field?

Globecom and Infocom doubled from 2000 to 2004. On the other hand SIGCOMM has remained the same size since 1983, while acceptance ratio goes from 68% down to 8-9%.

It is now so difficult to publish in SIGCOM that a single paper in SIGCOMM can make or break your career.

FYI, according to the FOCS 2005 rejection email:

Since we decided not to have parallel sessions we were only able to accept 65 papers out of 263 submissions, an acceptance rate of 25%.

It is plausible that the program committee felt that the next 10-20 submissions are not in par with the top accepted submissions, and I can definitely sympathize with this feeling. With the current funding shortage, one possible side effect is poor attendance (like in the recent Complexity 2005). The relative vicinity to the NY area might attract more people than expected, but on the other hand Pittsburgh might not be that much of a tourist attraction.

Ken Clarkson

A common question, but not yet raised here: what's the recycle rate? Perhaps the fields aren't growing quite so much as implied, but rather authors are just getting more aggressive about re-submitting the same paper. There's no record of this that I ever heard of, and it wouldn't be entirely meaningful anyway, since papers can improve and grow from one submission to the next. But many papers move down the "food chain", surely. An odd few are submitted over and over and over, earning a certain reputation for their authors. Maybe authors at a given conference could be confidentially polled regarding their papers' histories?

--- what's the recycle rate?

This seems an unlikely explanation. As you point out, papers move down the food chain, yet there is an observed increase in paper submission at the *top* level SODA/STOC/FOCS.


--- Once again, despite a near-record number of submissions, the FOCS PC chose to accept a near-record-low number of papers, apparently because parallel sessions are evil. Evil! EEEEVVIIILLLL!

Locally in the early nineties most of the local theory colleagues could be counted on going to STOC and FOCS. The Theory lab was pretty much deserted during those days. Nowadays only those who have a paper in there go and the rest stay here. Is this typical? I do not know.

Ken Clarkson

----This seems an unlikely explanation. As you point out, papers move down the food chain, yet there is an observed increase in paper submission at the *top* level SODA/STOC/FOCS.

Ah, yeah, quite true, although there could be recycling across "equal" conferences, and at one time SODA was a bit less equal than the others. For example, the SODA submission deadline follows FOCS notification fairly conveniently.

JeffE

“Locally in the early nineties most of the local theory colleagues could be counted on going to STOC and FOCS. The Theory lab was pretty much deserted during those days. Nowadays only those who have a paper in there go and the rest stay here. Is this typical?”

Yes, this is typical. It's become significantly harder to get grant money to pay for travel. So people have to choose which conferences to attend (and where to submit papers) much more carefully. This could account for the drop in STOC/FOCS submissions in the mid-1990s, as well as the current scarcity of subfields like computational geometry or data structures at STOC/FOCS—people chose to send their stuff to more specialized conferences (like SOCG and SODA) instead.

"I once submitted a paper to FOCS. It was rejected without a single comment."

This is surprising. Is that practice common among FOCS-rejected papers? Isn't each paper reviewed by people outside the PC anyway, who would then email comments to the PC which could easily be sent to the authors?

I don't think it's the PC's responsibility to coach the authors. But in the name of transparency it seems essential practice to justify every rejection with at least a short phrase or two saying what was worst about the paper ("results not significant enough", "impenetrable writing, hard to verify results", etc.). Even if each PC member has high volume, this cannot possibly be such a huge burden. Even if the reviewer takes only 15 minutes on average to form an opinion of each paper (an underestimate, I hope), summarizing it in a sentence or two can't take more than 3 minutes, thus increasing review time by just 20%.

--- Is that practice common among FOCS-rejected papers?

Actually, after participating in many PCs, I've come to realize that more often that not borderline papers tend to have the least comments. Very good papers are heavily defended, bad papers are thoroughly criticized. It is the ones where the PC says "ok if there is enough space" or "should be accepted if we go to parallel sessions" that have no comments (note that this is just a general rule, there are plenty of exceptions).

Also in most conferences there are several natural cut off points. E.g. those with effusive acceptances only, those with two effusive acceptances and one ok acceptance, those with one effusive and two ok, and so on and so forth. A conference can choose where to draw the line at any one of these and still have only papers that are of unanimously recognized as of high quality.

--- Once again, despite a near-record number of submissions, the FOCS PC chose to accept a near-record-low number of papers, apparently because parallel sessions are evil. Evil! EEEEVVIIILLLL!

- There are people that feel very strongly about this, and if you think that having STOC and FOCS with different format and acceptance rates is wrong [despite the fact that they had virtually identical submissions number], then you should raise the topic in the FOCS buisness meeting.

- Unrelated, is the topic of merge or weak merge. I think there should be similar policy between STOC and FOCS, and it should be that only weak merge are allowed (my opinion, of course).

- Finally, if your paper got rejected from FOCS 05, you will get comments in a week or so. Patience...

--S

JeffE

What's a "weak merge"?

Piotr

"Weak merge" is when both papers appear separately in the proceedings, but only one talk is given. This recognizes that it is the conference time, not the amount of paper in proceedings, which is the critical resource.

--- There are people that feel very strongly about [parallel sessions == evil]

As a rule of thumb a subset of those who personally benefit the most from this state of affairs are the main defenders of this sad state of affairs. See the minutes of the SoCG meetings for a reference point...

As it was pointed before these people sacrifice the health of the field for the ego trip of publishing in an ultra exclusive conference or to avoid the mental effort of selecting talks among parallel sessions.

The comments to this entry are closed.