Meta-meta-STOC/FOCS comments.
My responses to Bill Gasarch's questions in response to reader comments about STOC and FOCS (and theory conferences more generally):
Is the community really driven by these conferences?
That depends on how you define “the community”, doesn't it. If you mean the people that regularly attend and publish in and review for FOCS and STOC, then no, they drive the conference, not the other way around. If you mean the slightly larger community of people who want to reguarly publish in STOC and FOCS—either out of real interest or pressure from hiring/tenure committees—then yes, somewhat. If you mean the even larger community of theoretical computer scientists, then no, not at all. The community members who don't fit into the STOC/FOCS subcommunity have their own venues, their own peers, their own values.
Is it bad that we are so judged?
I don't understand the question. How are we so judged?
Other fields do not have high-prestige conferences—why do we and is it a good thing?
It's neither “good” nor “bad”. It is what it is—a generally healthy but imperfect social agreement that evolved with the research community. Of course different communities have different avenues for communication.
Are the papers [at STOC and FOCS] mostly good?
Yes. Mostly. Except for the ones that aren't.
Is there a Name-School-bias? Is their a Name-person-bias? Some have suggested anonymous submissions to cure this problem.
Don't confuse correlation with causality. STOC and FOCS have a high-quality and cultural-conformity bias. People who consistently produce high quuality results that conform to community standards become Big Names. Schools that hire Big Names become Big Names themselves, so Big Name schools tend to hire and propmote Big Name people. The Big Names are the result of the quality bias, not the other way around.
In any research community, people are more willing to believe results from Big Names without reading their papers in detail than from random n00bs, sometimes unfairly so. I've seen less of this bias in the STOC/FOCS community than in other communities.
There is no problem here.
Is there an area-bias? There are several questions here: (1) is the list-of-topics on the conference annoucement leaving off important parts of theory? (2) is the committee even obeying the list as is? (3) have some areas just stopped submitting?
Who cares, no, and yes. STOC/FOCS accurately reflects a large and important subcommunity of theoretical computer scientists, but it does not cover everything, nor does it really pretend to. Most important results in computational geometry, data structures, machine learning, computational biology, logic and automata theory, and structural complexity theory are published in their own conferences (SOCG, SODA, COLT, RECOMB, LICS, Complexity) instead of STOC/FOCS, because those subfields have evolved into mature communities with their own vocabulary and culture, which most theoretical computer scientists don't understand or care about.
To be specific: Computational geometers don't submit many results to STOC/FOCS because almost nobody at STOC and FOCS goes to the computational geometry talks on purpose. STOC/FOCS attendees can be tricked into going to computational geometry talks if they don't realize that's what they are—for example, Arora's TSP approximation, Spielman and Teng's smoothed analysis, Kelner's surface circle packing—but mention Voronoi diagrams or upper envelopes or surface reconstruction and people will stay away in droves. If nobody is going to come to my talk, why should I bother submitting my work?
This is natural. Communities are defined by their mutual interests. Given limited travel budgets, nobody should be surprised that people in maturing subcommunities gravitate to their own conferences, develop their own vocabularies, and attract people who have no interest in “core” theory problems at all.
What makes the situation unhealthy is the continued belief in the STOC/FOCS community that STOC and FOCS represent all of theoretical computer science, or even the best of theoretical computer science. They don't. They represent the best of what the STOC/FOCS community has to offer, as judged by the STOC/FOCS community, but nothing more.
Is there a Hot-area-bias?
Probably. Is that bad?
Is there a mafia that controls which topics gets in?
Don't be stupid. Do you really think that anyone could control the opinions of an ever-changing programming committee? Okay, in principle, the steering committee might decide who gets to be PC chair based on their loyalty to traditional theory, and then the PC chair could select the other committee members by their reputation for loyalty to Truth, Justice, and Hardness of Approximation. The program committee could then receive its marching orders from Avi Widgerson, wringing his hands and cackling his evil cackle, deep undeground in the Kurt Gödel Memorial Orthodox Theory Lair below Einstein Drive. And then the faithful Widgersonian minions would do the bidding our their One True Enlightened Master, lest they face banishment to the intellectual hinterlands of (gasp) databases. (Dunh dunh duuuuuh.) “That's a nice thesis you have there,” Wiggy would say to each new theory PhD. “It'd be a real shame if something should happen to it. A real shame.”
Right. Have you ever tried to get fifteen academics to agree on where to eat lunch? Please.
Is there a bias towards people who can sell themselves better? To people that can write well?
I certainly hope so!
Is there a bias towards making progress on old problems rather than starting work on new problems?
If anything, I think there's a bias the other way.
Is there a bias towards novel or hard techniques?
Yes and yes, one good and one bad. The bias toward novel techniques is good; the results don't matter as much as the techniques that yield them. The comunity's preference for hard techniques, however, is misplaced; simple proofs have more impact in the long run.
Is it just random?
Of course it's "just" random! It's far from uniform, or identically distributed, or even independent, but yes, it's definitely random. Is anyone seriously suggesting that it could be anything else? (See “There is No Cabal” above.)
Are there many very good papers that do not get in? It has been suggested that we go to double sessions so that more get in. If the quality of papers has been going up over time this might make sense and would not dilute quality.
Ah, yes, the apologist's argument for grade inflation—students are better now than they used to be, so they deserve higher grades. (Also, they pay more tuition, and we need those alumni donations more.) This is usually uttered just one short breath before the ancient lament on the declining literacy, numeracy, moral fortitude, and musical taste of Kids These Days™, with the hippin and the hoppin and the bippin and the boppin.
It may very well be true that today's papers are “better” than papers published at FOCS in the 1970s, but only because the field has matured, thanks to a few giants with broad shoulders. It's not like “quality” is an absolute thing. The properties that make up a good paper are decided by the community, and the community evolves over time (or stagnates). The standards that make up good papers likewise evolve.
Yes, there are a few excellent papers that don't get in. That's to be expected; committee members are human; they disagree; they make mistakes. But much more often, there's a huge gray area of pretty good papers, some of which get in and others not, depending on the personal preferences of the committee members, the topics of other papers, the phase of the moon, the brand of coffee served at the committee meeting, and whether said coffee has already run out when a given paper is discussed.
And whadya mean "go to double sessions"? FOCS has had double sessions for years.
Is 10 pages too short for submissions? This was part of Vijay's Video Suggestion. Are figures and diagrams counted for those 10 pages? If they are they shouldn't be.
Nonsense. Of course they should be! Page limits should apply to all content, and figures and diagrams (if they're worth including at all) are definitely content. And yes, 10 pages is enough. If it really takes more than 10 pages to get the main ideas across, it's not a good conference paper and it won't make a good conference talk. Put it on the ArXiv and send it straight to a journal.
I actually like Vijay's suggestion, but not because I think theory results should be judged by presentations. The only purpose of a presentation is to inspire people to read the paper. Important details should be omitted from talks. No, I like Vijay's suggestions because it would force people to think about how they present their work when they are forced to omit all the details. Would this bias the acceptance process toward people who give good talks? Of course. Isn't that what conferences are for?
Are many submissions written at the last minute and hence badly written?
Yes. Also, many papers are written badly weeks in advance. A miniscule number of papers are written well at the last minute, too. Just not yours (or mine).
Are many submissions written by the authors taking whatever they have by the deadline and shoving it into a paper?
Yes. And sometimes these are the best papers at the conference. And sometimes they're crap.
Since the conference is about all of theory, can any committee do a good job?
Bad framing. The conference isn't about all of theory. The theoretical computer science community is too big and too diverse for a single conference. If it were, the current committees would be doing a horrible job, since they don't represent all of theory. But in fact, the committees are doing exactly what they're supposed to be doing—representing the interests of the STOC/FOCS community.
Do other conferences have these problems?
Yes. At least, to the extent that these are “problems”. Yes, other theory conferences have a bias toward quality that is misinterpreted as bias toward Big Names and/or Mafia control. (When in Tel Aviv, be sure to visit the Paul Erdös Memorial Computational Geometry Mafia Bunker And Falafel Stand. Say hi to Mickey.) Yes, other theory conferences get a lot of badly written submissions. Yes, attendees at other theory conferences tend to think they're about all research in their nominal field (comptuational geometry, say) when in fact the conference covers only a small fraction of that field.
Do you actually get that much out of the talks? If not then it is still valuable to to go for the people you meet in the hallways?
Sometimes (but not often) and yes, absolutely.
Recent Comments