Hi Joe thanks. I think I cannot argue with your experience of course which I don' have.
But why the following system is not useful to me? 1. I can browse others' work (e.g. arxiv) 2. I can ask questions, provide comments, get answers etc. 3. My input is archived. 4. I get comments to my work. If I don't there is a problem with my work and I update it or see similar work. Using this system, if I provide good feedback, I can form a network for myself without necessarily attending conferences. Pars On Thu, Nov 3, 2011 at 5:44 PM, Joe Touch <[email protected]> wrote: > > > On 11/3/2011 7:00 AM, Pars Mutaf wrote: > ... > > The reviews should be publicly available to everyone. >> > > There have been attempts to explore this and other models, e.g., in no > particular order: > > A- author rebuttal of reviews > B- blind reviews > C- double-blind process > where the paper authors are hidden during review > D- public reviews > where reviews are published with the paper > E- open reviews > where the author sees the reviewer's names > F- adding a venue for papers on the 'borderline' of the > main conference > > Speaking as someone who has participated as a PC member in these in > various places (as an individual, not as TCCC Chair): > > A was tried at Infocom (and elsewhere). The goal was to avoid a paper > being discarded because of an incorrect review. The result was a > substantial increase in review time (actually, it ended up resulting in > less time for reviewers to complete their reviews due to a fixed yearly > cycle), but no substantial change in paper handling. Most of the rebuttals > did not point out review errors, but rather disagreed with review opinion. > > B is currently typical. > > C is used at Sigcomm and more recently at ICNP. It is intended to avoid > favoritism, but IMO it also tends to work against systems work that has > been vetted in workshops and symposia in parts. > > D has been tried for some CCR papers, where a single review or summary of > the reviews is presented. > > E was tried at Global Internet a number of years ago, and nearly killed > the meeting. Submissions went down over 50%. The result was much more > pleasantly-written reviews, but the reviews were (IMO) less useful. > > F was introduced at Infocom several years ago. IMO, it simply introduced a > second borderline, and made it very difficult to distinguish between full > accepts and "consolation prize" accepts. > > All of the above were introduced to address a perceived or real concern. > None of them was tested in a true experiment (e.g., with a control group > during the same year). Most of them (IMO) were introduced because chairs > believe that mechanism can address review process problems. IMO, there is > only one good solution for all such problems: > > PC chairs MUST review the reviews. EVERY review. EVERY year. > Reviews whose ranks are not substantiated by > meaningful comment must be both discarded and > replaced. > > Overall, IMO, it is useful to understand that: > > - reviewing is an imperfect process > > - a paper's quality is determined by what the reader > receives (goodput), not what is sent (offered load) ;-) > > - papers are rejected because of the lack of positive comments, > not for any single negative comment > (so arguing each negative comment in a review > won't fix a paper - many reviewers simply provide > sufficient negatives to justify a decision, but > could provide other negatives if asked) > > - at large conferences, papers are rejected after substantial > decision > e.g., at Infocom, a paper is either a unanimous reject > by three reviewers, OR is then considered by at least > an additional 8-10 people during the PC meeting > > I see none of these changing in an open process. > > Joe > > > _______________________________________________ IEEE Communications Society Tech. Committee on Computer Communications (TCCC) - for discussions on computer networking and communication. [email protected] https://lists.cs.columbia.edu/cucslists/listinfo/tccc
