On 11/03/2011 06:24 PM, Pars Mutaf wrote: > Hi all, > > If you like the following idea, have similar ones, suggest modifications, > helpful strategies to get it done please contact me ([email protected]). It seems to me that a good place to start would be something like this: http://myreview.sourceforge.net/
-Martin > A system providing the following: > > 1. I can browse others' scientific work (e.g. arxiv.org) > 2. I can ask questions, provide comments, get answers etc. > 3. My input is archived. > 4. I get comments to my work. If I don't there is a problem with my work > and I update it or see similar work. > > Using this system, if I provide good feedback, I can form a network for > myself without necessarily attending conferences. > > === > Pars Mutaf > > > > On Thu, Nov 3, 2011 at 6:11 PM, Pars Mutaf<[email protected]> wrote: > >> Hi Joe thanks. I think I cannot argue with your experience of course which >> I don' have. >> >> But why the following system is not useful to me? >> >> 1. I can browse others' work (e.g. arxiv) >> 2. I can ask questions, provide comments, get answers etc. >> 3. My input is archived. >> 4. I get comments to my work. If I don't there is a problem with my work >> and I update it or see similar work. >> >> Using this system, if I provide good feedback, I can form a network for >> myself without necessarily attending conferences. >> >> Pars >> >> >> >> On Thu, Nov 3, 2011 at 5:44 PM, Joe Touch<[email protected]> wrote: >> >>> >>> On 11/3/2011 7:00 AM, Pars Mutaf wrote: >>> ... >>> >>> The reviews should be publicly available to everyone. >>> There have been attempts to explore this and other models, e.g., in no >>> particular order: >>> >>> A- author rebuttal of reviews >>> B- blind reviews >>> C- double-blind process >>> where the paper authors are hidden during review >>> D- public reviews >>> where reviews are published with the paper >>> E- open reviews >>> where the author sees the reviewer's names >>> F- adding a venue for papers on the 'borderline' of the >>> main conference >>> >>> Speaking as someone who has participated as a PC member in these in >>> various places (as an individual, not as TCCC Chair): >>> >>> A was tried at Infocom (and elsewhere). The goal was to avoid a paper >>> being discarded because of an incorrect review. The result was a >>> substantial increase in review time (actually, it ended up resulting in >>> less time for reviewers to complete their reviews due to a fixed yearly >>> cycle), but no substantial change in paper handling. Most of the rebuttals >>> did not point out review errors, but rather disagreed with review opinion. >>> >>> B is currently typical. >>> >>> C is used at Sigcomm and more recently at ICNP. It is intended to avoid >>> favoritism, but IMO it also tends to work against systems work that has >>> been vetted in workshops and symposia in parts. >>> >>> D has been tried for some CCR papers, where a single review or summary of >>> the reviews is presented. >>> >>> E was tried at Global Internet a number of years ago, and nearly killed >>> the meeting. Submissions went down over 50%. The result was much more >>> pleasantly-written reviews, but the reviews were (IMO) less useful. >>> >>> F was introduced at Infocom several years ago. IMO, it simply introduced >>> a second borderline, and made it very difficult to distinguish between full >>> accepts and "consolation prize" accepts. >>> >>> All of the above were introduced to address a perceived or real concern. >>> None of them was tested in a true experiment (e.g., with a control group >>> during the same year). Most of them (IMO) were introduced because chairs >>> believe that mechanism can address review process problems. IMO, there is >>> only one good solution for all such problems: >>> >>> PC chairs MUST review the reviews. EVERY review. EVERY year. >>> Reviews whose ranks are not substantiated by >>> meaningful comment must be both discarded and >>> replaced. >>> >>> Overall, IMO, it is useful to understand that: >>> >>> - reviewing is an imperfect process >>> >>> - a paper's quality is determined by what the reader >>> receives (goodput), not what is sent (offered load) ;-) >>> >>> - papers are rejected because of the lack of positive comments, >>> not for any single negative comment >>> (so arguing each negative comment in a review >>> won't fix a paper - many reviewers simply provide >>> sufficient negatives to justify a decision, but >>> could provide other negatives if asked) >>> >>> - at large conferences, papers are rejected after substantial >>> decision >>> e.g., at Infocom, a paper is either a unanimous reject >>> by three reviewers, OR is then considered by at least >>> an additional 8-10 people during the PC meeting >>> >>> I see none of these changing in an open process. >>> >>> Joe >>> >>> >>> > _______________________________________________ > IEEE Communications Society Tech. Committee on Computer Communications > (TCCC) - for discussions on computer networking and communication. > [email protected] > https://lists.cs.columbia.edu/cucslists/listinfo/tccc > _______________________________________________ IEEE Communications Society Tech. Committee on Computer Communications (TCCC) - for discussions on computer networking and communication. [email protected] https://lists.cs.columbia.edu/cucslists/listinfo/tccc
