Re: [TYPES/announce] Reviewing for POPL: a concrete proposal
[ The Types Forum (announcements only), http://lists.seas.upenn.edu/mailman/listinfo/types-announce ] Dear all First, I completely agree with Simon's proposal. Then, concerning how to accommodate more papers at the conference, my preference also goes to solution A (i.e., parallel sessions). cheers Alain If this proposal were to be accepted, we would need to figure out how to accommodate many more papers at the physical meeting. How to achieve this is secondary to my main proposal, but a number of proposals have been floated, including * Parallel sessions * A lottery among accepted papers * Voting by conference registrants * Program committee decision I know this is secondary, but I want to make sure I get my two cents in: the only rational choice is to go to parallel sessions or to extend the length of the conference. I believe that voting, either by PC or conference registrants, has the potential to be much more unfair than current paper selection practice. If part of the voting explicitly depends upon answering the question who will give a good talk? as opposed to what is the content of the paper then this introduces an extreme bias towards old, famous, successful researchers and away from young, new, unheard of researchers and students. Whereas we now at least try to judge POPL papers purely on the merit of the current technical document, we would instead be veering away from that crucial principle. And the more we start asking personality-based questions such as who will give a good talk, the more we may be susceptible to subconscious biases against various minorities (women, racial, etc) or the more we may try to overcompensate for such biases, resulting in reverse-discrimination. I also believe that lottery for talks is bad. What a lottery does is select some set of papers for which the talk audience is zero. With parallel sessions, the talk audiences will be smaller, but not zero. If I had a really great idea, I'd rather present it 6 months later at PLDI than have it appear 6 months earlier in the POPL proceedings, but not have the chance to give a talk. One last thing: while we may be getting all tied in knots over this popl review process right now, from what I've heard, within computer science, our community is really pretty great when it comes to selecting papers for inclusion in conferences based on their merits. I've heard of all kinds of dysfunctionality and biases and turf wars and sketchiness in other communities that we don't seem to be suffering from at all. Of course, that's probably because we're constantly working to try to make the process better and more fair to all. Cheers, Dave -- - Alain GIRAULT http://pop-art.inrialpes.fr/~girault INRIA senior researcher tel: +(33|0) 476 61 53 51 Head of the POP ART project-teamfax: +(33|0) 476 61 52 52 - Sauvons la Recherche ! http://www.sauvonslarecherche.fr -
Re: [TYPES/announce] Reviewing for POPL: a concrete proposal
[ The Types Forum (announcements only), http://lists.seas.upenn.edu/mailman/listinfo/types-announce ] The POPL event is actually a multi-event, with a lot of workshops around one main conference with top papers. One of the problems with refereeing is that first papers are sent to POPL and once rejected are resubmitted as a workshop paper. Notwithstanding the good work done by the POPL-PC's I know from my own experience that: - quite a number of papers submitted are probably not so excellent; but why not try since there always is a second chance at a more specialised event? - papers come form a large variety of subjects, and it is not always easy to find a sufficient number of informed PC members I can see a solution in which: - people initially/only submit to one of the more specialised conferences and workshops - the PC's of these conferences select the papers they think are of high quality and deserve to be presented to a wider community because they represent something new and interesting for everyone - the POPL PC constructs a nice single track conference out of these preselected papers, and there are no conferences scheduled in parallel when these papers are presented (e.g. in the morning) - the other events run in the afternoon and in parallel with the papers which were accepted by their PC's except those who made to the plenary POPL sessions. As a result we have more accepted papers, a better balanced program, automatic decisions about paralell session, and the committee has an easier job, since fewer and better papers have to be judged, and an initial review has already been done by the experts in the PC of the associated conferences. It also takes a bit of the gambling effect away. From my Haskell Symposium/ICFP experience I would rather have the HS PC select which are the best Haskell papers and send them one level up to the ICFP plenary forum, then to leave the selection to the ICFP PC. Given the broader scope of POPL I would expect this effect to be even stronger. Doaitse On 20 jan 2010, at 11:35, Alain Girault wrote: [ The Types Forum (announcements only), http://lists.seas.upenn.edu/mailman/listinfo/types-announce ] Dear all First, I completely agree with Simon's proposal. Then, concerning how to accommodate more papers at the conference, my preference also goes to solution A (i.e., parallel sessions). cheers Alain If this proposal were to be accepted, we would need to figure out how to accommodate many more papers at the physical meeting. How to achieve this is secondary to my main proposal, but a number of proposals have been floated, including * Parallel sessions * A lottery among accepted papers * Voting by conference registrants * Program committee decision I know this is secondary, but I want to make sure I get my two cents in: the only rational choice is to go to parallel sessions or to extend the length of the conference. I believe that voting, either by PC or conference registrants, has the potential to be much more unfair than current paper selection practice. If part of the voting explicitly depends upon answering the question who will give a good talk? as opposed to what is the content of the paper then this introduces an extreme bias towards old, famous, successful researchers and away from young, new, unheard of researchers and students. Whereas we now at least try to judge POPL papers purely on the merit of the current technical document, we would instead be veering away from that crucial principle. And the more we start asking personality-based questions such as who will give a good talk, the more we may be susceptible to subconscious biases against various minorities (women, racial, etc) or the more we may try to overcompensate for such biases, resulting in reverse-discrimination. I also believe that lottery for talks is bad. What a lottery does is select some set of papers for which the talk audience is zero. With parallel sessions, the talk audiences will be smaller, but not zero. If I had a really great idea, I'd rather present it 6 months later at PLDI than have it appear 6 months earlier in the POPL proceedings, but not have the chance to give a talk. One last thing: while we may be getting all tied in knots over this popl review process right now, from what I've heard, within computer science, our community is really pretty great when it comes to selecting papers for inclusion in conferences based on their merits. I've heard of all kinds of dysfunctionality and biases and turf wars and sketchiness in other communities that we don't seem to be suffering from at all. Of course, that's probably because we're constantly working to try to make the process better and more fair to all. Cheers, Dave -- - Alain GIRAULT
Re: [TYPES/announce] Reviewing for POPL: a concrete proposal
[ The Types Forum (announcements only), http://lists.seas.upenn.edu/mailman/listinfo/types-announce ] If this proposal were to be accepted, we would need to figure out how to accommodate many more papers at the physical meeting. How to achieve this is secondary to my main proposal, but a number of proposals have been floated, including * Parallel sessions * A lottery among accepted papers * Voting by conference registrants * Program committee decision I know this is secondary, but I want to make sure I get my two cents in: the only rational choice is to go to parallel sessions or to extend the length of the conference. I believe that voting, either by PC or conference registrants, has the potential to be much more unfair than current paper selection practice. If part of the voting explicitly depends upon answering the question who will give a good talk? as opposed to what is the content of the paper then this introduces an extreme bias towards old, famous, successful researchers and away from young, new, unheard of researchers and students. Whereas we now at least try to judge POPL papers purely on the merit of the current technical document, we would instead be veering away from that crucial principle. And the more we start asking personality-based questions such as who will give a good talk, the more we may be susceptible to subconscious biases against various minorities (women, racial, etc) or the more we may try to overcompensate for such biases, resulting in reverse-discrimination. I also believe that lottery for talks is bad. What a lottery does is select some set of papers for which the talk audience is zero. With parallel sessions, the talk audiences will be smaller, but not zero. If I had a really great idea, I'd rather present it 6 months later at PLDI than have it appear 6 months earlier in the POPL proceedings, but not have the chance to give a talk. One last thing: while we may be getting all tied in knots over this popl review process right now, from what I've heard, within computer science, our community is really pretty great when it comes to selecting papers for inclusion in conferences based on their merits. I've heard of all kinds of dysfunctionality and biases and turf wars and sketchiness in other communities that we don't seem to be suffering from at all. Of course, that's probably because we're constantly working to try to make the process better and more fair to all. Cheers, Dave
