Re: [OSGeo-Discuss] FOSS4G presentation review process
Al 02/10/12 16:16, En/na Barry Rowlingson ha escrit: On Tue, Oct 2, 2012 at 3:03 PM, Adrian Custer acus...@gmail.com wrote: Just picking the 'good' talks may lead the conference to once again have many talks about the same projects that have come to dominate and fewer talks from new talent. Therefore picking talks only on the individual merits, whether of the abstract, topic, or speaker, leads to a particular kind of gathering, geared towards past success rather than towards fostering future diversity. Yes, this is why the community rating is only part of the process. The intention of it is, I think, to give the committee an idea of community feeling for the proposals. If the top 100 community-related proposals are all about PostGIS the committee should make some adjustments for balance, but would at least know what the top 10 rated PostGIS proposals were and have a PostGIS stream for them. At FOSSG2010 in Barcelona, considering that we had different sized rooms at the venue, we used the community rating as a valuable information to assign rooms to each presentation, according to its popularity. Again, as Barry pointed out, the community rating was considered as a part of the process of selecting presentations. Then the LOC assumed the final selection of presentations, trying to keep a logical balance between topics, presenters, etc... and its popularity. My 2cts Regards, Lluís Also, FOSS4G is not an Unconference... Barry ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss -- Lluís Vicens SIGTE - Universitat de Girona Pl. Ferrater Mora, 1 17071 Girona ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On Tue, Oct 2, 2012 at 1:01 AM, Bruce Bannerman b.banner...@bom.gov.au wrote: Agreed. Well said Cameron, with the aside that there may be an interesting talk from a previously little known person. I suggest leaving this to the discretion of the LOC and interested parties who subscribe to that year’s FOSS4G mailing list. A popularity campaign is not required or wanted. With my statistician hat on, and not speaking as a member of the committee, it seems that we have two processes going on - what sounds like a good talk, and who sounds like a good speaker. Maybe we should run two review systems - one with *just* names and not abstracts or titles, and the other with just abstracts and no names. That would give us a measure of who the community wanted to see at the conference, and what the community thought were great talks unbiased by the name. The committee would then take both these reports into consideration for the final selection. My extreme statistician hat gave me another idea. For each review, present a random speaker with a random talk abstract, and ask for a rating on the whole package. With enough randomized reviews, it would be possible to get a ranking for speakers and talks as well as a correlation between speakers and talks. Perhaps we could even suggest that if speaker A did talk C instead of talk B, more people would be interested! There may be ways to stop popularity-contest ballot stuffing - reviewers could get a random subset of the presentations for review, with no guarantee that their friend's proposal is going to be there - and prevent them reloading the page until it appears. Or you could present multiple random pairs of proposals and ask which of the two you'd attend. Committee hat back on, I'm glad we're having this discussion. Barry ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On 2/10/2012 5:05 PM, Barry Rowlingson wrote: With my statistician hat on, and not speaking as a member of the committee, it seems that we have two processes going on - what sounds like a good talk, and who sounds like a good speaker. Maybe we should run two review systems - one with*just* names and not abstracts or titles, and the other with just abstracts and no names. That would give us a measure of who the community wanted to see at the conference, and what the community thought were great talks unbiased by the name. The committee would then take both these reports into consideration for the final selection. If you are inviting the community to review, then I suggest Keep the review process Simple. With my simple maths hat on: Expect 150+ abstracts. Each abstract takes say 2 mins to read, think about, and provide a ranking. Total review time = 300 minutes = 6 hours. Best not to complicate the review process thus increasing review time. -- Cameron Shorter Geospatial Solutions Manager Tel: +61 (0)2 8570 5050 Mob: +61 (0)419 142 254 Think Globally, Fix Locally Geospatial Solutions enhanced with Open Standards and Open Source http://www.lisasoft.com ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On Tue, Oct 2, 2012 at 9:39 AM, Cameron Shorter cameron.shor...@gmail.com wrote: With my simple maths hat on: Expect 150+ abstracts. Each abstract takes say 2 mins to read, think about, and provide a ranking. Total review time = 300 minutes = 6 hours. Best not to complicate the review process thus increasing review time. Agreed - and if I was presented with a big list of 150 abstracts and 150 radio buttons from -2 to +2 I'd get to about 20 before giving up. However, a system that presented random pairs of abstracts or names and asked simply which would you like to see?, then took a this one/that one/dont know response (with big fat buttons to easily click), and then presented another pair would enable reviewers to 'dip in' and do a bit more reviewing at any time. Its the kitten war method: http://kittenwar.com/ Barry ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On 10/02/2012 11:24 AM, Barry Rowlingson wrote: On Tue, Oct 2, 2012 at 9:39 AM, Cameron Shorter cameron.shor...@gmail.com wrote: With my simple maths hat on: Expect 150+ abstracts. Each abstract takes say 2 mins to read, think about, and provide a ranking. Total review time = 300 minutes = 6 hours. Best not to complicate the review process thus increasing review time. Agreed - and if I was presented with a big list of 150 abstracts and 150 radio buttons from -2 to +2 I'd get to about 20 before giving up. However, a system that presented random pairs of abstracts or names and asked simply which would you like to see?, then took a this one/that one/dont know response (with big fat buttons to easily click), and then presented another pair would enable reviewers to 'dip in' and do a bit more reviewing at any time. Its the kitten war method: http://kittenwar.com/ IIRC, the abstracts were always in random order, so that even if everyone did a subset things should work out fine. Cheers, Volker ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
One thing I really disliked in the past is that the size of abstracts really differed and abstracts were sometimes really large. Please limit people to a small and to-the-point abstract, at least for the voting process. Best regards, Bart -- Bart van den Eijnden OSGIS - http://osgis.nl On Oct 2, 2012, at 1:30 PM, Volker Mische volker.mis...@gmail.com wrote: On 10/02/2012 11:24 AM, Barry Rowlingson wrote: On Tue, Oct 2, 2012 at 9:39 AM, Cameron Shorter cameron.shor...@gmail.com wrote: With my simple maths hat on: Expect 150+ abstracts. Each abstract takes say 2 mins to read, think about, and provide a ranking. Total review time = 300 minutes = 6 hours. Best not to complicate the review process thus increasing review time. Agreed - and if I was presented with a big list of 150 abstracts and 150 radio buttons from -2 to +2 I'd get to about 20 before giving up. However, a system that presented random pairs of abstracts or names and asked simply which would you like to see?, then took a this one/that one/dont know response (with big fat buttons to easily click), and then presented another pair would enable reviewers to 'dip in' and do a bit more reviewing at any time. Its the kitten war method: http://kittenwar.com/ IIRC, the abstracts were always in random order, so that even if everyone did a subset things should work out fine. Cheers, Volker ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
Hi all, I don't agree. I like the idea of having the community vote on the abstracts only and then the organising committee can make the call of adding some some big names to draw the expected attention to the conference. They may even use some abstract that wasn't voted that much, but the committee still thinks it's an good idea. The important thing is, that the *process* is transparent, the result doesn't need to be transparent and can solely be based on what the committee thinks is the best way to do. This includes e.g. to reduce the votes if there was obvious cheating. Cheers, Volker On 10/02/2012 02:01 AM, Bruce Bannerman wrote: Agreed. Well said Cameron, with the aside that there may be an interesting talk from a previously little known person. I suggest leaving this to the discretion of the LOC and interested parties who subscribe to that year’s FOSS4G mailing list. A popularity campaign is not required or wanted. Bruce On 2/10/12 9:36 AM, Cameron Shorter cameron.shor...@gmail.com wrote: I believe that for the general program, we should publish both the presenter and abstract. Reasons: 1. I'm attracted to a talk by both the topic and the presenter. I'm more likely to listen to a talk by someone who has a deep knowledge of a topic, which typically equates to someone with a big reputation. 2. And I think it is appropriate that people who have committed much time to the Open Source community, and hence have built up a big reputation, are allowed to be recognised by the selection community. 3. It also makes good business sense to the FOSS4G conference, as big names on the program will likely attract more delegates, and will likely have the delegates going away satisfied that they have seen presentations that they wanted to see. 4. The alternative of only seeing an abstract when voting is that anyone who can write a good abstract can potentially present on a topic, even if they don't have a deep insight in the topic of interest. On 2/10/2012 4:59 AM, Schlagel, Joel D IWR wrote: I believe anonymous reviews has a place as a component of paper selection - as a compliment to editorial review and professional judgement.FOSS4G conference is the number one marketing opportunity for the OSGEO community. We should make a deliberate effort to have a balance between inward focused technical / developer oriented presentations and outward focused policy / success / benefit type good news presentations. -joel From: discuss-boun...@lists.osgeo.org [discuss-boun...@lists.osgeo.org] on behalf of Paul Ramsey [pram...@opengeo.org] Sent: Monday, October 01, 2012 2:43 PM To: Volker Mische Cc: osgeo-discuss Subject: Re: [OSGeo-Discuss] FOSS4G presentation review process I'm in favour too. It has potential, let's see how an anonymous community process works in practice. P. On Mon, Oct 1, 2012 at 11:31 AM, Volker Mische volker.mis...@gmail.com wrote: Hi all, On 10/01/2012 06:10 PM, Barry Rowlingson wrote: In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On Tue, Oct 2, 2012 at 1:36 PM, Volker Mische volker.mis...@gmail.com wrote: Hi all, I don't agree. I like the idea of having the community vote on the abstracts only and then the organising committee can make the call of adding some some big names to draw the expected attention to the conference. They may even use some abstract that wasn't voted that much, but the committee still thinks it's an good idea. I dislike the popularity contest because it is too easily biased with lobbying (please vote for me) as we have seen in the past years. We'd better aim at quality than lobbying... Best, Markus ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On 10/02/2012 01:43 PM, Markus Neteler wrote: On Tue, Oct 2, 2012 at 1:36 PM, Volker Mische volker.mis...@gmail.com wrote: Hi all, I don't agree. I like the idea of having the community vote on the abstracts only and then the organising committee can make the call of adding some some big names to draw the expected attention to the conference. They may even use some abstract that wasn't voted that much, but the committee still thinks it's an good idea. I dislike the popularity contest because it is too easily biased with lobbying (please vote for me) as we have seen in the past years. We'd better aim at quality than lobbying... Hi Markus, does it means you against a public voting in general? Or against one which includes names? Or do you like the idea of a public voting which only contains the abstracts but nothing else? Cheers, Volker ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
Hi Volker, On Tue, Oct 2, 2012 at 1:57 PM, Volker Mische volker.mis...@gmail.com wrote: Hi Markus, does it means you against a public voting in general? Or against one which includes names? Or do you like the idea of a public voting which only contains the abstracts but nothing else? I never liked much the public voting. And abstracts without names are often still easy to attribute to a name due to the topic... Best Markus ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process
I fully agree with Cameron on this one. Our development and discussions are sufficiently open that any anonymous person providing the community with valuable contributions can become the next big name, or at least big enough to be selected by the current selection process. regards, thomas On Tue, Oct 2, 2012 at 1:36 AM, Cameron Shorter cameron.shor...@gmail.com wrote: I believe that for the general program, we should publish both the presenter and abstract. Reasons: 1. I'm attracted to a talk by both the topic and the presenter. I'm more likely to listen to a talk by someone who has a deep knowledge of a topic, which typically equates to someone with a big reputation. 2. And I think it is appropriate that people who have committed much time to the Open Source community, and hence have built up a big reputation, are allowed to be recognised by the selection community. 3. It also makes good business sense to the FOSS4G conference, as big names on the program will likely attract more delegates, and will likely have the delegates going away satisfied that they have seen presentations that they wanted to see. 4. The alternative of only seeing an abstract when voting is that anyone who can write a good abstract can potentially present on a topic, even if they don't have a deep insight in the topic of interest. On 2/10/2012 4:59 AM, Schlagel, Joel D IWR wrote: I believe anonymous reviews has a place as a component of paper selection - as a compliment to editorial review and professional judgement.FOSS4G conference is the number one marketing opportunity for the OSGEO community. We should make a deliberate effort to have a balance between inward focused technical / developer oriented presentations and outward focused policy / success / benefit type good news presentations. -joel From: discuss-boun...@lists.osgeo.org [discuss-boun...@lists.osgeo.org] on behalf of Paul Ramsey [pram...@opengeo.org] Sent: Monday, October 01, 2012 2:43 PM To: Volker Mische Cc: osgeo-discuss Subject: Re: [OSGeo-Discuss] FOSS4G presentation review process I'm in favour too. It has potential, let's see how an anonymous community process works in practice. P. On Mon, Oct 1, 2012 at 11:31 AM, Volker Mische volker.mis...@gmail.com wrote: Hi all, On 10/01/2012 06:10 PM, Barry Rowlingson wrote: In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what do you - the OSGeo community at large - think? I do have a google poll nearly ready on this, but lets have a bit of a debate here and maybe it won't even be necessary. I think an anonymous selection process makes a lot of sense. I personally always hoped that people don't do a please up-vote me campaigns on blogs or Twitter, but it happened. It will still be possible as people could publish the titles of the abstract, but I hope this won't happen and everyone will play along nicely. One thing we have to keep in mind, that this conference is different from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's hot in the fast changing JavaScript world. The audience are definitely non-beginners. At the FOSS4G the audience is way more wide-spread. It ranges from beginners to absolute pros. Hence there are also talks that are kind of the same every year. Things that come to my mind are my own talks, which are always about GeoCouch, or the State of ... talks. They have a place, but you'd know upfront the the State of GeoServer
Re: [OSGeo-Discuss] FOSS4G presentation review process
On 10/2/12 3:05 AM, Barry Rowlingson wrote: On Tue, Oct 2, 2012 at 1:01 AM, Bruce Bannerman b.banner...@bom.gov.au wrote: (snip) it seems that we have two processes going on - what sounds like a good talk, and who sounds like a good speaker. (snip) Beyond that, there is the question of selecting between talks to establish the balance you intend. Does the conference want to promote new, quirky, innovative, free software projects or to market the large, established, already successful projects? To promote the former, you might have to accept talks by folk you do not know, by folk who have a hard time speaking English, by folk who do not yet have the polish of the dominant players, by folk who are younger or less in the know. Also, you might have to limit the number of talks on dominant projects or by dominant groups. Just picking the 'good' talks may lead the conference to once again have many talks about the same projects that have come to dominate and fewer talks from new talent. Therefore picking talks only on the individual merits, whether of the abstract, topic, or speaker, leads to a particular kind of gathering, geared towards past success rather than towards fostering future diversity. ~adrian ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process
On Tue, Oct 2, 2012 at 3:03 PM, Adrian Custer acus...@gmail.com wrote: Just picking the 'good' talks may lead the conference to once again have many talks about the same projects that have come to dominate and fewer talks from new talent. Therefore picking talks only on the individual merits, whether of the abstract, topic, or speaker, leads to a particular kind of gathering, geared towards past success rather than towards fostering future diversity. Yes, this is why the community rating is only part of the process. The intention of it is, I think, to give the committee an idea of community feeling for the proposals. If the top 100 community-related proposals are all about PostGIS the committee should make some adjustments for balance, but would at least know what the top 10 rated PostGIS proposals were and have a PostGIS stream for them. Also, FOSS4G is not an Unconference... Barry ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
On 2/10/2012 9:34 PM, Bart van den Eijnden wrote: One thing I really disliked in the past is that the size of abstracts really differed and abstracts were sometimes really large. Please limit people to a small and to-the-point abstract, at least for the voting process. +1 It is important for both the reviewing and for readability of the program for presentation descriptions to be concise. I suggest encouraging abstracts to only have 2, and a max of 50 words. (Preferably 30 words). -- Cameron Shorter Geospatial Solutions Manager Tel: +61 (0)2 8570 5050 Mob: +61 (0)419 142 254 Think Globally, Fix Locally Geospatial Solutions enhanced with Open Standards and Open Source http://www.lisasoft.com ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
[OSGeo-Discuss] FOSS4G presentation review process
In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what do you - the OSGeo community at large - think? I do have a google poll nearly ready on this, but lets have a bit of a debate here and maybe it won't even be necessary. Barry ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process
On Oct 1, 2012, at 9:10 AM, Barry Rowlingson b.rowling...@lancaster.ac.uk wrote: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. If they don't have anything interesting to say, they should not be big draws. Selection should be on the character of content rather than the size of the badge. -- Puneet Kishor ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process
Hi all, On 10/01/2012 06:10 PM, Barry Rowlingson wrote: In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what do you - the OSGeo community at large - think? I do have a google poll nearly ready on this, but lets have a bit of a debate here and maybe it won't even be necessary. I think an anonymous selection process makes a lot of sense. I personally always hoped that people don't do a please up-vote me campaigns on blogs or Twitter, but it happened. It will still be possible as people could publish the titles of the abstract, but I hope this won't happen and everyone will play along nicely. One thing we have to keep in mind, that this conference is different from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's hot in the fast changing JavaScript world. The audience are definitely non-beginners. At the FOSS4G the audience is way more wide-spread. It ranges from beginners to absolute pros. Hence there are also talks that are kind of the same every year. Things that come to my mind are my own talks, which are always about GeoCouch, or the State of ... talks. They have a place, but you'd know upfront the the State of GeoServer e.g. is done by one of the big names of GeoServer and respectively a talk mentioning GeoCouch is probably me. What I want to say is, you can't fully prevent that people up-vote well known names. Of course there still needs to be the review process by the programm committee that makes the final call, so that we e.g. don't have 5 talks from the same person. To conclude: I'm in favour of trying it and seeing how it works. Cheers, Volker ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] FOSS4G presentation review process
I believe that for the general program, we should publish both the presenter and abstract. Reasons: 1. I'm attracted to a talk by both the topic and the presenter. I'm more likely to listen to a talk by someone who has a deep knowledge of a topic, which typically equates to someone with a big reputation. 2. And I think it is appropriate that people who have committed much time to the Open Source community, and hence have built up a big reputation, are allowed to be recognised by the selection community. 3. It also makes good business sense to the FOSS4G conference, as big names on the program will likely attract more delegates, and will likely have the delegates going away satisfied that they have seen presentations that they wanted to see. 4. The alternative of only seeing an abstract when voting is that anyone who can write a good abstract can potentially present on a topic, even if they don't have a deep insight in the topic of interest. On 2/10/2012 4:59 AM, Schlagel, Joel D IWR wrote: I believe anonymous reviews has a place as a component of paper selection - as a compliment to editorial review and professional judgement.FOSS4G conference is the number one marketing opportunity for the OSGEO community. We should make a deliberate effort to have a balance between inward focused technical / developer oriented presentations and outward focused policy / success / benefit type good news presentations. -joel From: discuss-boun...@lists.osgeo.org [discuss-boun...@lists.osgeo.org] on behalf of Paul Ramsey [pram...@opengeo.org] Sent: Monday, October 01, 2012 2:43 PM To: Volker Mische Cc: osgeo-discuss Subject: Re: [OSGeo-Discuss] FOSS4G presentation review process I'm in favour too. It has potential, let's see how an anonymous community process works in practice. P. On Mon, Oct 1, 2012 at 11:31 AM, Volker Mische volker.mis...@gmail.com wrote: Hi all, On 10/01/2012 06:10 PM, Barry Rowlingson wrote: In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what do you - the OSGeo community at large - think? I do have a google poll nearly ready on this, but lets have a bit of a debate here and maybe it won't even be necessary. I think an anonymous selection process makes a lot of sense. I personally always hoped that people don't do a please up-vote me campaigns on blogs or Twitter, but it happened. It will still be possible as people could publish the titles of the abstract, but I hope this won't happen and everyone will play along nicely. One thing we have to keep in mind, that this conference is different from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's hot in the fast changing JavaScript world. The audience are definitely non-beginners. At the FOSS4G the audience is way more wide-spread. It ranges from beginners to absolute pros. Hence there are also talks that are kind of the same every year. Things that come to my mind are my own talks, which are always about GeoCouch, or the State of ... talks. They have a place, but you'd know upfront the the State of GeoServer e.g. is done by one of the big names of GeoServer and respectively a talk mentioning GeoCouch is probably me. What I want to say is, you can't fully prevent that people up-vote well known names. Of course there still needs to be the review process by the programm committee that makes the final call, so that we e.g. don't have 5 talks from the same person. To conclude: I'm in favour of trying it and seeing how it works. Cheers
Re: [OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]
Agreed. Well said Cameron, with the aside that there may be an interesting talk from a previously little known person. I suggest leaving this to the discretion of the LOC and interested parties who subscribe to that year's FOSS4G mailing list. A popularity campaign is not required or wanted. Bruce On 2/10/12 9:36 AM, Cameron Shorter cameron.shor...@gmail.com wrote: I believe that for the general program, we should publish both the presenter and abstract. Reasons: 1. I'm attracted to a talk by both the topic and the presenter. I'm more likely to listen to a talk by someone who has a deep knowledge of a topic, which typically equates to someone with a big reputation. 2. And I think it is appropriate that people who have committed much time to the Open Source community, and hence have built up a big reputation, are allowed to be recognised by the selection community. 3. It also makes good business sense to the FOSS4G conference, as big names on the program will likely attract more delegates, and will likely have the delegates going away satisfied that they have seen presentations that they wanted to see. 4. The alternative of only seeing an abstract when voting is that anyone who can write a good abstract can potentially present on a topic, even if they don't have a deep insight in the topic of interest. On 2/10/2012 4:59 AM, Schlagel, Joel D IWR wrote: I believe anonymous reviews has a place as a component of paper selection - as a compliment to editorial review and professional judgement.FOSS4G conference is the number one marketing opportunity for the OSGEO community. We should make a deliberate effort to have a balance between inward focused technical / developer oriented presentations and outward focused policy / success / benefit type good news presentations. -joel From: discuss-boun...@lists.osgeo.org [discuss-boun...@lists.osgeo.org] on behalf of Paul Ramsey [pram...@opengeo.org] Sent: Monday, October 01, 2012 2:43 PM To: Volker Mische Cc: osgeo-discuss Subject: Re: [OSGeo-Discuss] FOSS4G presentation review process I'm in favour too. It has potential, let's see how an anonymous community process works in practice. P. On Mon, Oct 1, 2012 at 11:31 AM, Volker Mische volker.mis...@gmail.com wrote: Hi all, On 10/01/2012 06:10 PM, Barry Rowlingson wrote: In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we intended to select presentations for the main track of the conference. Some discussion amongst the committee has been going on, and we think it necessary to informally poll the community to get a feel for what method is preferred. Previous FOSS4Gs have not used anonymous reviews (note: the Academic Track will be a double-blind review process, we are discussing the main conference presentations here), and have used a blend of committee reviews and community reviews. Note that even with a numerical ranking system its normally still necessary to do a manual step to get a balanced conference. The big change we could do would be to have anonymous community reviews. Proposals would be rated based on title and abstract only. The arguments for this include: * selection is on quality of proposal rather than bigness of name * rating procedure can prevent up-votes from whoever has the most followers on twitter * promotes inclusivity: http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html and against arguments include: * some names are big draws, and it would be disappointing to not have someone because their abstract wasn't that exciting. * previous FOSS4Gs have used non-anonymous reviewing and that worked fine. Why change it? * it may be hard to distil an exciting talk into an abstract without losing the excitement. So, as this would be quite a change for FOSS4G, what do you - the OSGeo community at large - think? I do have a google poll nearly ready on this, but lets have a bit of a debate here and maybe it won't even be necessary. I think an anonymous selection process makes a lot of sense. I personally always hoped that people don't do a please up-vote me campaigns on blogs or Twitter, but it happened. It will still be possible as people could publish the titles of the abstract, but I hope this won't happen and everyone will play along nicely. One thing we have to keep in mind, that this conference is different from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's hot in the fast changing JavaScript world. The audience are definitely non-beginners. At the FOSS4G the audience is way more wide-spread. It ranges from beginners to absolute pros. Hence there are also talks that are kind of the same every year. Things that come to my mind are my own talks, which are always about GeoCouch, or the State of ... talks. They have a place, but you'd know upfront the the State of GeoServer e.g
