Two reviews per submission might work if we had a clear set of criteria that the reviewers were following and sufficient training of the reviewers that they were broadly consistent in their marking. But when you get the same presentation being marked as 5 and 8, as one of mine was then the suspicion is that the assessors are not working to the same criteria as each other. That wouldn't matter so much if they were all assessing all submissions, except that an assessor who varied between 0 and ten points would have far more influence than assessors who usually voted 6, 7 or 8. But having that level of inconsistency and only two reviews per submission makes the process a lottery that depends on who the two reviewers are for your submission.
As for the content of the reviews, I don't consider that either "5 (average)" or "6 (rather interesting) tell me anything as to why my submissions were rejected. The other two reviews at least managed one or two lines. One of them even stretched to two sentences. Hope Montreal manages something a bit better, I'm sure either Manilla or Perth would have done. WereSpielChequers > On 3 Feb 2016, at 23:22, Dariusz Jemielniak <[email protected]> wrote: > > hi, > > I have some comments as a person from Academia (and not involved in Wikimania > process in any way): > > 1. Short reviews are definitely not helping in addressing the frustration of > rejection, yet are quite common in academic peer reviewing, especially for > conferences. > > 2. Double blind peer review (not knowing who is reviewed, and not knowing who > reviews) is a standard in Academia, although some perceive it as contributing > to lack of responsibility (especially true in competitive journal > submissions). > > 3. Two reviewers per submission is absolutely on par with the conference > standards I'm used to. Sometimes there are three, but two is absolutely > acceptable (although a third opinion should be used if the two disagree too > much). > > 4. It could be useful to sensitize the reviewers that the main purpose of the > review is to help the author to do better next time. > > 5. All this is volunteer work. We should be, generally, grateful to reviewers > (but in the same time grateful to the contributors, too). > > best, > > dj > >> On Wed, Feb 3, 2016 at 5:26 PM, Maarten Dammers <[email protected]> wrote: >> What kind of ridiculous process is this? This is all I got: >> >> =============== >> >> ----------------------- REVIEW 1 --------------------- >> PAPER: 194 >> TITLE: GLAM+Wikidata >> AUTHORS: Sandra Fauconnier and Maarten Dammers >> >> OVERALL EVALUATION: 8 (Very good) >> >> ----------- REVIEW ----------- >> 8 >> >> >> ----------------------- REVIEW 2 --------------------- >> PAPER: 194 >> TITLE: GLAM+Wikidata >> AUTHORS: Sandra Fauconnier and Maarten Dammers >> >> OVERALL EVALUATION: 6 (Rather interesting) >> >> ----------- REVIEW ----------- >> 6 >> >> ============== >> >> So only two people reviewed this? Who are these people? Why is this secret? >> Last year I had 5 people reviewing my submission [1]. >> >> Maarten >> >> [1] https://wikimania2015.wikimedia.org/wiki/Submission_review/5 >> >> Op 3-2-2016 om 23:15 schreef Andy Mabbett: >>> I've just received feedback on one of my pitches saying, in part: >>> >>> "Bad boy Andy! This is supposed to be an anonymous review process, so >>> starting your abstract with your own name, is not entirely fair." >>> >>> -- >>> Andy Mabbett >>> @pigsonthewing >>> http://pigsonthewing.org.uk >>> >>> >>> >>> _______________________________________________ >>> Wikimania-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/wikimania-l >> >> >> _______________________________________________ >> Wikimania-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wikimania-l > > > > -- > > __________________________ > prof. dr hab. Dariusz Jemielniak > kierownik katedry Zarządzania Międzynarodowego > i grupy badawczej NeRDS > Akademia Leona Koźmińskiego > http://nwrds.kozminski.edu.pl > > członek Akademii Młodych Uczonych Polskiej Akademii Nauk > członek Komitetu Polityki Naukowej MNiSW > > Wyszła pierwsza na świecie etnografia Wikipedii "Common Knowledge? An > Ethnography of Wikipedia" (2014, Stanford University Press) mojego autorstwa > http://www.sup.org/book.cgi?id=24010 > > Recenzje > Forbes: http://www.forbes.com/fdc/welcome_mjx.shtml > Pacific Standard: > http://www.psmag.com/navigation/books-and-culture/killed-wikipedia-93777/ > Motherboard: http://motherboard.vice.com/read/an-ethnography-of-wikipedia > The Wikipedian: > http://thewikipedian.net/2014/10/10/dariusz-jemielniak-common-knowledge > _______________________________________________ > Wikimania-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikimania-l
_______________________________________________ Wikimania-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikimania-l
