Re: [Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
Replicated it how? Jonathan On Wed, Aug 12, 2015 at 3:06 AM, James Salsman jsals...@gmail.com wrote: Has anyone replicated the experiment described in http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ yet? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF) ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] Studies of text quality
Hi David, Welcome to wiki-research-l! What are you trying to learn from this study? Also, what is this Quality Assisted Editor for Wikipedia you speak of? Is it something we can try out? -Aaron On Wed, Aug 12, 2015 at 2:01 PM, David Strohmaier david.strohma...@gmx.at wrote: To whome it may concern, My name is David Strohmaier and I'm doing my Master Thesis. We implemented a Quality Assisted Editor for Wikipedia, that should help to detect quality flaws quite quickly. Right now we are in the middle of evaluating the tool. If you would spend us 15 - 25 minutes of your time you would really help us with the evaluation. All you have to do is evaluate the quality of two wikipedia articles. * How to do it? 1. Please read the Wikipedia:Featured article criteria before you start. Each question in the surveys is tagged with a hint on which criteria it is targeting. Wikipedia:Featured article criteria: https://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteriahttps://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria https://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria 2. Open the first Wikipedia article (Doctor Phosphorus) and the connected survey. It is important that you open the article of Doctor Phosphorus with the link in the Mail, because a special article revision is chosen! Doctor Phosphorus: https://en.wikipedia.org/w/index.php?title=Doctor_Phosphorusoldid=445748339https://en.wikipedia.org/w/index.php?title=Doctor_Phosphorusoldid=445748339 https://en.wikipedia.org/w/index.php?title=Doctor_Phosphorusoldid=445748339 http://goo.gl/forms/ESgoyfd8ADhttp://goo.gl/forms/ESgoyfd8AD http://goo.gl/forms/ESgoyfd8AD 3. Please fill out the survey. The questions concerning the whole article should be answered based on the three sections you have read. 4. Open the second Wikipedia article (Moon) and the connected survey. It is important that you open the article of Moon with the link in the Mail, because a special article revision is chosen! Moon: https://en.wikipedia.org/w/index.php?title=Moonoldid=670521118https://en.wikipedia.org/w/index.php?title=Moonoldid=670521118 https://en.wikipedia.org/w/index.php?title=Moonoldid=670521118 http://goo.gl/forms/PnlbMtA1X3http://goo.gl/forms/PnlbMtA1X3 http://goo.gl/forms/PnlbMtA1X3 5. Please fill out the survey. The questions concerning the whole article should be answered based on the three sections you have read. Thank you! I*f you have any questions, pls contact me: david.strohma...@gmx.at * Best regards, David Strohmaier * ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
Google the paper's title to see many related papers, including this one http://onlinelibrary.wiley.com/doi/10./j.1083-6101.2011.01551.x/full Not the same as that paper, but orbiting that space... On Wed, Aug 12, 2015 at 3:37 PM, James Salsman jsals...@gmail.com wrote: Jonathan, I am so sorry http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ is behind a paywall. It wasn't when I first found it, and that version is miles away from me at the moment. It describes a truly fascinating empirical simulation laboratory participation experiment, which shows that anonymous review is more accurate than review with identity disclosure, which is actually very easy to find literally centuries of replication, but it also found that the costs were more similar than conventional wisdom. I want everyone to see it because of what the specific experiment says about ways to detect bias at the lowest possible cost. I have a feeling that you will quickly think of ways to extend it to study projects' editing. Can someone who has access to that paper please share the method and results as fair use? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
Thanks, James. That sounds really interesting. I hope to read it. NOTE TO EVERYONE: if you do have access to this closed-access paper, please do NOT attach a PDF of it to an email you send to this mailing list. Turns out it's a real pain to remove these files from our public list archive (I found that out the hard way recently... :/). /me shakes fist at the publishing-industrial complex J On Wed, Aug 12, 2015 at 3:37 PM, James Salsman jsals...@gmail.com wrote: Jonathan, I am so sorry http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ is behind a paywall. It wasn't when I first found it, and that version is miles away from me at the moment. It describes a truly fascinating empirical simulation laboratory participation experiment, which shows that anonymous review is more accurate than review with identity disclosure, which is actually very easy to find literally centuries of replication, but it also found that the costs were more similar than conventional wisdom. I want everyone to see it because of what the specific experiment says about ways to detect bias at the lowest possible cost. I have a feeling that you will quickly think of ways to extend it to study projects' editing. Can someone who has access to that paper please share the method and results as fair use? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF) ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
Jonathan, I am so sorry http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ is behind a paywall. It wasn't when I first found it, and that version is miles away from me at the moment. It describes a truly fascinating empirical simulation laboratory participation experiment, which shows that anonymous review is more accurate than review with identity disclosure, which is actually very easy to find literally centuries of replication, but it also found that the costs were more similar than conventional wisdom. I want everyone to see it because of what the specific experiment says about ways to detect bias at the lowest possible cost. I have a feeling that you will quickly think of ways to extend it to study projects' editing. Can someone who has access to that paper please share the method and results as fair use? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Re: [Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
I assume they mean whether we use identity disclosure mechanisms. And the answer is, to do so would compromise the anonymity on which many of our submitter's depend. That is a better question for wikileaks, though perhaps coming at it in not quite the angle you were expecting, or yelp, or google. On Wed, Aug 12, 2015 at 10:50 AM, Jonathan Morgan jmor...@wikimedia.org wrote: Replicated it how? Jonathan On Wed, Aug 12, 2015 at 3:06 AM, James Salsman jsals...@gmail.com wrote: Has anyone replicated the experiment described in http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ yet? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF) ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
[Wiki-research-l] identity disclosure hurt the reliability of review systems, but not necessarily efforts provision
Has anyone replicated the experiment described in http://aisel.aisnet.org/amcis2015/e-Biz/GeneralPresentations/11/ yet? ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
[Wiki-research-l] Studies of text quality
To whome it may concern, My name is David Strohmaier and I'm doing my Master Thesis. We implemented a Quality Assisted Editor for Wikipedia, that should help to detect quality flaws quite quickly. Right now we are in the middle of evaluating the tool. If you would spend us 15 - 25 minutes of your time you would really help us with the evaluation. All you have to do is evaluate the quality of two wikipedia articles.* How to do it? 1. Please read the Wikipedia:Featured article criteria before you start. Each question in the surveys is tagged with a hint on which criteria it is targeting. Wikipedia:Featured article criteria: https://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria 2. Open the first Wikipedia article (Doctor Phosphorus) and the connected survey. It is important that you open the article of Doctor Phosphorus with the link in the Mail, because a special article revision is chosen! Doctor Phosphorus: https://en.wikipedia.org/w/index.php?title=Doctor_Phosphorusoldid=445748339 http://goo.gl/forms/ESgoyfd8AD 3. Please fill out the survey. The questions concerning the whole article should be answered based on the three sections you have read. 4. Open the second Wikipedia article (Moon) and the connected survey. It is important that you open the article of Moon with the link in the Mail, because a special article revision is chosen! Moon: https://en.wikipedia.org/w/index.php?title=Moonoldid=670521118 http://goo.gl/forms/PnlbMtA1X3 5. Please fill out the survey. The questions concerning the whole article should be answered based on the three sections you have read. Thank you! I*f you have any questions, pls contact me: david.strohma...@gmx.at * Best regards, David Strohmaier * ___ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
