This proposal isn't really about the merits of any particular study. I only offered a link to the discussions about my most recent user study because I felt it was a good example of push-back from Wikipedia editors.
For a better view of the *initial* troubles with the first failed study, see Katherine's talk page http://en.wikipedia.org/wiki/User_talk:KatherinePanciera. I'm most concerned about what happen in that study. After Katherine asked for Wikipedia users to participate simply by taking a survey, her account was nominated for deletion for allegedly violating an obscure policy that did not match he actions. Katherine had no policy to cite in order to defend herself. For simply asking people to participate in a survey, her account was nearly banned. I'd suggest you have a read through the proposal (it is actually quite small) in order to more clearly understand the problem we wish to solve. The first section is devoted to just that. Thanks for the comments! -Aaron Brendan O'Connor wrote: > On Wed, Jun 3, 2009 at 12:06 PM, Aaron Halfaker <[email protected]> wrote: > >> Hello, >> >> I am re-posting here because I thought people might be interested in a >> proposal >> (http://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Wikipedia_research_review >> <http://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28proposals%29#Wikipedia_research_review>) >> that my lab-mates and I have been working on. The gestalt of the >> proposal is that Wikipedia needs a review system to both protect itself >> from questionable research activities and to allow good research to take >> place within the system. Currently research involving anything other >> than the database snapshots is difficult because there is no formal >> policy related to research within Wikipedia. >> > > Could you explain a little more what's the exact problem you want to solve? > > I followed the links to individual studies, but I only understood the > 2nd one about an interface modification. > ( > http://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Wikipedia_interface_modification_.28NICE.29_study > ) > > It sounds like there were two problems here: (1) you didn't explain > what your software was -- people wanted assurance that the thing was > safe and the like; and (2) people weren't persuaded that participating > in the study was worth their time. > > I don't understand why having an IRB-like committee will help solve > things. In the academic setting, IRB's are good, or at least > important, because they try to stop unethical research that harms the > overall research community (or at the very least exposes the > sponsoring university to lawsuits). > > It's not clear to me why Wikipedia research's relationship to > Wikipedians is analogous. The bigger problem seems to be persuading > people they should participate in the study. Having approval of a > committee might have helped a little bit for the first objection you > had -- that they have no idea what the software was -- but it wouldn't > have helped with the second -- that they didn't think it was worth > participating in. > > -- > Brendan O'Connor - http://anyall.org > > > > > >> See failed examples of attempted research here: >> * Failed mentoring study: >> http://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/IncidentArchive395#Canvassing >> * Failed interface launch: >> http://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Wikipedia_interface_modification_.28NICE.29_study >> >> -Aaron Halfaker >> GroupLens Research >> University of Minnesota >> >> _______________________________________________ >> Wiki-research-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > _______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
