Hello, Thanks for signalling me Aaron. I would participate in talks about research standards with other people. Perhaps the research committee can convene at the hackathon in Mexico City.
I feel that if these issues were addressed, then there would be many fewer problems. 1. All research on human subjects in the Wikimedia community has to be registered by creating a research page on meta. The research page needs to identify the researcher, give their contact information (for reporting complaints), describe the research minimally (2-3 sentences at least), and say which demographic of Wikimedians will be contacted and how many of them will be contacted. 2. There should be a research review process. To start, this could simply require that researchers have set up a research page. If there is demand for more checks then they can be added after review is set up. One additional demand or check which we might want to mandate is some cap on how many times and how many Wikimedians any given research study is allowed to contact, and in what ways. The page for the study in question says nothing about how the study will contact Wikipedians or how many it will contact, and this issue has caused a problem in this case. 3. Whenever researchers advertise a study, they must be required to provide a link to their research registration page. This gives the Wikimedia community a place to have discussions about the research and lodge complaints if necessary. The problem that I would like to see addressed is a lack of respect for the time of Wikimedia community members. Practically all researchers assume that the Wikimedia community is infinitely large, and that it is no burden to consume volunteer time with research when that volunteer time could be spent developing Wikimedia projects, and that they are the only researchers seeking to tap into Wikimedia community volunteer time. The reality is that most researchers want access to small slices of the Wikimedia community, and that those people's time is scarce and that it is costly to provide them volunteer time, and that most researchers want access to the same demographics so any given study usually seeks to contact Wikipedians who are routinely contacted for research. As more research is done, there will be more tension. I want to encourage more and better research and I know if we promoted better research practices then researchers would comply. They are accustomed to following special guidelines for special populations and I think that presenting them clear demands would make them happier and make Wikimedians happier. yours, On Jun 27, 2015 12:59 PM, "Aaron Halfaker" <[email protected]> wrote: > RCom, as far as I know has not been active in the past year or more (last >> meeting was on Dec. 22, 2011). > > > *RCom is not dead. It changed into something less formal and less > hierarchical. You can still email me and Dario to get support for your > research plans. We'd still reconvene the committee if it looks like > that'll help. * > > While RCom hasn't met in a long time, the process for subject recruitment > hasn't slowed. We don't have a technical requirement that all recruitment > studies must follow The Process, but I have been helping researchers > document their studies and obtain feedback and sometimes consensus for more > than five years now. > > Really, RCom has morphed slowly into the Research Team at the WMF + a few > interested volunteers that we can manage to pull in to help us with review > work (shout out to Daniel Mietchen, Nemo, Yaroslav & BluRasberry). Within > the research team, we *do* have structured processed for supporting > researchers access to data and engineering support, but subject recruitment > has been mostly left in my (volunteer time) hands. > > Regretfully, I wasn't involved in the planning of this project or I would > have directed it towards best practices for minimizing disruption > <https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment> -- e.g. an > RFC. I would have also pushed Leila to find a way to make posts on talk > pages work (since they are known to be generally preferable, police-able, > etc.), but I can understand why concerns around privacy might be worth > discussion. I regret that this discussion only happened after-the-fact as > it could have informed the study design for the better. FWIW, SuggestBot > posts recommendations on user talk pages and also does not filter for > offensive content (to my knowledge). > > Finally, I think it is important to consider the source of this research > work. Leila is not some random academic or industry researcher who is > planning to take advantage of Wikipedians for a study, but not give back. > Leila is working with a team at the WMF tasked with building better > translation tools. She helped them design an experiment that would explore > the effectiveness of these tools so that when something is deployed, it's > actually better and we know it scientifically. A lot of the work I do with > external researchers is to help make sure that their work has the potential > to benefit Wikipedia/Wikipedians/Wikimedia/Open knowledge. In this case, > the Leila's team is just helping the product teams engage in best practices > around empirical software change practice. After all, every software > deployment is an experiment that is inflicted upon you without consent. In > this case, Leila's job is making sure that we know the effect before we > deploy. > > So, what I really mean to say is: > > 1. You're right. We should do this better. We have a process and > everyone should go through it. It might have caught some of the issues > that have been raised. > 2. Leila is WMF staff. She's trying to help the WMF build better > software for the purpose of benefiting Wikipedians. Her team deserves some > slack. The alternative of not running the study is less desirable. > > -Aaron > > On Sat, Jun 27, 2015 at 12:56 PM, Michelle Paulson <[email protected] > > wrote: > >> Hi All, >> >> Please see in-line below. >> >> -Michelle >> >> On Saturday, June 27, 2015, Leila Zia <[email protected]> wrote: >> >> > + Michelle Paulson >> > >> > On Sat, Jun 27, 2015 at 7:37 AM, Pine W <[email protected] >> > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > >> >> This issue is also being discussed on the Research mailing list. >> >> >> >> I have three questions: >> >> >> >> 1. Was this outreach method approved by RCom? >> >> >> > No, and RCom, as far as I know has not been active in the past year or >> > more (last meeting was on Dec. 22, 2011). This is a research from the >> > Research team in the WMF. >> > >> >> 2. Email addresses are nonpublic information on-wiki unless they are >> >> proactively and publicly disclosed by users. Does the bulk collection >> of >> >> nonpublic email addresses in this manner and the bulk provision of >> those >> >> addresses to researchers for their use in this campaign violate the >> >> Wikimedia privacy policy? The policy states regarding email, "We use >> your >> >> email address to let you know about things that are happening with the >> >> Foundation, the Wikimedia Sites, or the Wikimedia movement, such as >> telling >> >> you important information about your account, letting you know if >> something >> >> is changing about the Wikimedia Sites or policies, and alerting you >> when >> >> there has been a change to an article that you have decided to >> follow." The >> >> bulk scraping of email addresses from account registrations for >> research >> >> and outreach purposes doesn't appear to be contemplated or authorized >> under >> >> the privacy policy. >> >> >> > Michelle can help with this one as this is related to Legal. Note that >> > it's weekend here and this may have to wait until Monday. >> > >> >> The research team did speak to me prior to beginning this project to >> ensure >> that they complied with the WMF privacy policy. It is my view that this >> type of use falls within the permissible potential uses for email >> addresses >> under the policy. The examples listed in the policy are meant to be >> illustrative, not exclusive -- the absence of this situation as an >> enumerated example shouldn't be taken as a prohibition. >> >> That said, it is a new use and therefore, will and should be the subject >> of >> discussion and debate. It is such feedback and testing that will help us >> refine email practices to be both effective and reflective of community >> values. >> >> > 3. Wouldn't talk pages be a more appropriate outreach method than bulk >> >> email? >> >> >> > The reason we chose email over talk pages (or Echo notifications) is >> > explained here >> > < >> https://meta.wikimedia.org/wiki/Research_talk:Increasing_article_coverage#.C2.AB_recommander_par_courriel_des_articles_.C3.A0_cr.C3.A9er.E2.80.A6_.C2.BB >> >. >> > >> > >> > Hope this helps. >> > >> > Best, >> > Leila >> > >> >> Thanks, >> >> >> >> Pine >> >> >> >> _______________________________________________ >> >> Wiki-research-l mailing list >> >> [email protected] >> >> <javascript:_e(%7B%7D,'cvml','[email protected]');> >> >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> >> >> >> > >> >> -- >> == >> Michelle Paulson >> Senior Legal Counsel >> Wikimedia Foundation >> 149 New Montgomery Street, 6th Floor >> San Francisco, CA 94105 >> [email protected] >> 415.839.6885 ext. 6608 (Office) >> 415.882.0495 (Fax) >> >> *NOTICE: This message may be confidential or legally privileged. If you >> have received it by accident, please delete it and let us know about the >> mistake. As an attorney for the Wikimedia Foundation and for legal/ethical >> reasons, I cannot give legal advice to, or serve as a lawyer for, >> community >> members, volunteers, or staff members in their personal capacity. For more >> on what this means, please see our legal disclaimer >> <https://meta.wikimedia.org/wiki/Wikimedia_Legal_Disclaimer>.* >> _______________________________________________ >> Wikimedia-l mailing list, guidelines at: >> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines >> [email protected] >> <https://meta.wikimedia.org/wiki/Mailing_lists/[email protected]> >> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, >> <mailto:[email protected]?subject=unsubscribe> >> > > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
_______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
