Re: [Wikimedia-l] Experimenting on Wikimedians

2014-11-28 Thread Federico Leva (Nemo)

David Gerard, 19/11/2014 13:12:

The increase in efficiency through the banner campaign has been truly
remarkable!

Are you able to provide data for any of these claims?



If you take a moment to do the obvious thing and look on Meta,
specifically at the obvious page ([[Fundraising]]), you'll see links
to that exact thing.


I've read that page several times and I'm unable to provide the data you 
claim it contains.

I've asked data 20 months ago and it's not been provided yet.
https://meta.wikimedia.org/wiki/Talk:Fundraising_2012/Report

Neom

___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe

Re: [Wikimedia-l] Experimenting on Wikimedians

2014-11-19 Thread David Gerard
On 19 November 2014 11:36, Federico Leva (Nemo) nemow...@gmail.com wrote:
 Chris Keating, 02/11/2014 10:52:

 The increase in efficiency through the banner campaign has been truly
 remarkable!

 Are you able to provide data for any of these claims?



If you take a moment to do the obvious thing and look on Meta,
specifically at the obvious page ([[Fundraising]]), you'll see links
to that exact thing.

Seriously, were you claiming Chris was lying? Were you seriously
asserting you didn't think such data existed? They've been rigorously
A/B testing the banners for the past several years.


- d.

___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe

Re: [Wikimedia-l] Experimenting on Wikimedians

2014-11-02 Thread Chris Keating


  Of course the stark reality is that A/B testing on users (typically
 readers, not editors) during the annual Wikimedia Foundation fundraiser
 has been a major component of the Wikimedia Foundation's growth.


 In part that's a myth. The income has been increased simply by making the
 banners larger, brighter, naughtier and alarming (we're in danger, bla
 bla). Sometimes they take more space than is left to the article; sometimes
 they can't be dismissed.


Hi Nemo - I can't agree with this at all. The banners from the 2013
campaign (the last I can readily find) are no bigger or scarier than those
from 2011. On the whole they are much less interruptive, as they are
displayed less consistently; and they attract far less third-party
attention than the Jimmy banners.

The increase in efficiency through the banner campaign has been truly
remarkable!

 If there was a way to get the same kind of result on (say) the number of
new editors who stick around and contribute more that would be great.

Chris
___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe

Re: [Wikimedia-l] Experimenting on Wikimedians

2014-11-01 Thread Pine W
Forwarding to Research and Analytics for discussion.

Pine
On Oct 30, 2014 8:13 PM, MZMcBride z...@mzmcbride.com wrote:

 Hi.

 Splitting this out from the GLAMs/Chapters thread, I continue to regularly
 wonder whether we need stricter guidelines and guidance in the area of
 experimenting on Wikimedians.

 Erik mentioned trying to further implement A/B testing in software
 development, but to me that quickly raises consent and trust issues. My
 view is that Wikimedians should be treated as colleagues, not customers.

 Of course the stark reality is that A/B testing on users (typically
 readers, not editors) during the annual Wikimedia Foundation fundraiser
 has been a major component of the Wikimedia Foundation's growth.

 Worth repeating, from https://meta.wikimedia.org/wiki/Experiments:

 ---
 Current practices in Web analytics reflect their commercial origins. For
 better or worse, the greatest motor behind the use of Web analytics has
 been the profit interests of online retailers and social networks, for
 whom the user is a commodity. These profit interests have profoundly
 shaped the discourse of Web analytics, setting both the tenor and the tone
 of debate (consider the values implicit in funnels, a term of art).

 A thoughtless application of Web analytics to Wikimedia wikis would import
 a moral outlook that is incompatible with (and, indeed, rightfully
 offensive to) its community. It also wouldn't work well, because neither
 Wikimedia wikis nor their editing communities are for sale. It is
 therefore crucial that technical efforts be accompanied by a process of
 reflection, the goal of which should be to articulate criteria for Web
 analytics that express and promote the broader ambitions of the Wikimedia
 movement and the moral commitments that underlie it.
 ---

 I think this about sums it up better than I ever could.

 MZMcBride



 ___
 Wikimedia-l mailing list, guidelines at:
 https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
 Wikimedia-l@lists.wikimedia.org
 Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
 mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe
___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe

Re: [Wikimedia-l] Experimenting on Wikimedians

2014-10-31 Thread Federico Leva (Nemo)
An interesting piece of corporate communication on the topic was 
http://blog.okcupid.com/index.php/we-experiment-on-human-beings/


I've expanded the Meta-Wiki page a bit, including the following additions:
* They number in the dozens and are usually documented in the Meta-Wiki 
[[Research]] namespace.
* Their outcome is often not used for any concrete deliverable, such as 
a merged change to MediaWiki core PHP code or a peer reviewed paper.
* Sometimes changes which are known to be potentially harmful, and would 
never (or hardly) pass standard code review, are deployed as 
experiments to bypass tougher public scrutiny. (This is also valid of 
fundraising banners, whose poor translations since 2011 are often 
actively damaging to the public opinion and understanding of Wikimedia 
projects.)


Nemo

P.s.:

MZMcBride, 31/10/2014 04:13:

Of course the stark reality is that A/B testing on users (typically
readers, not editors) during the annual Wikimedia Foundation fundraiser
has been a major component of the Wikimedia Foundation's growth.


In part that's a myth. The income has been increased simply by making 
the banners larger, brighter, naughtier and alarming (we're in danger, 
bla bla). Sometimes they take more space than is left to the article; 
sometimes they can't be dismissed.


___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe

[Wikimedia-l] Experimenting on Wikimedians

2014-10-30 Thread MZMcBride
Hi.

Splitting this out from the GLAMs/Chapters thread, I continue to regularly
wonder whether we need stricter guidelines and guidance in the area of
experimenting on Wikimedians.

Erik mentioned trying to further implement A/B testing in software
development, but to me that quickly raises consent and trust issues. My
view is that Wikimedians should be treated as colleagues, not customers.

Of course the stark reality is that A/B testing on users (typically
readers, not editors) during the annual Wikimedia Foundation fundraiser
has been a major component of the Wikimedia Foundation's growth.

Worth repeating, from https://meta.wikimedia.org/wiki/Experiments:

---
Current practices in Web analytics reflect their commercial origins. For
better or worse, the greatest motor behind the use of Web analytics has
been the profit interests of online retailers and social networks, for
whom the user is a commodity. These profit interests have profoundly
shaped the discourse of Web analytics, setting both the tenor and the tone
of debate (consider the values implicit in funnels, a term of art).

A thoughtless application of Web analytics to Wikimedia wikis would import
a moral outlook that is incompatible with (and, indeed, rightfully
offensive to) its community. It also wouldn't work well, because neither
Wikimedia wikis nor their editing communities are for sale. It is
therefore crucial that technical efforts be accompanied by a process of
reflection, the goal of which should be to articulate criteria for Web
analytics that express and promote the broader ambitions of the Wikimedia
movement and the moral commitments that underlie it.
---

I think this about sums it up better than I ever could.

MZMcBride



___
Wikimedia-l mailing list, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, 
mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe