Re: [Foundation-l] Donation Button Enhancement : Part 2
What about changing it every hour? Then you have sufficient randomness over time, and no flashy buttons. -- eia 2009/7/21 Erik Moeller e...@wikimedia.org 2009/7/20 Liam Wyatt liamwy...@gmail.com: Rotating them would seem like a more viable solution than randomised - We don't want the situation where every new page in WP someone reads there is a new/different coloured donation button where last week there was none at all - to go from nothing to that would be almost as bad as a flashing donate here, now! banner. Indeed, that's the reasoning behind the proposed approach. We don't want it to typically be changing constantly for an individual user. Yes, a sequential run does introduce various problematic biases. An IP-address based hack could work, but would need to take into account dynamic IP addresses and such, without introducing strange new biases of its own. We'll discuss a bit further - good ideas / algorithms welcome. :-) -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
On Mon, Jul 20, 2009 at 11:28 PM, Gregory Maxwellgmaxw...@gmail.com wrote: On Tue, Jul 21, 2009 at 2:15 AM, effe iets anderseffeietsand...@gmail.com wrote: What about changing it every hour? Then you have sufficient randomness over time, and no flashy buttons. You still have the problem that peoples response to future buttons will depend on the past buttons they've seen. Yeah, I'd suggest it is better to show each person one button type and stick with it. One can approximate that behavior either by assigning each user randomly and tracking their type via a cookie or by selecting based on higher order bytes in the IP address and assuming those are fairly stable (e.g. using BBB mod 4 where the address is AAA.BBB.CCC.DDD). Neither approach is foolproof or totally without bias, but one can do fairly well. -Robert Rohde ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
Donate Now Every donation helps us to keep free for everyone. Donate Now Keep Wikipedia free for everyone. Is no one else concerned by the use of the word free in the message options being tested. I wouldn't want these ambigous messages like these on the site no matter if they beat out the no message option by 10 to 1. Why can't we test messages that are actually clear and honest? Wikipedia will still be free for everyone if not a single further donation is ever made. Birgitte SB ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
on 7/21/09 10:33 AM, Birgitte SB at birgitte...@yahoo.com wrote: Donate Now Every donation helps us to keep free for everyone. Donate Now Keep Wikipedia free for everyone. Is no one else concerned by the use of the word free in the message options being tested. I wouldn't want these ambigous messages like these on the site no matter if they beat out the no message option by 10 to 1. Why can't we test messages that are actually clear and honest? Wikipedia will still be free for everyone if not a single further donation is ever made. Birgitte SB I agree with you very much, Birgitte. Both of these messages sound like threats. Marc Riddell ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
I agree with this, and said so at the original discussion--where I think the consensus was not to use that phrase. David Goodman, Ph.D, M.L.S. http://en.wikipedia.org/wiki/User_talk:DGG On Tue, Jul 21, 2009 at 10:39 AM, Marc Riddellmichaeldavi...@comcast.net wrote: on 7/21/09 10:33 AM, Birgitte SB at birgitte...@yahoo.com wrote: Donate Now Every donation helps us to keep free for everyone. Donate Now Keep Wikipedia free for everyone. Is no one else concerned by the use of the word free in the message options being tested. I wouldn't want these ambigous messages like these on the site no matter if they beat out the no message option by 10 to 1. Why can't we test messages that are actually clear and honest? Wikipedia will still be free for everyone if not a single further donation is ever made. Birgitte SB I agree with you very much, Birgitte. Both of these messages sound like threats. Marc Riddell ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
[Foundation-l] Donation Button Enhancement : Part 2
Wikimedians-- As many of you know, last month we began work on exploring the visibility of the donate button on all Wikimedia projects. After a long comment period, we received many comments and many new ideas. Some of these ideas we have incorporated into a new set of test buttons. Thank you to everyone who took the time to evaluate Round 1 buttons. You can see those discussions here: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade/Round1 We have 4 designs that we will be testing on the Wikipedia:EN main skin during August and the first part of September. We are going to evaluate each button for one full week. This process will unfold over the next two months. You can see the designs and timeline at this link: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade -Rand -- Rand Montoya Head of Community Giving Wikimedia Foundation www.wikimedia.org Email: r...@wikimedia.org Phone: 415.839.6885 x615 Fax: 415.882.0495 Cell: 510.685.7030 “At some future time, I hope to have something witty, intelligent, or funny in this space.” ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
On Mon, Jul 20, 2009 at 4:50 PM, Rand Montoyarmont...@wikimedia.org wrote: Wikimedians-- As many of you know, last month we began work on exploring the visibility of the donate button on all Wikimedia projects. After a long comment period, we received many comments and many new ideas. Some of these ideas we have incorporated into a new set of test buttons. Thank you to everyone who took the time to evaluate Round 1 buttons. You can see those discussions here: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade/Round1 We have 4 designs that we will be testing on the Wikipedia:EN main skin during August and the first part of September. We are going to evaluate each button for one full week. This process will unfold over the next two months. You can see the designs and timeline at this link: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade Testing should be done in parallel, not in sequence. History has demonstrated that donors have a tendency to respond disproportionately to the new thing. Which means that whatever button you test first will have an advantage over whichever one you test last. Probably the easiest way to get a reasonable distribution is to vary which button people see based on their IP. -Robert Rohde ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
On Mon, Jul 20, 2009 at 6:04 PM, Robert Rohde raro...@gmail.com wrote: On Mon, Jul 20, 2009 at 4:50 PM, Rand Montoyarmont...@wikimedia.org wrote: Wikimedians-- As many of you know, last month we began work on exploring the visibility of the donate button on all Wikimedia projects. After a long comment period, we received many comments and many new ideas. Some of these ideas we have incorporated into a new set of test buttons. Thank you to everyone who took the time to evaluate Round 1 buttons. You can see those discussions here: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade/Round1 We have 4 designs that we will be testing on the Wikipedia:EN main skin during August and the first part of September. We are going to evaluate each button for one full week. This process will unfold over the next two months. You can see the designs and timeline at this link: http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade Testing should be done in parallel, not in sequence. History has demonstrated that donors have a tendency to respond disproportionately to the new thing. Which means that whatever button you test first will have an advantage over whichever one you test last. Probably the easiest way to get a reasonable distribution is to vary which button people see based on their IP. -Robert Rohde It's also necessary to control for seasonal traffic (and thus donation) variations. I note that the first three button tests are at the end of summer while the fourth coincides with the beginning of the school year. It could be the case that there is no variation, or that the variation is highly significant. Since nobody has looked there is no way to tell if the test results are valid. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
2009/7/21 Robert Rohde raro...@gmail.com: Testing should be done in parallel, not in sequence. History has demonstrated that donors have a tendency to respond disproportionately to the new thing. Which means that whatever button you test first will have an advantage over whichever one you test last. Probably the easiest way to get a reasonable distribution is to vary which button people see based on their IP. Or simply to randomise it entirely. If either of those aren't possible for technical reasons, it might be practical to rotate them - run each button for x many hours at a stretch, rotating them so as to ensure they don't regularly go up at the same time (of the day or of the week) and so that they get roughly equal coverage. -- - Andrew Gray andrew.g...@dunelm.org.uk ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
2009/7/20 Liam Wyatt liamwy...@gmail.com: Rotating them would seem like a more viable solution than randomised - We don't want the situation where every new page in WP someone reads there is a new/different coloured donation button where last week there was none at all - to go from nothing to that would be almost as bad as a flashing donate here, now! banner. Indeed, that's the reasoning behind the proposed approach. We don't want it to typically be changing constantly for an individual user. Yes, a sequential run does introduce various problematic biases. An IP-address based hack could work, but would need to take into account dynamic IP addresses and such, without introducing strange new biases of its own. We'll discuss a bit further - good ideas / algorithms welcome. :-) -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
On Mon, Jul 20, 2009 at 9:19 PM, Erik Moellere...@wikimedia.org wrote: Indeed, that's the reasoning behind the proposed approach. We don't want it to typically be changing constantly for an individual user. Yes, a sequential run does introduce various problematic biases. An IP-address based hack could work, but would need to take into account dynamic IP addresses and such, without introducing strange new biases of its own. We'll discuss a bit further - good ideas / algorithms welcome. :-) For this the normal procedure is to give users a session cookie of some kind (either one handed out by the server or one just generated on the client) and base the selection on that. For caching reasons I suppose you'd just want to do this all client side. Should work fine. Alternatively, someone rigs up the front end caches to do this substitution based on IP at serving time. This would be non-trivial with squid. It would be much easier with varnish, alas. In any case, I strongly agree with the argument against running them sequentially. Not only do you get the uncertainty from changing habits over time but later buttons will suffer from the influence of prior ones. Whatever can be done to avoid sequential testing should be done. ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
Re: [Foundation-l] Donation Button Enhancement : Part 2
On Tue, Jul 21, 2009 at 12:30 PM, Gregory Maxwell gmaxw...@gmail.comwrote: On Mon, Jul 20, 2009 at 9:19 PM, Erik Moellere...@wikimedia.org wrote: Indeed, that's the reasoning behind the proposed approach. We don't want it to typically be changing constantly for an individual user. Yes, a sequential run does introduce various problematic biases. An IP-address based hack could work, but would need to take into account dynamic IP addresses and such, without introducing strange new biases of its own. We'll discuss a bit further - good ideas / algorithms welcome. :-) For this the normal procedure is to give users a session cookie of some kind (either one handed out by the server or one just generated on the client) and base the selection on that. For caching reasons I suppose you'd just want to do this all client side. Should work fine. Alternatively, someone rigs up the front end caches to do this substitution based on IP at serving time. This would be non-trivial with squid. It would be much easier with varnish, alas. In any case, I strongly agree with the argument against running them sequentially. Not only do you get the uncertainty from changing habits over time but later buttons will suffer from the influence of prior ones. Whatever can be done to avoid sequential testing should be done. Didn't we do this kind of trial during the last fundraiser - with the messages at the top? They were rotated each new day weren't they? In any case, whatever we worked out for last time can't we use that method again? wittylama.com/blog Peace, love metadata Sent from Sydney, Nsw, Australia ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l ___ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
