Thanks for taking an interest, Luis. This is one resource intensive project that I think may be worth the investment. WMF is getting a fair amount of flak for growing its budget at rates far higher than its growth in content contributors. (Just imagine what would happen if a chapter did this.) I think that making Wikipedia content contribution be a more emotionally rewarding experience, perhaps by using positive reinforcement in ways suggested by Kerry, might be helpful in our editor engagement. Are there any chances of starting early work on a project along these lines in Q3, perhaps aligned with the work on Wikiproject X and on improving ENWP mentorship?
Pine On Oct 13, 2015 6:43 PM, "Luis Villa" <[email protected]> wrote: > I think there's a lot to be done there (probably will blog soon about my > weekend experimenting with Genius, which had pretty extensive systems for > this). > > It is an interesting prioritization question: doing it > thoroughly/systematically would require a lot of software investment, > especially since we don't have structured conversation pages (which are the > basis for a lot of similar contributor recognition systems). > > Luis > > On Sun, Oct 11, 2015 at 5:25 AM, Pine W <[email protected]> wrote: > >> Kerry, >> >> Thanks so much for the comments. I will bring up the subjects of badges >> and cobtributor KPIs with Luis and/or Lila when I have a minute to refine >> my thinking. >> >> Pine >> On Oct 6, 2015 2:33 AM, "Kerry Raymond" <[email protected]> wrote: >> >>> Certainly there are a lot of sites with badges that do seem to encourage >>> certain behaviour. On Wikipedia, we have edit count and that seems to >>> generate editcountitis which (when gamed) tends to favour lots of little >>> housekeeping edits over content edits. But one of the things with badges on >>> most sites is that the site assigns the badge. Here on Wikipedia, I can put >>> any badge I want on my User Page (the pre-existing ones are mostly >>> edit-count based but I can roll my own as some users do). Indeed as I >>> discovered, other people can put badges on my user page and presumably take >>> them away. As edit count is our primary KPI, it doesn't address "cultural" >>> attributes. Should we be making more of an effort to promote other KPIs >>> that emphasise positive behaviour like thanks (given and received)? >>> Unfortunately our main interaction mechanism is writing on talk pages and >>> it's hard to tell whether any contribution on a talk page is a "positive" >>> behaviour or a negative one (short of some kind of sentiment analysis). >>> This is an unfortunate consequence of using a wiki for a conversation >>> rather than some more purpose-built tool. >>> >>> In principle one takes a KPI and then creates a badge to reward a >>> behaviour that improves that KPI. But that's all easier said than done. >>> >>> For content improvements, there are probably some things we can do. For >>> example, I presume looking at the edit deltas, we could tell if an edit to >>> an article added a citation (a pair of ref tag in the new version that >>> weren't there in the old version). Adding citations is a desirable >>> behaviour that we could report on and give badges for (although obviously >>> whether or not that citation in any way supports the claim cannot be >>> determined, so the "gaming" of this is to add random citations to offline >>> sources to lots of articles, which cannot be easily verified). In which >>> case maybe we need to give a better score to an online citation on that >>> grounds it is more likely to be verifiable). >>> >>> But positive "culture" or positive social behaviour is harder to detect >>> and reward. For example, we'd like to close the gendergap but firstly we >>> don't have KPI that measures it on an ongoing basis because we don't >>> actually know which contributors are male/female. And even if we had that >>> KPI, what users or their behaviours would we reward for having positive >>> impact on that KPI? In real-life, we might reward a customer who introduces >>> a new customer. Or we might have a "finders fee" for someone who introduces >>> a "new hire". How could we reward introducing new women to Wikipedia or >>> encouraging them (perhaps through mentoring) to contribute more? Or would >>> we reward contributors who contribute to articles about "women's topics" >>> (which is addressing the content gendergap rather than the contributor >>> gendergap, which aren't the same thing although many believe them to be >>> closely linked). [I won't disgress into the challenge of deciding how >>> "female" an article topic is.] >>> >>> On some sites, you need certain badges to "unlock" certain extra >>> functionalities. Are we happy for RfA to be a question of collecting up >>> enough badges? AFAIK, the only auto-implemented badge we have on Wikipedia >>> is the "auto-confirm" (4 days and 10 edits from memory). >>> >>> I think badges are a good idea but I think the way Wikipedia is >>> implemented makes it challenging to machine-identify desirable behaviours >>> to reward (particularly for social/culture metrics). I think badges have >>> (in the most part) to be machine-calculated and awarded or else it just >>> becomes a popularity content (who's mates with who). I know Aaron (or >>> someone) was toying with the idea of putting a value on each edit >>> (presumably based on some training set of edit data that humans rated). I >>> think it's not impossible to come up with some set of dimensions on which >>> an edit might be valued and, using some human evaluations on a test set, >>> come up with some kind of values for each dimension. It might be rough in >>> the first instance but I guess if it incorporated some ongoing feedback >>> mechanism, it could improve over time. >>> >>> A cheap thing that we could do (and I don't think we do) is have edit >>> count badges for "last week", "last month", "last year". ATM we only have >>> "lifetime" counts, which makes it hard for the new user to get any quick >>> positive acknowledgements for their efforts. >>> >>> Kerry >>> >>> -----Original Message----- >>> From: Wiki-research-l [mailto: >>> [email protected]] On Behalf Of Robert West >>> Sent: Tuesday, 6 October 2015 1:05 PM >>> To: Research into Wikimedia content and communities < >>> [email protected]> >>> Cc: Marti Johnson <[email protected]>; Patrick Earley < >>> [email protected]>; Jacob Orlowitz <[email protected]> >>> Subject: Re: [Wiki-research-l] Reinforcing or incentivizing desired user >>> behavior >>> >>> This paper is on using badges to steer user behavior: >>> https://www.cs.cornell.edu/home/kleinber/www13-badges.pdf >>> >>> On Mon, Oct 5, 2015 at 7:42 PM, Pine W <[email protected]> wrote: >>> > Hi all, >>> > >>> > Some of us plan to have a conversation at the WCONUSA unconference >>> > sessions about ENWP culture. Are there any recommended readings that >>> > you could suggest as preparation, particularly on the subject of how >>> > to reinforce or incentivize desirable user behavior? I think that >>> > Jonathan may have done some research on this topic for the Teahouse, >>> > and Ocassi may have for done research for TWA. I'm interested in >>> > applicable research as preparation both for the unconference >>> > discussion and for my planned video series that intends to inform and >>> inspire new editors. >>> > >>> > Thanks, >>> > Pine >>> > >>> > >>> > _______________________________________________ >>> > Wiki-research-l mailing list >>> > [email protected] >>> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>> > >>> >>> >>> >>> -- >>> Up for a little language game? -- http://www.unfun.me >>> >>> _______________________________________________ >>> Wiki-research-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>> >>> >>> _______________________________________________ >>> Wiki-research-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>> >> >> _______________________________________________ >> Wiki-research-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> > > > -- > Luis Villa > Sr. Director of Community Engagement > Wikimedia Foundation > *Working towards a world in which every single human being can freely > share in the sum of all knowledge.* > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
_______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
