Kerry, Thanks so much for the comments. I will bring up the subjects of badges and cobtributor KPIs with Luis and/or Lila when I have a minute to refine my thinking.
Pine On Oct 6, 2015 2:33 AM, "Kerry Raymond" <[email protected]> wrote: > Certainly there are a lot of sites with badges that do seem to encourage > certain behaviour. On Wikipedia, we have edit count and that seems to > generate editcountitis which (when gamed) tends to favour lots of little > housekeeping edits over content edits. But one of the things with badges on > most sites is that the site assigns the badge. Here on Wikipedia, I can put > any badge I want on my User Page (the pre-existing ones are mostly > edit-count based but I can roll my own as some users do). Indeed as I > discovered, other people can put badges on my user page and presumably take > them away. As edit count is our primary KPI, it doesn't address "cultural" > attributes. Should we be making more of an effort to promote other KPIs > that emphasise positive behaviour like thanks (given and received)? > Unfortunately our main interaction mechanism is writing on talk pages and > it's hard to tell whether any contribution on a talk page is a "positive" > behaviour or a negative one (short of some kind of sentiment analysis). > This is an unfortunate consequence of using a wiki for a conversation > rather than some more purpose-built tool. > > In principle one takes a KPI and then creates a badge to reward a > behaviour that improves that KPI. But that's all easier said than done. > > For content improvements, there are probably some things we can do. For > example, I presume looking at the edit deltas, we could tell if an edit to > an article added a citation (a pair of ref tag in the new version that > weren't there in the old version). Adding citations is a desirable > behaviour that we could report on and give badges for (although obviously > whether or not that citation in any way supports the claim cannot be > determined, so the "gaming" of this is to add random citations to offline > sources to lots of articles, which cannot be easily verified). In which > case maybe we need to give a better score to an online citation on that > grounds it is more likely to be verifiable). > > But positive "culture" or positive social behaviour is harder to detect > and reward. For example, we'd like to close the gendergap but firstly we > don't have KPI that measures it on an ongoing basis because we don't > actually know which contributors are male/female. And even if we had that > KPI, what users or their behaviours would we reward for having positive > impact on that KPI? In real-life, we might reward a customer who introduces > a new customer. Or we might have a "finders fee" for someone who introduces > a "new hire". How could we reward introducing new women to Wikipedia or > encouraging them (perhaps through mentoring) to contribute more? Or would > we reward contributors who contribute to articles about "women's topics" > (which is addressing the content gendergap rather than the contributor > gendergap, which aren't the same thing although many believe them to be > closely linked). [I won't disgress into the challenge of deciding how > "female" an article topic is.] > > On some sites, you need certain badges to "unlock" certain extra > functionalities. Are we happy for RfA to be a question of collecting up > enough badges? AFAIK, the only auto-implemented badge we have on Wikipedia > is the "auto-confirm" (4 days and 10 edits from memory). > > I think badges are a good idea but I think the way Wikipedia is > implemented makes it challenging to machine-identify desirable behaviours > to reward (particularly for social/culture metrics). I think badges have > (in the most part) to be machine-calculated and awarded or else it just > becomes a popularity content (who's mates with who). I know Aaron (or > someone) was toying with the idea of putting a value on each edit > (presumably based on some training set of edit data that humans rated). I > think it's not impossible to come up with some set of dimensions on which > an edit might be valued and, using some human evaluations on a test set, > come up with some kind of values for each dimension. It might be rough in > the first instance but I guess if it incorporated some ongoing feedback > mechanism, it could improve over time. > > A cheap thing that we could do (and I don't think we do) is have edit > count badges for "last week", "last month", "last year". ATM we only have > "lifetime" counts, which makes it hard for the new user to get any quick > positive acknowledgements for their efforts. > > Kerry > > -----Original Message----- > From: Wiki-research-l [mailto:[email protected]] > On Behalf Of Robert West > Sent: Tuesday, 6 October 2015 1:05 PM > To: Research into Wikimedia content and communities < > [email protected]> > Cc: Marti Johnson <[email protected]>; Patrick Earley < > [email protected]>; Jacob Orlowitz <[email protected]> > Subject: Re: [Wiki-research-l] Reinforcing or incentivizing desired user > behavior > > This paper is on using badges to steer user behavior: > https://www.cs.cornell.edu/home/kleinber/www13-badges.pdf > > On Mon, Oct 5, 2015 at 7:42 PM, Pine W <[email protected]> wrote: > > Hi all, > > > > Some of us plan to have a conversation at the WCONUSA unconference > > sessions about ENWP culture. Are there any recommended readings that > > you could suggest as preparation, particularly on the subject of how > > to reinforce or incentivize desirable user behavior? I think that > > Jonathan may have done some research on this topic for the Teahouse, > > and Ocassi may have for done research for TWA. I'm interested in > > applicable research as preparation both for the unconference > > discussion and for my planned video series that intends to inform and > inspire new editors. > > > > Thanks, > > Pine > > > > > > _______________________________________________ > > Wiki-research-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > > -- > Up for a little language game? -- http://www.unfun.me > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >
_______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
