Hoi,
Ask yourself, what is it that you get with a more "scientific" approach. Is
it commitment and involvement and who gets involved when science decides
who to select as a special case?
My point is very much that arguments like this forget what it is we want to
achieve. A barnstar is from me (my involvement) to someone else (my
appreciation). I do not care for scientific when it follows that my
involvement is not welcome.
Thanks,
GerardM
On 19 October 2015 at 07:24, Pine W <[email protected]> wrote:
> As much as I like the barnstar system, it's highly subjective and
> inconsistent. I'd like to see a more systematic approach. Perhaps this
> could be combined with some of Aaron's work about edit quality.
>
> On Sat, Oct 17, 2015 at 2:52 AM, WereSpielChequers <
> [email protected]> wrote:
>
>> We have a complex set of "badges", some, as Kerry pointed out are
>> available to everyone who qualifies for them, some are based on the
>> statistics of your account - tenure, edit count, articles created. Others
>> are based on things you've been awarded by others, the bronze stars for
>> featured articles, but also userboxes for everything from userrights to
>> number of DYKs. Barnstars are a key subset that can only be awarded by
>> others. There are Barnstars available for a huge range of things, even
>> civility and diplomacy. It would be interesting and probably salutary to do
>> a study on which Barnstars are awarded, my suspicion is that the anti
>> vandalism ones may well be the most frequent. I would also encourage
>> everyone to lead by example and actually use the Barnstar system for people
>> who have made extraordinary contributions. But be careful not to devalue
>> the system by for example giving one to everyone who reports a bug in
>> visual editor - in the past when we had lots of adolescents and teenagers
>> in the community there was a craze for creating secret pages with a
>> Barnstar award for finding them; so if you give out Barnstars too freely
>> you risk being thought of as the sort of immature adolescent that usually
>> makes that sort of mistake.
>>
>> Regards
>>
>> Jonathan
>>
>>
>> On 14 Oct 2015, at 02:42, Luis Villa <[email protected]> wrote:
>>
>> I think there's a lot to be done there (probably will blog soon about my
>> weekend experimenting with Genius, which had pretty extensive systems for
>> this).
>>
>> It is an interesting prioritization question: doing it
>> thoroughly/systematically would require a lot of software investment,
>> especially since we don't have structured conversation pages (which are the
>> basis for a lot of similar contributor recognition systems).
>>
>> Luis
>>
>> On Sun, Oct 11, 2015 at 5:25 AM, Pine W <[email protected]> wrote:
>>
>>> Kerry,
>>>
>>> Thanks so much for the comments. I will bring up the subjects of badges
>>> and cobtributor KPIs with Luis and/or Lila when I have a minute to refine
>>> my thinking.
>>>
>>> Pine
>>> On Oct 6, 2015 2:33 AM, "Kerry Raymond" <[email protected]> wrote:
>>>
>>>> Certainly there are a lot of sites with badges that do seem to
>>>> encourage certain behaviour. On Wikipedia, we have edit count and that
>>>> seems to generate editcountitis which (when gamed) tends to favour lots of
>>>> little housekeeping edits over content edits. But one of the things with
>>>> badges on most sites is that the site assigns the badge. Here on Wikipedia,
>>>> I can put any badge I want on my User Page (the pre-existing ones are
>>>> mostly edit-count based but I can roll my own as some users do). Indeed as
>>>> I discovered, other people can put badges on my user page and presumably
>>>> take them away. As edit count is our primary KPI, it doesn't address
>>>> "cultural" attributes. Should we be making more of an effort to promote
>>>> other KPIs that emphasise positive behaviour like thanks (given and
>>>> received)? Unfortunately our main interaction mechanism is writing on talk
>>>> pages and it's hard to tell whether any contribution on a talk page is a
>>>> "positive" behaviour or a negative one (short of some kind of sentiment
>>>> analysis). This is an unfortunate consequence of using a wiki for a
>>>> conversation rather than some more purpose-built tool.
>>>>
>>>> In principle one takes a KPI and then creates a badge to reward a
>>>> behaviour that improves that KPI. But that's all easier said than done.
>>>>
>>>> For content improvements, there are probably some things we can do. For
>>>> example, I presume looking at the edit deltas, we could tell if an edit to
>>>> an article added a citation (a pair of ref tag in the new version that
>>>> weren't there in the old version). Adding citations is a desirable
>>>> behaviour that we could report on and give badges for (although obviously
>>>> whether or not that citation in any way supports the claim cannot be
>>>> determined, so the "gaming" of this is to add random citations to offline
>>>> sources to lots of articles, which cannot be easily verified). In which
>>>> case maybe we need to give a better score to an online citation on that
>>>> grounds it is more likely to be verifiable).
>>>>
>>>> But positive "culture" or positive social behaviour is harder to detect
>>>> and reward. For example, we'd like to close the gendergap but firstly we
>>>> don't have KPI that measures it on an ongoing basis because we don't
>>>> actually know which contributors are male/female. And even if we had that
>>>> KPI, what users or their behaviours would we reward for having positive
>>>> impact on that KPI? In real-life, we might reward a customer who introduces
>>>> a new customer. Or we might have a "finders fee" for someone who introduces
>>>> a "new hire". How could we reward introducing new women to Wikipedia or
>>>> encouraging them (perhaps through mentoring) to contribute more? Or would
>>>> we reward contributors who contribute to articles about "women's topics"
>>>> (which is addressing the content gendergap rather than the contributor
>>>> gendergap, which aren't the same thing although many believe them to be
>>>> closely linked). [I won't disgress into the challenge of deciding how
>>>> "female" an article topic is.]
>>>>
>>>> On some sites, you need certain badges to "unlock" certain extra
>>>> functionalities. Are we happy for RfA to be a question of collecting up
>>>> enough badges? AFAIK, the only auto-implemented badge we have on Wikipedia
>>>> is the "auto-confirm" (4 days and 10 edits from memory).
>>>>
>>>> I think badges are a good idea but I think the way Wikipedia is
>>>> implemented makes it challenging to machine-identify desirable behaviours
>>>> to reward (particularly for social/culture metrics). I think badges have
>>>> (in the most part) to be machine-calculated and awarded or else it just
>>>> becomes a popularity content (who's mates with who). I know Aaron (or
>>>> someone) was toying with the idea of putting a value on each edit
>>>> (presumably based on some training set of edit data that humans rated). I
>>>> think it's not impossible to come up with some set of dimensions on which
>>>> an edit might be valued and, using some human evaluations on a test set,
>>>> come up with some kind of values for each dimension. It might be rough in
>>>> the first instance but I guess if it incorporated some ongoing feedback
>>>> mechanism, it could improve over time.
>>>>
>>>> A cheap thing that we could do (and I don't think we do) is have edit
>>>> count badges for "last week", "last month", "last year". ATM we only have
>>>> "lifetime" counts, which makes it hard for the new user to get any quick
>>>> positive acknowledgements for their efforts.
>>>>
>>>> Kerry
>>>>
>>>> -----Original Message-----
>>>> From: Wiki-research-l [mailto:
>>>> [email protected]] On Behalf Of Robert West
>>>> Sent: Tuesday, 6 October 2015 1:05 PM
>>>> To: Research into Wikimedia content and communities <
>>>> [email protected]>
>>>> Cc: Marti Johnson <[email protected]>; Patrick Earley <
>>>> [email protected]>; Jacob Orlowitz <[email protected]>
>>>> Subject: Re: [Wiki-research-l] Reinforcing or incentivizing desired
>>>> user behavior
>>>>
>>>> This paper is on using badges to steer user behavior:
>>>> https://www.cs.cornell.edu/home/kleinber/www13-badges.pdf
>>>>
>>>> On Mon, Oct 5, 2015 at 7:42 PM, Pine W <[email protected]> wrote:
>>>> > Hi all,
>>>> >
>>>> > Some of us plan to have a conversation at the WCONUSA unconference
>>>> > sessions about ENWP culture. Are there any recommended readings that
>>>> > you could suggest as preparation, particularly on the subject of how
>>>> > to reinforce or incentivize desirable user behavior? I think that
>>>> > Jonathan may have done some research on this topic for the Teahouse,
>>>> > and Ocassi may have for done research for TWA. I'm interested in
>>>> > applicable research as preparation both for the unconference
>>>> > discussion and for my planned video series that intends to inform and
>>>> inspire new editors.
>>>> >
>>>> > Thanks,
>>>> > Pine
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Wiki-research-l mailing list
>>>> > [email protected]
>>>> > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Up for a little language game? -- http://www.unfun.me
>>>>
>>>> _______________________________________________
>>>> Wiki-research-l mailing list
>>>> [email protected]
>>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>>
>>>>
>>>> _______________________________________________
>>>> Wiki-research-l mailing list
>>>> [email protected]
>>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>>
>>>
>>> _______________________________________________
>>> Wiki-research-l mailing list
>>> [email protected]
>>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>>
>>>
>>
>>
>> --
>> Luis Villa
>> Sr. Director of Community Engagement
>> Wikimedia Foundation
>> *Working towards a world in which every single human being can freely
>> share in the sum of all knowledge.*
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [email protected]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> [email protected]
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>
> _______________________________________________
> Wiki-research-l mailing list
> [email protected]
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
_______________________________________________
Wiki-research-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
