+1 WSC. When I thought about replicating it, I expected to see a dramatic decline in the impact of vandalism with the advent of counter-vandalism tools and abuse filter.
It would be interesting to see that on a cross-wiki basis as different wikis employ different strategies (or seemingly none at all) for counter-vandalism over time. On Tue, Jan 19, 2021 at 10:58 AM WereSpielChequers < [email protected]> wrote: > Hi Aaron, > > That was an interesting read and a bit of a time capsule. 2002-2006 is a > bit before I started editing Wikipedia. Before many of the tools such as > huggle that give vandalfighters such an advantage over vandals, I think > before the era of bot reversion of vandalism when vandalism had to be > reverted by humans rather than computers, and certainly before the edit > filters that prevent much, possibly most vandalism from even being saved. > It also seems to predate the whole panoply of page protection that stops > vandals even editing many common vandalism targets (they do say that every > single article is available for anyone to edit). > > It would be interesting to see a study now when recent changes patrollers > boast of the times they have got to some vandalism faster than Cluebot. > > I know there were predictions in the early years that eventually the tidal > wave of vandalism would overwhelm the defenders of the wiki, that study > seems to have been part of that. I wonder if anyone in 2004 predicted that > we would get to the current situation where adolescent vandalism has turned > out to be so predictable that dealing with it has been mostly automated and > now we are more worried about spam than vandalism. > > WSC > > On Mon, 18 Jan 2021 at 23:52, Aaron Halfaker <[email protected]> > wrote: > > > See page 7 of Priedhorsky, R., Chen, J., Lam, S. T. K., Panciera, K., > > Terveen, L., & Riedl, J. (2007, November). Creating, destroying, and > > restoring value in Wikipedia. In *Proceedings of the 2007 international > ACM > > conference on Supporting group work* (pp. 259-268). > > http://reidster.net/pubs/group282-priedhorsky.pdf > > > > They discuss the probability of a page view of Wikipedia containing > > vandalism rising over time. I wanted to replicate this analysis and > extend > > it past 2007 but I never got the chance. I think the methodology is > really > > interesting though. > > > > It doesn't directly answer the question but it does get at the *impact* > of > > vandalism. > > > > On Mon, Jan 18, 2021 at 12:13 PM Isaac Johnson <[email protected]> > > wrote: > > > > > To WSC's point about the difficulty of detecting such behavior or > > surveying > > > at a point in which it would still be salient, I'd add that in general > we > > > have a large gap in our knowledge about why people choose to stop > editing > > > because almost all of our survey mechanisms depend on existing > logged-in > > > usage of the wikis. This is a challenge with many other websites too > but > > > it's generally easier to find and survey who, for instance, has left > > > Facebook (example > > > < > > > > > > http://socialmedia.soc.northwestern.edu/wp-content/uploads/2013/05/CHI2013-FBLL.pdf > > > >) > > > by collecting a random sample of people than it is to find and survey > > > someone who was a former editor of Wikipedia. There were surveys that > did > > > ask about major barriers to editing (which presumably contribute to > > > burnout) such as the 2012 survey: > > > > > > > > > https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf#page=17 > > > (see the editor survey category > > > <https://meta.wikimedia.org/wiki/Category:Editor_surveys> if you're > > > looking > > > for others) > > > > > > Some things that come to mind though: > > > > > > - I suspect very few readers see vandalism in their daily browsing > > (as a > > > very frequent, long-term reader of English Wikipedia, I have trouble > > > recalling encountering any clear vandalism in the course of normal > > > reading). That said, I do suspect that most people have seen plenty > of > > > stories of outlandish vandalism to Wikipedia -- some legitimate but > > many > > > more about vandalism that literally lasted minutes -- that may lead > to > > > lower trust. Whether or not lower trust in Wikipedia leads to lower > > > readership is a separate question though. Jonathan Morgan ran some > > > recent > > > surveys on reader trust and what factors affected it that might be > > > relevant: > > > > > > > > > https://meta.wikimedia.org/wiki/Research:The_role_of_citations_in_how_readers_evaluate_Wikipedia_articles#Second_round_survey > > > - Specifically in the context of harassment and gender equity: > > > - Harassment as barrier: > > > > > > > > > https://meta.wikimedia.org/wiki/Gender_equity_report_2018/Barriers_to_equity > > > - Edit summaries in particular as harassment: > > > https://www.elizabethwhittaker.net/wmf-internship (more details > > > < > > > https://www.mediawiki.org/wiki/Wikimedia_Research/Showcase#July_2019> > > > ) > > > - Annual Community Insights Reports often have a section on this > -- > > > e.g., > > > > > > > > > https://meta.wikimedia.org/wiki/Community_Insights/Community_Insights_2020_Report/Thriving_Movement#Safe_and_Secure_Spaces > > > - 2015 Harassment Survey: > > > https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015 > > > - The body of work around barriers to newcomers might have some good > > > insights too -- e.g., > > > > > > > > > https://www-users.cs.umn.edu/~halfaker/publications/The_Rise_and_Decline/ > > > > > > > > > On Sun, Jan 17, 2021 at 5:44 AM WereSpielChequers < > > > [email protected]> wrote: > > > > > > > Hi Amir, > > > > > > > > This is one of those areas of research where we really need the > annual > > > > editor survey. I think it ran once after the 2009/10 Strategy > process, > > > and > > > > I don't know if the best questions got included. > > > > > > > > But the best time to ask editors what prompted them to start > editing > > > has > > > > to be fairly soon after they started as memories fade. I once went > back > > > to > > > > my early edits and the edit I remembered starting me editing barely > > made > > > it > > > > into my first 50. > > > > > > > > There is a longstanding theory that a lot of new editors start or > > started > > > > to fix some vandalism that they saw, and that this group went into > > steep > > > > decline a decade ago with the rise of Cluebot and other antivandalism > > > tools > > > > that work faster than a newbie could. But without an annual survey to > > ask > > > > editors what prompted them to edit you are going to struggle to > > research > > > > this. Of course you could look at the early logged in edits of > > > > active/prolific wikipedians, but if it is true that many/most > > Wikipedians > > > > start with some IP edits, the earliest edits of many Wikipedians > won't > > be > > > > available. > > > > > > > > Abuse one assumes has a differential effect on the targets of abuse, > > > > disproportionately women, gays and ethnic minorities. But I'd be > > inclined > > > > to look at stuff targeted at their user and usertalkpages rather than > > > > talkpages and edit summaries, though an email survey of former > editors > > > > would be useful. > > > > > > > > My suspicion is that when we revert, block and maybe even revdel or > > > > oversight abuse we assume that fixes the problem, and if we want to > > > tackle > > > > abuse we need more edit filters to prevent such abuse from going > live. > > > > > > > > WSC > > > > > > > > On Sat, 16 Jan 2021 at 15:16, Amir E. Aharoni < > > > > [email protected]> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Is there any research about the effect of vandalism in wiki content > > > pages > > > > > on readers, experienced editors, and new and potential editors? > > > > > > > > > > And of abuse in discussion pages and edit summaries on experienced > > > > editors > > > > > and new and potential editors? > > > > > > > > > > Intuitively and anecdotally one could think of the following: > > > > > 1. Vandalism in content pages (articles) wastes editors' and > > > patrollers' > > > > > time. This (probably) doesn't require proof (or does it?). But some > > > > people > > > > > say it also causes some experienced editors to burn out and leave. > Is > > > > there > > > > > any data about it, beyond intuition? > > > > > > > > > > 2. Does vandalism *measurably* affect the perception of the wikis' > > > > > reliability? (This may be wildly different in different languages > and > > > > > wikis.) > > > > > > > > > > 3. Abusive language on discussion pages and edit summaries affects > > > > editors, > > > > > and may cause them to reduce their editing, to stop editing about > > > certain > > > > > topics, or to leave the wiki entirely. Is this effect measurable? > How > > > > does > > > > > it differ for various groups by gender, age, religion, country, > > > > > professional and educational background, seniority at the wiki, > etc.? > > > > > > > > > > Thanks! :) > > > > > > > > > > -- > > > > > Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי > > > > > http://aharoni.wordpress.com > > > > > “We're living in pieces, > > > > > I want to live in peace.” – T. Moore > > > > > _______________________________________________ > > > > > Wiki-research-l mailing list > > > > > [email protected] > > > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > > > > > _______________________________________________ > > > > Wiki-research-l mailing list > > > > [email protected] > > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > > > > > > > > > -- > > > Isaac Johnson (he/him/his) -- Research Scientist -- Wikimedia > Foundation > > > _______________________________________________ > > > Wiki-research-l mailing list > > > [email protected] > > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > > _______________________________________________ > > Wiki-research-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > _______________________________________________ > Wiki-research-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > _______________________________________________ Wiki-research-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
