I think an important thing for the new definition is that is easy to update, as any definition we come up with will need updating and maintenance.
> On Feb 6, 2015, at 2:56 PM, Oliver Keyes <[email protected]> wrote: > > *nods glumly*. Every day we turn over a rock and find a new ants nest. > I'm waiting for the day, probably about 30 seconds after we declare it > The Definitive Pageviews Definition, when we start finding flaws in > the new one ;p. > > I feel like we should append to the end of the current Rules of > Analytics something like "Nothing will make you more cynical about a > class of metrics than trying to implement them" > >> On 6 February 2015 at 17:22, Erik Zachte <[email protected]> wrote: >> Yes, this has been an issue before. Squid log based reports filter these >> banners for years, but only after a similar distortion became very apparent, >> and a lot of data needed to be repaired. >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Oliver Keyes >> Sent: Friday, February 06, 2015 22:04 >> To: A mailing list for the Analytics Team at WMF and everybody who has an >> interest in Wikipedia and analytics. >> Subject: [Analytics] Drop in Commons mobile traffic - a diagnosis >> >> Hey all, >> >> The pageviews stored at stats.wikimedia.org and the Vital Signs dashboards >> showed a substantial drop in pageviews to Wikimedia Commons, primarily from >> mobile, beginning on 1 January 2015. I was tasked with investigating and I'm >> reporting what I found so that we have a note of the problems this brings up. >> >> From an investigation of requests to that site at that time, it appears that >> this is a perfect storm of known deficiencies in the legacy pageviews >> definition, fundraising changes, and mobile changes. >> To summarise: >> >> 1. The legacy Pageviews definition contains Special pages, including >> Special:BannerRandom and Special:HideBanner; 2. The mobile website was >> historically loading things from Commons in such a way as to trigger calls >> to Special:HideBanner, which were picked up by the legacy definition as >> "pageviews to commons"; 3. The Mobile team deployed changes to their image >> loading setup at the end of December that stopped this from happening, and >> that coincided with the disabling of the Fundraising primary campaign. >> 4. The result of this was an apparent massive drop in traffic to Commons >> from the mobile site - when the actual inaccuracy was the inclusion of that >> traffic in the first place. >> >> There are several lessons to be learned from this. First, it is worth >> reiterating the deficiencies and inaccuracies inherent in the legacy >> pageview definition, many (but certainly not all) of which centre on how it >> treats the fundraising banners. We are working as rapidly as we can to >> completely deprecate this definition, replacing it with a new one which is >> not subject to this kind of variation. We are currently in the middle of >> performing final QA testing on the new definition: >> once it is satisfactory, we will deploy it as soon as humanly possible and >> deprecate the legacy definition. >> >> Second, let me emphasise how critical it is that the teams building >> MediaWiki and our instances of it - Platform, Operations, Mobile, you name >> it - keep us in the loop about changes that they make. This was a very >> dramatic shift in client logic around requests: it flew under our radar. We >> should have a process in place for letting Analytics know about these >> changes before they happen so that we do not end up with inaccurate data and >> a constant game of catchup. >> >> Thanks, >> >> -- >> Oliver Keyes >> Research Analyst >> Wikimedia Foundation >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics _______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
