Thanks for taking the time to write this up, Oliver! Dario, Dan and I are going to work on how we might use Scrum of Scrums to get changes like this on the radar and find a way to communicate the impact of these changes when they happen under the radar.
On Fri, Feb 6, 2015 at 1:04 PM, Oliver Keyes <[email protected]> wrote: > Hey all, > > The pageviews stored at stats.wikimedia.org and the Vital Signs > dashboards showed a substantial drop in pageviews to Wikimedia > Commons, primarily from mobile, beginning on 1 January 2015. I was > tasked with investigating and I'm reporting what I found so that we > have a note of the problems this brings up. > > From an investigation of requests to that site at that time, it > appears that this is a perfect storm of known deficiencies in the > legacy pageviews definition, fundraising changes, and mobile changes. > To summarise: > > 1. The legacy Pageviews definition contains Special pages, including > Special:BannerRandom and Special:HideBanner; > 2. The mobile website was historically loading things from Commons in > such a way as to trigger calls to Special:HideBanner, which were > picked up by the legacy definition as "pageviews to commons"; > 3. The Mobile team deployed changes to their image loading setup at > the end of December that stopped this from happening, and that > coincided with the disabling of the Fundraising primary campaign. > 4. The result of this was an apparent massive drop in traffic to > Commons from the mobile site - when the actual inaccuracy was the > inclusion of that traffic in the first place. > > There are several lessons to be learned from this. First, it is worth > reiterating the deficiencies and inaccuracies inherent in the legacy > pageview definition, many (but certainly not all) of which centre on > how it treats the fundraising banners. We are working as rapidly as we > can to completely deprecate this definition, replacing it with a new > one which is not subject to this kind of variation. We are currently > in the middle of performing final QA testing on the new definition: > once it is satisfactory, we will deploy it as soon as humanly possible > and deprecate the legacy definition. > > Second, let me emphasise how critical it is that the teams building > MediaWiki and our instances of it - Platform, Operations, Mobile, you > name it - keep us in the loop about changes that they make. This was a > very dramatic shift in client logic around requests: it flew under our > radar. We should have a process in place for letting Analytics know > about these changes before they happen so that we do not end up with > inaccurate data and a constant game of catchup. > > Thanks, > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
