Hi Magnus -- These are totally reasonable expectations and thanks for the understanding our situation. You can look for more detailed communications from us starting tomorrow.
Thanks, -Toby On Thu, Feb 20, 2014 at 2:51 AM, Magnus Manske <[email protected]>wrote: > Hi Toby, Everyone, > > thanks for this update. I think the approach you describe, boosting > stats.grok.se with hardware support while developing the "official", > scalable WMF solution, is quite sensible in this situation. > > I do understand having just too many things on your plate to deal with, > and I also know the difficulties inherent to dealing with large amounts of > data in a scalable fashion. No one expects you to "press a button" and view > data appears as if by magic; dedicated developer time, steady progress with > the occasional update, support of Henrik's current interim solution, and > maybe some rough time estimates for milestones would be what I (and people > from the GLAM community) would hope for. > > Thanks, > Magnus > > > > On Wed, Feb 19, 2014 at 11:14 PM, Toby Negrin <[email protected]>wrote: > >> Hi Everyone, >> >> Many of you have read Magnus' post on his >> blog<http://magnusmanske.de/wordpress/?p=173>. >> I've commented on his blog and I wanted to repost here and address Magnus' >> concerns directly. >> >> First of all, I'm sorry that we let Magnus and other folks down on the >> page view APIs -- we made some commitments late last year that we weren't >> able to meet. Not only that, these failures echoed previous points of >> frustration with the Foundation. >> >> I do want to note that we actively support the infrastructure that feeds >> data to stats.grok.se. We've fixed a number of issues with that >> pipeline, most recently last week. We understand the importance of this >> data to the community. >> >> The page view API project has been challenging for a number of reasons -- >> the size of the data, the fact that definitions of page views have not been >> updated to stay in line with the changing traffic (mobile, bots, API >> requests, etc) and the challenges in aggregating various aliases. We've >> needed to revisit our definitions of page views in order to get this right >> as well as design and build a global architecture for collecting these and >> other metrics. In addition, we've tried to do this with a perspective of >> privacy and respect for our users. >> >> To this end, we presented an approach to measuring page views in >> MediaWiki at FOSDEM in January and have made progress towards our new >> infrastructure by deploying middleware delivering unsampled page view data >> from mobile devices from our globally distributed datacenters to our >> compute cluster for analysis. >> >> However, these initiatives are complex and will take several months to >> complete at the earliest. In the meantime, we're working with Henrik to >> scale up stats.grok.se. >> >> I also want to call out that the Analytics team has been supporting a >> wide range of users and stakeholders during the year. We've developed >> WikiMetrics, a tool for measuring editor productivity that is used by WMF >> program evaluation and community members; provided dashboards and support >> for Wikipedia Zero, our program to partner with our mobile partners to >> enable mobile Wikipedia access free from data charges; and supported >> product teams and researchers both inside and outside of the foundation. >> >> We've been prioritizing and working on these projects as our resources >> allow and it's important to understand that the team has not been idle. >> While we've done a less than stellar job in communicating our progress to >> the community, information on what we've been doing is available via our >> planning >> pages <https://www.mediawiki.org/wiki/Analytics/Prioritization_Planning> on >> mediawiki. In the future, we will be more proactive in communicating with >> the community regarding our goals and projects. >> >> -Toby >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > > > -- > undefined > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
