Addendum: run same queries for desktop and that bumps up percentage to 0.052% for fedora on desktop only pageviews on December 2016.
Thanks, Nuria On Thu, Mar 16, 2017 at 1:11 PM, Nuria Ruiz <[email protected]> wrote: > >Hmm, does not make sense to me that the traffic caused by our users > would be that small, > Overall? I disagree, I think it does, think that wikipedia (our main > source of traffic for all wikimedia projects) is fastly moving to mobile, > thus mobile OS are the bulk of the requests, desktop are the minority and, > in that minority, Linux is the minority. > > Just looked at December 2016 overall pageviews for desktop and mobile > coming from "users" (not self-identified-bots) and for that month about 20% > of pageviews are on iOS, 25% are on Android and Fedora is 0.027%. This > data is counting all projects for the whole world at large, probably Fedora > represents a larger chuck of traffic in US-desktop only traffic. > > I think we are going to be adding a bit more info to our browser reports > with desktop-only data but still, Fedora traffic is probably not going to > display. > > >Anyway, I will install the analytics stuff myself on a local machine and > do some testing, to see if I > >can see a reason for things to fail register properly. > > If you end up committing any fix to ua-parser please let us know > > > > > > > > > > > > > On Thu, Mar 16, 2017 at 9:28 AM, Christian Schaller <[email protected]> > wrote: > >> Hmm, does not make sense to me that the traffic caused by our users would >> be that small, >> and there is no version string for Fedora in the user agent, it is just: >> Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like >> Gecko) Chrome/56.0.2924.87 Safari/537.36 >> >> Anyway, I will install the analytics stuff myself on a local machine and >> do some testing, to see if I >> can see a reason for things to fail register properly. Thanks for the >> quick and helpful answers so far. >> >> Christian >> >> >> >> ----- Original Message ----- >> > From: "Nuria Ruiz" <[email protected]> >> > To: "A mailing list for the Analytics Team at WMF and everybody who has >> an interest in Wikipedia and analytics." >> > <[email protected]> >> > Cc: "Christian Schaller" <[email protected]>, "Tomas Popela" < >> [email protected]> >> > Sent: Thursday, March 16, 2017 12:12:28 PM >> > Subject: Re: [Analytics] Os stats >> > >> > Small correction, threshold of browser reporting is 0.05%: >> > https://github.com/wikimedia/analytics-refinery/blob/master/ >> oozie/browser/general/coordinator.properties#L62 >> > Even for our traffic below that number reporting is really not that >> > meaningful. Now because the way that grouping happens if 'Fedora 23' and >> > 'Fedora 24' (imaginary versions) have 0.025% traffic neither will get >> > reported. This is something we would like to improve and we have a >> ticket >> > for it here: https://phabricator.wikimedia.org/T131127 (feel free to >> chime >> > in) >> > >> > Now, even with big traffic like ours there is a threshold below which >> > reporting data is not meaningful as numbers in some instances oscillate >> a >> > lot and that means that there is more noise than signal, we will try to >> get >> > an specific "desktop" tab (so only requests to desktop site are counted) >> > but even then, Fedora traffic might be too small to display. >> > >> > On Thu, Mar 16, 2017 at 6:09 AM, Dan Andreescu < >> [email protected]> >> > wrote: >> > >> > > The threshold is actually at 0.1%, though you are right that this is >> > > fairly arbitrary. We have sanitizing data on our goals next quarter, >> and >> > > that's when we'll take a more mathematical approach at the problem. >> > > >> > > Original Message >> > > From: Christian Schaller >> > > Sent: Thursday, March 16, 2017 08:44 >> > > To: Dan Andreescu >> > > Cc: A mailing list for the Analytics Team at WMF and everybody who >> has an >> > > interest in Wikipedia and analytics.; Tomas Popela >> > > Subject: Re: [Analytics] Os stats >> > > >> > > Been thinking a bit about this and while I do appreciate the privacy >> > > concerns I would assume that >> > > even if you set the threshold to 0.5% the amount of traffic on >> Wikipedia >> > > would still be great enough >> > > for that to not be a real privacy risk? It is just that wikimedia is >> one >> > > of the few open sources with >> > > a huge traffic base for this kind of information and we would love to >> use >> > > it as a neutral way to track >> > > our own userbase growth in comparison with the wider market. So we >> know >> > > from our internal statistics that we >> > > more than doubled our userbase over the last year, but having a >> resource >> > > like wikimedia would allow us to see >> > > how those numbers play out in the bigger picture. So any chance of >> > > convincing you to lower the threshold >> > > to 0.5% to hopefully allow us to start using the statistics already >> now? >> > > >> > > Sincerely, >> > > Christian F.K. Schaller >> > > Manager for Fedora & Red Hat Desktop efforts >> > > >> > > >> > > >> > > ----- Original Message ----- >> > > > From: "Dan Andreescu" <[email protected]> >> > > > To: "A mailing list for the Analytics Team at WMF and everybody who >> has >> > > an interest in Wikipedia and analytics." >> > > > <[email protected]> >> > > > Cc: "Christian Schaller" <[email protected]>, "Tomas Popela" < >> > > [email protected]> >> > > > Sent: Tuesday, March 14, 2017 2:10:38 PM >> > > > Subject: Re: [Analytics] Os stats >> > > > >> > > > Christian, >> > > > >> > > > I wanted to make sure our code is working well so I took a look. We >> use >> > > UA >> > > > Parser, a regex-based community-maintained user agent identifier. It >> > > > correctly identified Fedora as the OS in all of the strings I found >> like >> > > > '%Fedora%' for the hour of raw webrequests I looked at. However, >> there >> > > > were less than 0.1% requests that were identified as Fedora. We cut >> off >> > > > reporting statistics when numbers get that low for privacy reasons. >> But >> > > > everything is detected correctly, so if Fedora's share of requests >> > > > increases, it will show up on the charts. >> > > > >> > > > Hope this helps. >> > > > >> > > > On Tue, Mar 14, 2017 at 1:51 PM, Erik Zachte <[email protected] >> > >> > > wrote: >> > > > >> > > > > Hi Christian, >> > > > > >> > > > > I'm forwarding your question to the WMF Analytics Team who >> authored >> > > this >> > > > > report. >> > > > > >> > > > > Cheers, >> > > > > Erik >> > > > > >> > > > > -----Original Message----- >> > > > > From: Christian Schaller [mailto:[email protected]] >> > > > > Sent: Monday, March 13, 2017 16:07 >> > > > > To: Erik Zachte >> > > > > Cc: Tomas Popela >> > > > > Subject: Re: Os stats >> > > > > >> > > > > Hi Erik, >> > > > > Thanks for getting the new OS stats up on: >> > > > > https://analytics.wikimedia.org/dashboards/browsers/#all- >> > > > > sites-by-os/os-family-timeseries >> > > > > >> > > > > That said as far as we can tell the detection of Fedora does not >> work >> > > at >> > > > > all currently and we can not figure out why. Ubuntu which is >> detected >> > > uses >> > > > > the following user agent: >> > > > > Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:52.0) Gecko/20100101 >> > > > > Firefox/52.0 >> > > > > >> > > > > While Fedora which isn't detected uses this user agent: >> > > > > Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:52.0) Gecko/20100101 >> > > > > Firefox/52.0 >> > > > > >> > > > > Would you be so kind to let us know what the wikimedia analytics >> engine >> > > > > uses to try to identify Fedora systems? We can tweak our user >> agents >> > > quite >> > > > > easily if that is easier than updating the analytics engines way >> of >> > > > > detecting Fedora. >> > > > > >> > > > > Sincerely, >> > > > > Christian F.K. Schaller >> > > > > >> > > > > >> > > > > ----- Original Message ----- >> > > > > > From: "Erik Zachte" <[email protected]> >> > > > > > To: "Christian Schaller" <[email protected]> >> > > > > > Sent: Tuesday, October 6, 2015 11:28:55 AM >> > > > > > Subject: RE: Os stats >> > > > > > >> > > > > > Hi Christian, >> > > > > > >> > > > > > Sorry since my previous response we put the reports on hold, as >> there >> > > > > > are issues with reliability now that we migrated https almost >> fully. >> > > > > > >> > > > > > Can you please add your signature to >> > > > > > https://www.mediawiki.org/wiki/Analytics/Wikistats/ >> > > TrafficReports/Futu >> > > > > > re_per_report_B2 I can do it for you, but I don't know: can I >> add >> > > your >> > > > > > full name or do you have a Wikipedia nick name that you prefer >> to >> > > use? >> > > > > > >> > > > > > We are working on migration of the reports. More here: >> > > > > > https://phabricator.wikimedia.org/T114379 >> > > > > > >> > > > > > Cheers, >> > > > > > Erik >> > > > > > >> > > > > > -----Original Message----- >> > > > > > From: Christian Schaller [mailto:[email protected]] >> > > > > > Sent: Tuesday, October 06, 2015 16:16 >> > > > > > To: Erik Zachte >> > > > > > Subject: Re: Os stats >> > > > > > >> > > > > > Hi Erik, >> > > > > > Just checking what the current plans are for the OS statistics >> on the >> > > > > > wikimedia site. As I mentioned in my first email to you, we >> would >> > > love >> > > > > > to use these numbers as a way to estimate how we are doing with >> > > Fedora >> > > > > > Linux as they are one of the few sources for such statistics >> where we >> > > > > > can be fairly sure the data is not biased one way or the other >> (due >> > > to >> > > > > > the huge number of people using wikipedia). Of course with the >> old >> > > > > > stats being discontinued I am know waiting for the new data to >> be >> > > made >> > > > > > available to start building my usage trend statistics :) >> > > > > > >> > > > > > So on the page it says to let us know if we want a specific >> report >> > > > > > kept, so I would like to repeat my wish that there is a version >> of >> > > > > > report '2' kept available. >> > > > > > >> > > > > > Anyway, I realize that maintaining these website statistics is >> a bit >> > > > > > of a sideshow for you guys and not a core part of what your >> doing, so >> > > > > > I just want to say that I do truly appreciate the effort to try >> to >> > > > > > have something at all available. >> > > > > > >> > > > > > Sincerely, >> > > > > > Christian Schaller >> > > > > > >> > > > > > >> > > > > > >> > > > > > ----- Original Message ----- >> > > > > > > From: "Erik Zachte" <[email protected]> >> > > > > > > To: "Christian Schaller" <[email protected]> >> > > > > > > Sent: Monday, June 22, 2015 10:41:40 AM >> > > > > > > Subject: RE: Os stats >> > > > > > > >> > > > > > > Hi Christian, >> > > > > > > >> > > > > > > I started a job to catch-up for the last 3 months, will take >> 4-5 >> > > days. >> > > > > > > >> > > > > > > FYI these reports are almost end-of-life. Expect a complete >> > > overhaul >> > > > > > > of Wikimedia traffic and core metrics reporting based on >> bigger >> > > iron >> > > > > > > and new paradigms (e.g. hadoop) in 2015 Q3/A4. >> > > > > > > >> > > > > > > Cheers, >> > > > > > > Erik >> > > > > > > >> > > > > > > -----Original Message----- >> > > > > > > From: Christian Schaller [mailto:[email protected]] >> > > > > > > Sent: Tuesday, June 16, 2015 16:46 >> > > > > > > To: [email protected] >> > > > > > > Subject: Os stats >> > > > > > > >> > > > > > > Hi Erik, >> > > > > > > Been checking out the stats on >> > > > > > > https://stats.wikimedia.org/wikimedia/squids/ >> > > > > SquidReportOperatingSystems.htm. >> > > > > > > Are you planning on updating that page again soon? >> > > > > > > We are using your numbers as one of the datapoints for >> estimating >> > > > > > > how Fedora Linux is doing, so I hope you plan on pulling new >> > > numbers >> > > > > > > from time to time. >> > > > > > > >> > > > > > > Christian >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > _______________________________________________ >> > > > > Analytics mailing list >> > > > > [email protected] >> > > > > https://lists.wikimedia.org/mailman/listinfo/analytics >> > > > > >> > > > >> > > >> > > _______________________________________________ >> > > Analytics mailing list >> > > [email protected] >> > > https://lists.wikimedia.org/mailman/listinfo/analytics >> > > >> > >> > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
