[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE Do we have any additional requirements here or shall we resolve the ticket? TASK DETAIL https://phabricator.wikimedia.org/T220977 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic Cc:

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

Milimetric added a comment. To clear up what Joseph said, we're never going to have more than 90 days of geolocated edits for privacy reasons. We do have two aggregated datasets that go back more than a year:

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @JAllemandou Thanks for feedback! @Lea_WMDE Given the current situation with the geo-localized edits (see T220977#5186818 ), do you want me to proceed with the per continent analysis for pageviews now,

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

Lea_WMDE added a comment. @GoranSMilovanovic great, thanks! We definitely know more now :) TASK DETAIL https://phabricator.wikimedia.org/T220977 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, Lea_WMDE Cc: JAllemandou,

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE Here we go: - the following chart shows mobile edits vs. mobile pageviews separately for users and spiders; - what we can learn from this chart is that **the growth is certainly natural**, given that the spiders have made a minimal number

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

Lea_WMDE added a comment. @GoranSMilovanovic that's great! If we compare the percentage increases (I don't know how this is done best statistically, but I'm sure you do :) ), do we get a similar trend between mobile edits and mobile page views? And is it possible for us to roughly group

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE Yes, we do have a more or less steady increase in mobile edits on Wikidata: F29055561: MobileEdits_2019.png TASK DETAIL https://phabricator.wikimedia.org/T220977 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE Here's what was happening with the mobile edits since the beginning of the year. Note: the last data point is May 2019, it's incomplete of course. F29055381: MobileEdits_2019.png I've run this

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

Lea_WMDE added a comment. Thanks, @JAllemandou! TASK DETAIL https://phabricator.wikimedia.org/T220977 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, Lea_WMDE Cc: JAllemandou, Milimetric, RazShuty, Lea_WMDE, Aklapper,

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @JAllemandou You're the man, I see now that `revision_tags` is a new field since the `2019-04` (April 2019) snapshot of `mediawiki_history`: >

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

JAllemandou added a comment. Hi @Lea_WMDE and @GoranSMilovanovic - I think the answer the your problem is solved in this month snapshot with the `revision_tags` field of mediawiki_history: spark.sql(""" SELECT substr(event_timestamp, 0, 4) as year,

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE from Analytics/Data Lake/Edits Wikitech documentation page: > When we import, we grab all the data available from all tables except the revision table, for which we filter by

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE @RazShuty Another possibility would be to parse the X-Analytics field of the `wmf.webrequest` table and look into the values of the `mf-m` key: If set, then the value b indicates that

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE Unfortunately, our edits data currently do not encompass any fields that would allow us to separate edits made from mobile vs. desktop. TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE > If the growth is natural, would it be a valid assumption to assume that the editing behavior for mobile also increased? Inspecting this now. TASK DETAIL https://phabricator.wikimedia.org/T220977 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

Lea_WMDE added a comment. @GoranSMilovanovic thanks for all the insights! One thing that came to my mind is the following: If the growth is natural, would it be a valid assumption to assume that the editing behavior for mobile also increased? I don't think I can find out through the stats

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE @RazShuty I have inspected all mobile pageviews of Wikidata for April 2019. April 2019 is quite representative for the phenomenon that we are investigating: since the sudden increase in Wikidata mobile pageviews, only in January 2019

[Wikidata-bugs] [Maniphest] [Commented On] T220977: Investigate surprising rise in mobile page views for wikidata

GoranSMilovanovic added a comment. @Lea_WMDE @RazShuty It's definitely not Googlebot (Smartphone), I've checked the `wmf.webrequest` for a sample: # - wmf.webrequest dataset: parse user_agent df = sqlContext.sql('SELECT year, month, day, hour, user_agent, agent_type,