[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Here are the values for `Tizen` as well, @Manuel: | access_method | total_requests | percent | | - | -- | --- | | desktop | 10874 | 93.7333 | | mobile web| 727| 6.2667 | | I'd assume

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-09-01 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. We'd talked "Tizen" a bit this morning, @Manuel, but let's not focus on it. Did a bit of Wikipedia research <https://en.wikipedia.org/wiki/Tizen> and since since 2021 it's mostly in use in Samsung Smart TVs. That leaves us with `A

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Here are the answers to the three questions we had from the daily, @Manuel: `uri_host LIKE 'wikidata'` as mentioned above: | uri_host | | -- | | test.m.wikidata.org| | query-preview.wikidata.org

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a subscriber: JAllemandou. AndrewTavis_WMDE added a comment. And here are the finalized heuristics (@JAllemandou, tagging you as well). The following query is saved as a temporary view as `df_requests_subset`: SELECT user_agent_map.os_family AS os_family

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, I think we can throw out the idea of creating an edits subset of webrequests, sadly :( The following would be where we'd find the various actions that we'd need to collect to define as edits fully: https://www.wikidata.org/w/api.php. We know

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Here are the tables that break down the `device_family` values, @Manuel :) As before: **Population for the following is**: All requests to `www.wikidata.org` and `m.wikidata.org` inclusively between 25/7/2023 and 31/7/2023 that are further

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

2023-08-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. As far as totals for this task are concerned, @Manuel, what I'm getting is the following: **Population for the following is**: All views of `www.wikidata.org` and `m.wikidata.org` inclusively between 01/7/2023 and 31/7/2023 that are further

[Wikidata-bugs] [Maniphest] T341589: Improve df_to_remarkup formatting for wmfdata-python

2023-10-16 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Thanks, @nshahquinn-wmf! I've made wikimedia/wmfdata-python/pull/46 <https://github.com/wikimedia/wmfdata-python/pull/46> with the discussed changes :) :) TASK DETAIL https://phabricator.wikimedia.org/T341589 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T348240: [EPIC] How are Wikidata editors modifying their Item UI?

2023-10-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T348240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Manuel, me, Danny_Benjafield_WMDE, Astuthiodit_1, BeautifulBold, Suran38

[Wikidata-bugs] [Maniphest] T348240: [EPIC] How are Wikidata editors modifying their Item UI?

2023-10-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. mediawiki_wikitext_current <https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Content/Mediawiki_wikitext_current> can likely be used for the `common.js` and `common.css` files :) TASK DETAIL https://phabricator.wikimedia.org/T348240

[Wikidata-bugs] [Maniphest] T348240: [EPIC] How are Wikidata editors modifying their Item UI?

2023-10-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. wser_properties_table <https://www.mediawiki.org/wiki/Manual:User_properties_table> can be used for preferences. How we access it is still to be figured out. TASK DETAIL https://phabricator.wikimedia.org/T348240 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "active UI editors"

2023-10-17 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T349074 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T349071: [Analytics] Explore Babels

2023-10-17 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T349071 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE edited projects, added Wikidata Analytics (Kanban); removed Wikidata Analytics. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE edited projects, added Wikidata Analytics; removed Wikidata Analytics (Kanban). TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE moved this task from Incoming to Kanban on the Wikidata Analytics board. AndrewTavis_WMDE edited projects, added Wikidata Analytics (Kanban); removed Wikidata Analytics. TASK DETAIL https://phabricator.wikimedia.org/T350157 WORKBOARD https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Will do, @Manuel! TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE edited projects, added Wikidata Analytics (Kanban); removed Wikidata Analytics. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE edited projects, added Wikidata Analytics; removed Wikidata Analytics (Kanban). TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-10-31 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. I'm really not sure why I can't assign both `Wikidata Analytics` and `Wikidata Analytics Kanban`, @Manuel. Did you change the settings? TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-11-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-11-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-11-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-11-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T350157: [Analytics] Extract a representative sample of SPARQL queries from the query logs

2023-11-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T350157 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

2023-11-03 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Just checked in with @Michael on this and we did a check of my ability to run `sudo -u analytics-wmde crontab -l` on `stat1007`. I sadly don't have access :( TASK DETAIL https://phabricator.wikimedia.org/T350252 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T348999: Add linter and formatter to wmfdata-python (and link check)

2023-10-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. We could further make use of pre-commit <https://pre-commit.com/> to check code before commits, which is further something that's suggested in the WMF Airflow docs <https://wikitech.wikimedia.org/wiki/Data_Engineering/Systems/Airflow/Develo

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "UI editors"

2023-10-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. For namespace we're talking about `User:` and similar spaces for projects, discussions, etc etc :) TASK DETAIL https://phabricator.wikimedia.org/T349074 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] T348999: Add linter and formatter to wmfdata-python (and link check)

2023-10-30 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Ruff also just brought out its own formatter <https://github.com/astral-sh/ruff/blob/main/crates/ruff_python_formatter/README.md> that's fully compatible with Black. As far as formatting adoption at WMF, product analytics states here in their styl

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "UI editors"

2023-10-26 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Generally yes, @Manuel :) Working to get all of these columns figured out now  The last thing I'm considering is how we're subsetting by namespace. We have conditions for UI edits that are inherently based on Wikidata related tags with the catch-all being

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "UI editors"

2023-10-26 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Also as far as `device_type` is concerned, @Manuel, we do not have access to a `user_agent` field in wmf.mediawiki_history <https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/MediaWiki_history> where we have the revision tags. As users can

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-09-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. > Also you are explicitly filering for `is_pageview = True` Just a quick note on this, @Manuel: I was told by WMF that for `wmf.pageview_actor` we still need to do a filter for `WHERE is_pageview = True` as the filtering that makes this ta

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-21 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Some general notes on this: as we're working from `wmf.pageview_actor` and `wmf_raw.mediawiki_private_cu_changes`, there might be a way to leverage their expanded `agent_type` field such that for at least the former we have `automated` as an option within

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, re the question of what kind of `referer` values we have for "desktop" requests, the following query was used to get the results below. **Population for the following is**: All requests from `www.wikidata.org` and `m.wi

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Here's the above `referer` breakdown for mobile for reference, with the big difference being that we have dramatically less `-` requests - good for thinking that these are APIs - and have a lot of extension requests: | referer

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-09-25 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, as mentioned on Mattermost there as of now doesn't seem to be a good way of deriving `agent_type` for those tables that don't have it. We can get `spider` through a `UDF`, but `automated` isn't possible at the moment. This makes the final division

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Your suggestion make more sense now, @Michael :) If we're looking for pageviews by users, then we can make the assumption that we don't want `api.php` and `load.php` for `uri_path` as views should come from another page. When considering traffic

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T344052 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, as the plan is to start working on a canonical edits data table that will fit our needs with WMF, do you envision tracking that in this task, or would it make sense to make a new one for that that we can then include all the stakeholders we'd

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-09 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T344052 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-09 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T344052 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-09 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T344052 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-10-06 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, in following @Michael's suggestion for looking into all paths that end in `api.php` and `load.php` we're getting the following: URI Path ends in API or load **Population is**: All requests from

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-10-05 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Summarizing the above, @Manuel: - We're generally able to get a good understanding of mobile users via looking at `user_agent_map.os_family = 'Android'`, `'iOS'` or `'KaiOS'` - If we don't have `user_agent_map` as a column, we can create

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access from mobile vs. desktop devices

2023-10-05 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "active UI editors"

2023-10-19 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, do we want namespace to just be a column? Simplicity wise it might make sense to split all the metrics by namespace and then get aggregates via a group by on user id rather than saving data and the aggregate of data in the same table. TASK DETAIL

[Wikidata-bugs] [Maniphest] T345262: [Analytics] Enable Global North and Global South aggregations

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, edited the task description above with an outline of what I think we could do on this :) TASK DETAIL https://phabricator.wikimedia.org/T345262 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] T345262: [Analytics] Enable Global North and Global South aggregations

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added subscribers: mpopov, mforns. AndrewTavis_WMDE added a comment. @mpopov and @mforns: pinging the both of you as github.com/wikimedia-research/canonical-data <https://github.com/wikimedia-research/canonical-data> is maintained by Product Analytics and Data Engin

[Wikidata-bugs] [Maniphest] T345262: [Analytics] Enable Global North and Global South aggregations

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T345262 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T345262: [Analytics] Enable Global North and Global South aggregations

2023-10-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T345262 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE

[Wikidata-bugs] [Maniphest] T343690: [Analytics] Wikidata edits by UI variant

2023-10-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T343690 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T349074: [Analytics] Operationalize "active UI editors"

2023-10-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Something to also consider here @Manuel is what we like for a timestamp. We have aggregations here, so a granular timestamp would inherently go against the 30 day period. What were you thinking on that? TASK DETAIL https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T348999: wmfdata-python formatting and link check

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. I'd be happy to look into this, @xcollazo! We were discussing linting and checks like this in an open-source project I work on and decided to go with Ruff <https://github.com/astral-sh/ruff>. Would that be worth adopting instead of `flake8`? I coul

[Wikidata-bugs] [Maniphest] T341589: Improve df_to_remarkup formatting for wmfdata-python

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T341589 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, nshahquinn-wmf, AndrewTavis_WMDE, Danny_Benjafield_WMDE

[Wikidata-bugs] [Maniphest] T341589: Improve df_to_remarkup formatting for wmfdata-python

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @nshahquinn-wmf, resolving this as the PR was merged  Thanks for the support here! TASK DETAIL https://phabricator.wikimedia.org/T341589 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc

[Wikidata-bugs] [Maniphest] T348999: wmfdata-python formatting and link check

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Ok, great! I'll check out some ideas for Ruff and maybe get started on this next week :) I'll hold off on the GitHub action as well as I'm remembering now that @nshahquinn-wmf mentioned the move to GitLab in T341589: Improve df_to_remarkup formatting

[Wikidata-bugs] [Maniphest] T348999: wmfdata-python formatting and link check

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. See T349531: Add testing framework to wmfdata-python <https://phabricator.wikimedia.org/T349531>, @xcollazo! Added you and @nshahquinn-wmf as subscribers already :) TASK DETAIL https://phabricator.wikimedia.org/T348999 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T348999: Add linter and formatter to wmfdata-python (and link check)

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE renamed this task from "wmfdata-python formatting and link check" to "Add linter and formatter to wmfdata-python (and link check)". AndrewTavis_WMDE claimed this task. AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedi

[Wikidata-bugs] [Maniphest] T348999: Add linter and formatter to wmfdata-python (and link check)

2023-10-23 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Updated the task and also mentioned maybe adding an auto-formatter that could help to make sure that things are running smoothly before we have workflows set up to test PRs/MRs :) TASK DETAIL https://phabricator.wikimedia.org/T348999 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T344052: [Analytics] Join data from all-access-tables (like webrequest) with edit-tables (like revision)

2023-10-06 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T344052 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel

[Wikidata-bugs] [Maniphest] T345262: [Analytics] Enable Global North and Global South aggregations

2023-11-10 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. A PR has been opened on GitHub <https://github.com/wikimedia-research/canonical-data/pull/10> that sends along the needed changes to the countries table. Once that's merged and the update notebook is ran we'll be good to resolve this as the neede

[Wikidata-bugs] [Maniphest] T332899: [EPIC] Save our R-based Wikidata products

2023-04-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T332899 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Manuel, AndrewTavis_WMDE Cc: Pamputt, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T332899: [EPIC] Save our R-based Wikidata products

2023-04-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T332899 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Manuel, AndrewTavis_WMDE Cc: Pamputt, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T332899: [EPIC] Save our R-based Wikidata products

2023-04-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Added a link to the task description pointing people to the following doc where I've been documenting the code base: https://docs.google.com/document/d/1aYWcStCZySbKTKbHdFzb4n9ZBpBntIbSZsnd99nriuc/edit?usp=sharing TASK DETAIL https

[Wikidata-bugs] [Maniphest] T332899: [EPIC] Save our R-based Wikidata products

2023-04-20 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T332899 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Manuel, AndrewTavis_WMDE Cc: Pamputt, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API

2023-05-02 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, in reference to the following open question: > Do we need more detail than months or not? Who are the stakeholders beyond you that we'd need to check with for the proper gradation? Let's also discuss the following in one of our n

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API

2023-06-12 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael, Manuel

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API

2023-06-12 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael, Manuel

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API

2023-06-12 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael, Manuel

[Wikidata-bugs] [Maniphest] T337245: Wikidata Quarterly Reporting Milestone 2 Aggregations

2023-06-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337245 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Lydia_Pintscher, Aklapper, Manuel, AndrewTavis_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T337245: Wikidata Quarterly Reporting Milestone 2 Aggregations

2023-07-06 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337245 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Lydia_Pintscher, Aklapper, Manuel, AndrewTavis_WMDE, Astuthiodit_1

[Wikidata-bugs] [Maniphest] T337245: Wikidata Quarterly Reporting Milestone 2 Aggregations

2023-07-06 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, the numbers have been updated in the description for the task :) The discrepancy is pretty dramatic, so I'm unsure if you have some info on why that could be or if we can attribute it to an overall adoption trend. Happy to look a bit more

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API

2023-07-06 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: mforns, xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael

[Wikidata-bugs] [Maniphest] T340718: Investigate prior WMDE analytics tables / assets

2023-07-03 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T340718 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, Aklapper, Manuel, AndrewTavis_WMDE, Astuthiodit_1, karapayneWMDE

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: mforns, xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: mforns, xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: mforns, xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: mforns, xcollazo, Ottomata, lbowmaker, WMDE-leszek, AndrewTavis_WMDE, Michael

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-13 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. @Manuel, the task has now been updated with all the metrics. Took a bit for all the queries to run :) Let me know if you have further thoughts on the final subtask: > [ ] Let's discuss what we can do to make the metric more robust and reliable (

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Manuel, Aklapper, Lydia_Pintscher, Astuthiodit_1, AWesterinen, karapayneWMDE

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Manuel, Aklapper, Lydia_Pintscher, Astuthiodit_1, AWesterinen, karapayneWMDE

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T342111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja

[Wikidata-bugs] [Maniphest] T341330: [Analytics] Airflow implementation of unique ips accessing Wikidata's REST API metrics

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE renamed this task from " [Analytics] Airflow implementation of unique user-agents accessing Wikidata's REST API metrics" to " [Analytics] Airflow implementation of unique ips accessing Wikidata's REST API metrics". AndrewTavis_WMDE updated the task descr

[Wikidata-bugs] [Maniphest] T341330: [Analytics] Airflow implementation of unique ips accessing Wikidata's REST API metrics

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T341330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Manuel, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T342111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T334558: [Analytics] Unique user-agents accessing Wikidata's REST API for Q2/2023

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Sure, I'll head over to T341330 <https://phabricator.wikimedia.org/T341330> and edit the task so it's aligned with what we discussed :) TASK DETAIL https://phabricator.wikimedia.org/T334558 EMAIL PREFERENCES https://phabricator.wikimedia.org/se

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Manuel, Aklapper, Lydia_Pintscher, Astuthiodit_1, AWesterinen, karapayneWMDE

[Wikidata-bugs] [Maniphest] T341330: [Analytics] Airflow implementation of unique user-agents accessing Wikidata's REST API metrics

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. Note that this task is blocked by {https://phabricator.wikimedia.org/T340648} :) TASK DETAIL https://phabricator.wikimedia.org/T341330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a subscriber: Manuel. AndrewTavis_WMDE added a comment. @Manuel, the task description has now been updated with the aggregate values for the dump from `2023-7-10`. As this is weekly, the next dump will be made on the 19th, so I'll go ahead and rerun the process then so

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T342111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T342111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Lydia_Pintscher, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Aklapper, Lydia_Pintscher, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. > We might create a little report for this and the coming WDQS task. We could do this, yes. Do you mean something in Google Docs? (I mention a readme with the code below as another alternative) > In terms of code documentation GitHub woul

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T342111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Aklapper, Manuel, Astuthiodit_1, karapayneWMDE, Invadibot

[Wikidata-bugs] [Maniphest] T337021: [Analytics] Find out size of term subgraph

2023-07-18 Thread AndrewTavis_WMDE
AndrewTavis_WMDE added a comment. And would a table output be preferable for easier comparison? TASK DETAIL https://phabricator.wikimedia.org/T337021 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Manuel, Aklapper

  1   2   3   4   5   6   >