[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic closed this task as "Resolved". GoranSMilovanovic added a comment. We're in production: https://wikidata-analytics.wmcloud.org/app/WikidataAnalytics Closing as resolved. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikime

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. - In production on test server: http://datakolektiv.org/app/WD_Inequality - Deploying to Wikidata Analytics now. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wiki

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic reopened this task as "Open". GoranSMilovanovic added a comment. @Lydia_Pintscher Re-opened; the system is (1) not in production state, and (2) not included to Wikidata Analytics yet. It should not take more than several hours to complete (1, 2) but I need the

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Lydia_Pintscher closed this task as "Resolved". Lydia_Pintscher added a comment. \o/ Looking good. Let's close this. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, Lydia_P

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. Looks fine to me! Am So., 14. März 2021 um 14:54 Uhr schrieb GoranSMilovanovic < no-re...@phabricator.wikimedia.org>: > GoranSMilovanovic added a comment. View Task > https://phabricator.wikimedia.org/T270109 > > @Lydia_Pintscher https://phabricat

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Lydia_Pintscher @Jan_Dittrich @WMDE-leszek Please help me clear my backlog a bit: this analytics dashboard, for example, needs a review before I productionize it. Thank you. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich Fixed: > current_year → this year > current_snapshot → over last month > history_to_current_snapshot → all time see http://datakolektiv.org:3838/WD_Inequality/ TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PRE

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. > Plotly already provides for that: if a user zooms in the diagram, an overlay message should pop up … Ahh, OK, works for me, too. Not great UX, I think, but if it is build-in that way lets keep it as it is. > It is the current calendar year OK, th

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich > One variable I did not ask for: current_year – is this from beginning of the calendar year to the current snapshot or from the snapshot 12 months ago until the current snapshot? It is the **current calendar year**, e.g. 2021, 2022,

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. - If it is possible to give "human" labels to the variables, and //if// I understand them right it might be useful to do the following - current_year → over last 12 month - current_snapshot → over last month - history_to_current_snapshot → all time -

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. One variable I did not ask for: `current_year` – is this from beginning of the calendar year to the current snapshot or from the snapshot 12 months ago until the current snapshot? TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich @guergana.tzatchkova @WMDE-leszek @Lydia_Pintscher Dashboard on the test-server, please review: http://datakolektiv.org:3838/WD_Inequality/ TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wi

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @guergana.tzatchkova @Jan_Dittrich @WMDE-leszek - The data in your public directory have just received an update; - I will now start building a si

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich - If you visit the public data directory at https://analytics.wikimedia.org/published/datasets/wmde-analytics-engineering/Wikidata/WD_Inequality/ - you will find one `.csv` file there: hoover measurement

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. Thanks! Just to understand it correctly – what do the 3 values refer to, particularly, are they from beginning of time until some point ("wealth") or for a specific timeframe ("income")? TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENC

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, Maintenance_bot Cc: Lydia_Pintscher, WMDE-leszek, Jan_Dittrich, GoranSMilovan

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich @Lydia_Pintscher - the computation of the Hoover inequality index will be run every time a new snapshot of wmf.mediawiki_history is detected, - in Pyspark

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

gerritbot added a comment. Change 662093 **merged** by GoranSMilovanovic: [analytics/wmde/WD/WikidataAnalytics@master] T270109 init WD_Inequality https://gerrit.wikimedia.org/r/662093 TASK DETAIL https://phabricator.wikimedia.org/T270109 E

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, gerritbot Cc: Lydia_Pintscher, WMDE-leszek, Jan_Dittrich, GoranSMilovanovic, Aklapper

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

gerritbot added a comment. Change 662093 had a related patch set uploaded (by GoranSMilovanovic; owner: GoranSMilovanovic): [analytics/wmde/WD/WikidataAnalytics@master] T270109 init WD_Inequality https://gerrit.wikimedia.org/r/662093 TASK DE

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Lydia_Pintscher @Jan_Dittrich On the test server, of course - no WMF or WMDE domains are there. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilova

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Lydia_Pintscher added a comment. Can we see it somewhere before adding it to the portal? In general probably fine but quick review for understandability etc would be good. TASK DETAIL https://phabricator.wikimedia.org/T270109 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/p

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a subscriber: Lydia_Pintscher. Jan_Dittrich added a comment. > The dashboard for this can be produced in a minimal time. It will be included to our Wikidata Analytics portal, under "Analytics", if you agree. Would make sense to me; I guess the final decision is with @Ly

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich > I do not know how costly running "begin of time" is – I am pretty fine with running it every 3 month or so, too to save some computing time and CO2. The planet would survive a monthly update. Many of our system run similar, regular

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich added a comment. Thanks! If I understand you correctly, you can/did calculate scores based on Edits from within a month ("income", monthly) Edits from within a year ("income", yearly) for the "since the beginning of time" ("wealth") however, you wonder why there

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added a comment. @Jan_Dittrich @guergana.tzatchkova @WMDE-leszek Everything is ready to put this on regular updates. There are some constraints in relation to the cost of data engineering procedures. Let me share with you: - Essentially, our ETL procedure extracts

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

GoranSMilovanovic added projects: WMDE-Analytics-Engineering, User-GoranSMilovanovic. GoranSMilovanovic claimed this task. GoranSMilovanovic added a comment. @Jan_Dittrich Got it. I will get back to you if it turns that I need more info. TASK DETAIL https://phabricator.wikimedia.org/T27010

[Wikidata-bugs] [Maniphest] T270109: Hoover Inequality Score Data Retrival and Calculation

Jan_Dittrich created this task. Jan_Dittrich added a project: Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION **User Story:**: As a PM/UX-Person I want to use an inequality score as part of assessing community health Based on the input of @GoranSMilovanovic,