[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Christopher added a comment. @Addshore yes, I will just create a separate remote download set function and point it at your sources so that we can use both local and remote data. One thing to note (particular to Dygraphs) is that the order of the columns in the datasource matters. The first column is the one that appears first in the legend (or on the bottom for stacked). So, if we are shooting for consistency with the Magnus graphs, this is worth considering in your scripts. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, Christopher Cc: Milimetric, Nuria, aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Addshore added a comment. After further discussion with @Milimetric I think Dashiki might be the way forward with some / all of this. And yet again I may now not end up trying to use limn TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Milimetric, Nuria, aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Addshore added a comment. Well, after speaking with nuria in #wikimedia-analytics it looks like limn isn't really used any more apparently? SO maybe we wont move forward with this bah, such confusion! TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Nuria added a subscriber: Nuria. Nuria added a comment. Wait, one thing is limn, other (i know, confusing) the limn-data repositories. those are not tied to limn necessarily, they are just poorly named. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Nuria Cc: Nuria, aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
aude added a comment. \o/ TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, aude Cc: Milimetric, Nuria, aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Christopher added a comment. FYI: I am working on the dashboards and have made some progress using the shiny-server. Check out the very preliminary prototype at http://wdm.wmflabs.org/wdm The repo is here https://git.wikimedia.org/tree/wikidata%2Fanalytics%2Fdashboard/HEAD @Addshore if you can generate/export the Magnus and Lydia tsv data (for starters) as they are being used (examples in the /data directory) and put them in an accessible place , this would be extremely helpful. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, Christopher Cc: Milimetric, Nuria, aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
aude added a subscriber: aude. aude added a comment. where, what data? TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aude Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Addshore added a comment. Indeed! Some of the information can already be pulled from the dbs. Other information will need scripts to run to collect and put in some tables in some other db on the analytics cluster. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
aude added a comment. so, we make one of https://meta.wikimedia.org/wiki/Research:Data/Dashboards ? looks somewhat straightforward to do TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aude Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
aude added a comment. hmmm... no idea what that is but ok. replacing a spreadsheet sounds good :) TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aude Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Addshore added a comment. Initially looking to replace the "Wikidata community engagement tracking" spreadsheet. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
JanZerebecki added a comment. What type of repo are we talking about here? TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Addshore added a comment. Indeed. We would need somewhere that runs scripts to pull data and store in some sql tables somewhere on the analytics cluster. Then this repo would contain the stuff to read from those repos and make the limn graphs TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
JanZerebecki added a comment. Ah this: https://wikitech.wikimedia.org/wiki/Analytics/Dashboards#Make_repository_for_queries_and_configuration TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
aude added a comment. @JanZerebecki that's one of the reasons I am asking. I would like to proceed with getting that done. :) TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aude Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
Christopher added a comment.
@JanZerebecki It would nice to consolidate the post-processed "ready to use"
export data sets in a single directory that can be accessed by different
presentation clients simply over http. Like here
http://datasets.wikimedia.org/aggregate-datasets/{wikidata} for example.
I highly recommend enabling simple REST access to the data rather than having
to use SQL to get values for analytics presentation.
TASK DETAIL
https://phabricator.wikimedia.org/T112506
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Christopher
Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
JanZerebecki added a comment. https://wikitech.wikimedia.org/wiki/Analytics/Dashboards also describes how it can contain SQL queries that get executed regularly on stat1003.eqiad.wmnet . But it seems the output is only stored in csv files. So not sure if we want to adapt that to also insert in an mysql table or make it only read from the mysql tables we generate from a different script. If we want to display some of that in grafana, it seems fairly easy to also put the values into graphite: https://wikitech.wikimedia.org/wiki/Graphite#Record_data TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T112506: Dashboard repository for limn-wikidata-data
JanZerebecki added a comment. Yes that would be what the csv/tsv files are for, I was in no way suggesting not to create them. TASK DETAIL https://phabricator.wikimedia.org/T112506 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: aude, JanZerebecki, Christopher, Addshore, Aklapper, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
