Milimetric added subscribers: VirginiaPoundstone, Milimetric.
Milimetric moved this task from Backlog to Other teams on the Dumps-Generation
board.
Milimetric added a comment.
indeed, there are quite some differences in the different pipelines. When
the #wikidata <ht
Milimetric added a comment.
Just a random drive-by note, since I'm not the one playing with this, but it
might be interesting to instrument EventBus a little bit. For example, from
the deferred job that publishes to Kafka, we could log a basic key for each
event that we publish. It should
Milimetric added a comment.
@Addshore: this
<https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blame/cee58924d2ce954a3d2eb9dfd6ab3b55426fd572/includes/WikimediaEventsHooks.php#L83>
seems to only be happening on logged-in requests, at least a lot of the time
if n
Milimetric added a comment.
@Manuel & @GoranSMilovanovic: sorry for the long delay, no we don't back up
the stat machines or other analytics clients. From the "Local data storage"
section on the Analytics Clients
<https://wikitech.wikimedia.org/wiki/Analytics/Systems
Milimetric added a comment.
We are forwarding this request to #privacy_engineering
<https://phabricator.wikimedia.org/tag/privacy_engineering/> but please don't
hesitate to loop us back in if you have questions.
TASK DETAIL
https://phabricator.wikimedia.org/T291186
EMAIL PREFE
Milimetric closed this task as "Declined".
Milimetric added a comment.
Yeah, the Vue adoption is going down a different path.
TASK DETAIL
https://phabricator.wikimedia.org/T167145
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Mil
Milimetric closed subtask T167145: Evaluate OOUI integration with vue.js as
Declined.
TASK DETAIL
https://phabricator.wikimedia.org/T167134
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Lydia_Pintscher, Milimetric
Cc: Lydia_Pintscher
Milimetric added a comment.
This is a bit of a drive-by, but have we considered https://min.io/? I went
a bit deeper than just the marketing and was impressed by their
error-correcting implementation.
TASK DETAIL
https://phabricator.wikimedia.org/T247058
EMAIL PREFERENCES
https
Milimetric edited projects, added Wikidata-Query-Service, Discovery; removed
Analytics.
Restricted Application added a project: Wikidata.
TASK DETAIL
https://phabricator.wikimedia.org/T255044
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric triaged this task as "High" priority.
Milimetric assigned this task to Ottomata.
Milimetric added a project: Analytics-Kanban.
TASK DETAIL
https://phabricator.wikimedia.org/T253753
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To
Milimetric added a comment.
FYI: checked monthly sqoop and it's not referenced there so puppet is good to
go
(https://github.com/wikimedia/puppet/blob/ef13ee28202b2ff9653bfaf3cbf6b2a0433aa8c2/modules/profile/templates/analytics/refinery/job/refinery-sqoop-mediawiki.sh.erb)
TASK DETAIL
Milimetric added a comment.
my apologies, was running through triage, we'll take a look and ping here
TASK DETAIL
https://phabricator.wikimedia.org/T236895
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup, Milimetric
Cc: Milimetric
Milimetric added a comment.
Added @Nuria as a reviewer where she was pinged, she'll comment there,
waiting on WMDE on the other patch (review done)
TASK DETAIL
https://phabricator.wikimedia.org/T236895
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric added a comment.
The output is here:
https://analytics.wikimedia.org/published/datasets/periodic/reports/metrics/structured-data/
TASK DETAIL
https://phabricator.wikimedia.org/T239565
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric added a comment.
@Nuria: what I thought we were doing is productionizing @matthiasmullie's
approach from T239565#5728117
<https://phabricator.wikimedia.org/T239565#5728117>. This seemed sensible to
me, and the change I submitted for T239565
<https://phabricator.wiki
Milimetric added a comment.
Ok, seems like some of this confusion is getting cleared up. For my part,
here's what I'm planning to do next:
- Productionize the query currently getting the 3 million or so `role_name =
mediainfo` slots
- Productionize the query currently getting the 7
Milimetric added a comment.
I translated T238878#5708511
<https://phabricator.wikimedia.org/T238878#5708511> to Hive to familiarize
myself with it and get ahead of productionizing it. I got similar numbers
first as a sanity check and then grouped the numbers by the
Milimetric added a comment.
Yay, I get to work with @mpopov :) Ok, questions:
- how often should this report be updated?
- is it exactly that query? This task mentions "queries" plural, just making
sure
- given the confusion about deletion (T238878#57068
Milimetric added a comment.
I noticed that three folks used `rev_deleted = 0` to mean "revision not
deleted", but this field means something completely different
(https://www.mediawiki.org/wiki/Manual:Revision_table#rev_deleted). Basically,
it's about parts of the revision that
Milimetric removed a project: TechCom-RFC.
Milimetric added a comment.
Removing TechCom-RFC for now, just add it back when this task is ready to go
through the RFC process.
TASK DETAIL
https://phabricator.wikimedia.org/T236150
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings
Milimetric added a comment.
https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/wikidata/all-access/user/daily/2019061300/2019061300
> 1420740
https://stats.wikimedia.org/v2/#/wikidata.org/reading/total-page-views/normal|bar|1-month|agent~user|daily
(note sp
Milimetric added a comment.
To clear up what Joseph said, we're never going to have more than 90 days of
geolocated edits for privacy reasons. We do have two aggregated datasets that
go back more than a year:
https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors
Milimetric added a comment.
@Lucas_Werkmeister_WMDE do you think Tech Com can be useful here or should
this task just live in a backlog?
TASK DETAIL
https://phabricator.wikimedia.org/T176312
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric added a comment.
oh man, revoke my arithmetic license, I did that completely wrong, will edit my commentTASK DETAILhttps://phabricator.wikimedia.org/T213318EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MilimetricCc: jijiki, CDanis, ArielGlenn
Milimetric added a comment.
A few quick stats to contextualize @daniel's point above. Wikidata has* 400 million edits [1] and 900 million pageviews [2] over the past 2 years. This represents 43% of all edits and 24% of all pageviews. So Wikidata definitely skews more towards editing. It's
Milimetric added a comment.
I agree that we can't go back on decisions that are 3 years in the making. But I do like Timo's point that we should state the problem. Here's an attempt:
"Implementing interaction in JS on top of static html rendered by PHP is inflexible and complicated. It
Milimetric added a comment.
@WMDE-leszek ok, we're on the same page, except the crazy part of my proposal. I was saying directly routed to SSR service as in, without ever hitting mediawiki and spinning up the mediawiki context. So this would expose SSR publicly. The fallback stub HTML generated
Milimetric added a comment.
Thanks @Jakob_WMDE, I think we're saying the same thing in slightly different terms, and it's because I'm not being precise. It's ok for the client and server to have different implementations, but you're saying the have the same capabilities, right? I was thinking
Milimetric added a comment.
In T212189#4838090, @Addshore wrote:
In T212189#4835359, @Milimetric wrote:
Now, I started looking through the code and it looks like there's an effort to keep server and client logic as common as possible, with factories and interfaces and nice patterns
Milimetric added a comment.
In https://wikitech.wikimedia.org/wiki/WMDE/Wikidata/SSR_Service we see that "There is a server-side and the client-side variant of the code, which are distributions of the same implementation." Looking at the current repository, we see that it shares d
Milimetric added a comment.
@Nuria, I'm catching up on the task that Jaime recommended and will comment there.TASK DETAILhttps://phabricator.wikimedia.org/T209031EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MilimetricCc: bd808, tstarling, Fjalapeno, Cloud
Milimetric assigned this task to Nuria.
TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Nuria, MilimetricCc: Nuria, mpopov, chelsyx, Aklapper, Addshore, Smalyshev, Lydia_Pintscher, Akovalyov, Lahi, Gq86
Milimetric moved this task from Incoming to Radar on the Analytics board.
TASK DETAILhttps://phabricator.wikimedia.org/T199517WORKBOARDhttps://phabricator.wikimedia.org/project/board/11/EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MilimetricCc: Nuria
Milimetric added a comment.
To add relevance to this, Wikistats 2.0 which will be launching soon, and is hosted on stats.wikimedia.org, is also using Vue.TASK DETAILhttps://phabricator.wikimedia.org/T168264EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric added a comment.
Yes, I agreed with you when you raised this issue at the hackathon. It's riskier to depend on a smaller library. It might not be as big of a deal here because preact and react are compatible, so if the performance on mobile is a big deal, may be worth considering.TASK
Milimetric added a comment.
@Jan_Dittrich: It seems like the mobile folks are leaning towards preact, but it's worth pinging them. I don't know of anyone else and we stay away from both mediawiki and php for the most part. We do have some extension work coming up soon though, and it's great
Milimetric added a comment.
Yep, ask me anything.TASK DETAILhttps://phabricator.wikimedia.org/T167144EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MilimetricCc: Milimetric, Aklapper, Jonas, thiemowmde, Aleksey_WMDE, Jan_Dittrich, daniel, GoranSMilovanovic
Milimetric added a comment.
Great points, Magnus. As for .vue files and webpack setup, there is vue-cli which generates some common configuration according to configurable templates. So for example, if you want Vue with webpack, hot reload, .vue transpiling, etc. you can do:
$ npm install -g
Milimetric added a comment.
Magnus, that's great, also interested in your thoughts. SSR on vue: https://vuejs.org/v2/guide/ssr.html. Seems like some basic support for vue ssr is possible with php's V8 engine, not sure if we're considering that or just de-coupling and running on node (I share
Milimetric added a comment.
@Jan_Dittrich: I read your notes and I think your choice between React ecosystem and Vue ecosystem is a good one, no matter which way you go. I prefer Vue but that's totally subjective, so I would recommend doing a small project in each to see how your team feels.
I'd
Milimetric added a comment.
@Jan_Dittrich this is exciting. We're also looking at Vue.js for the Wikistats 2.0 rewrite. We have to write a prototype and make sure it performs the way we need, so we haven't decided yet but we have similar questions and concerns. We are coming from knockout, so
Milimetric added a comment.
works for me, brainstorm "meetings" could be new tasks in
https://phabricator.wikimedia.org/tag/tabular-data/?
TASK DETAIL
https://phabricator.wikimedia.org/T134426
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailp
Milimetric added a comment.
> Another needed feature is to reuse metadata from another page, thus
allowing multiple pages to have the same structure.
We should brainstorm this external schema idea, set up those weekly meetings
I mentioned :)
TASK DETAIL
ht
Milimetric added a comment.
@Yurik,
I agree that the majority seem to be supporting, but I think the opposing
voices should be taken into consideration more carefully. I read some of the
opposition and your replies, and I agree with you about the idea of releasing
early and releasing
Milimetric claimed this task.
Milimetric added a project: Analytics-Kanban.
TASK DETAIL
https://phabricator.wikimedia.org/T120452
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Milimetric
Cc: MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus
Milimetric added a comment.
I'd like to help with this discussion, but not sure where to start.
Commons - seems like a decent fit, but not sure what the process is to get
approval
Wikidata - seems like the best fit, but the team managing it seems to
disagree strongly, so do we just
Milimetric closed blocking task T116430: Requests to dumps.wikimedia.org should
end up in hadoop wmf.webrequest via kafka! as "Declined".
TASK DETAIL
https://phabricator.wikimedia.org/T117205
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To
Milimetric closed this task as "Declined".
Milimetric added a comment.
So we're leaning towards declining this unless requests for
dumps.wikimedia.org go through varnish.
What about using piwik for tracking stats on dumps.wikimedia.org? Is that
enough or do you want other st
Milimetric added a comment.
nice job organizing, Ariel, let me know if you want to bounce the questions off
someone
TASK DETAIL
https://phabricator.wikimedia.org/T114019
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: ArielGlenn, Milimetric
Cc
Milimetric edited projects, added Analytics; removed Analytics-Kanban.
TASK DETAIL
https://phabricator.wikimedia.org/T114443
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ottomata, Milimetric
Cc: yuvipanda, JanZerebecki, Glaisher, Milimetric, RobLa
Milimetric moved this task to Radar on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T114443
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ottomata
Milimetric added a comment.
@BBlack - so you think cache_status is not even close to accurate? Do we have
other accurate measurements of it so we could compare to what extent it's
misleading? I'm happy to remove it from the data if it's really bad.
TASK DETAIL
https
Milimetric added a comment.
@ArielGlenn - looking forward to the discussion
TASK DETAIL
https://phabricator.wikimedia.org/T114019
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: ArielGlenn, Milimetric
Cc: jberkel, NealMcB, jcrespo, Bianjiang
Milimetric added a subscriber: Milimetric.
TASK DETAIL
https://phabricator.wikimedia.org/T114019
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: ArielGlenn, Milimetric
Cc: Milimetric, RobLa-WMF, GWicke, TTO, zhuyifei1999, StudiesWorld, gnosygnu,
LA2
Milimetric removed a project: Analytics.
TASK DETAIL
https://phabricator.wikimedia.org/T114443
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ottomata, Milimetric
Cc: Glaisher, Milimetric, RobLa-WMF, brion, intracer, Smalyshev, mark,
MZMcBride
Milimetric moved this task to Radar on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T84923
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Milimetric
Cc
Milimetric moved this task to Radar on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T116247
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: mobrovac
Milimetric moved this task to Move To Prioritized on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T114443
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric moved this task to Prioritized on the Analytics-Backlog workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T64874
WORKBOARD
https://phabricator.wikimedia.org/project/board/1308/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences
Milimetric added a subscriber: Milimetric.
Milimetric added a comment.
In https://phabricator.wikimedia.org/T65808#1811005, @Magnus wrote:
> Not sure if to add this here or start a new task, but havinf CORS for Labs
> (or anyone) for the new pageviews API, e.g.
>
> https://wikim
Milimetric moved this task to Done on the Scrum-of-Scrums workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T78822
WORKBOARD
https://phabricator.wikimedia.org/project/board/64/
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
Milimetric moved this task to Done on the Scrum-of-Scrums workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T1221
WORKBOARD
https://phabricator.wikimedia.org/project/board/64/
REPLY HANDLER ACTIONS
Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign
62 matches
Mail list logo
