Nuria added a comment.
I second @ottomata
TASK DETAIL
https://phabricator.wikimedia.org/T117732
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Lydia_Pintscher, fgiunchedi, Christopher, JanZerebecki, Nuria, Ottomata,
Aklapper, Addshore
Nuria added a comment.
> As mentioned, we might want to use a single node process exposing parsoid,
> restbase & eventbus for small (third party) installs, but might as well use
> the new EventLogging service in production.
To date we do not have a third party install s
Nuria added a comment.
> I don't see these two as being mutually-exclusive. In order to meet the end
> goal of a generalised event service we are starting with the Services' use
> case. The MVP is part of >one of our quarterly goals. We have almost
> finalised the events and
Nuria added a project: Analytics-Backlog.
TASK DETAIL
https://phabricator.wikimedia.org/T117402
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Addshore, Nuria
Cc: JanZerebecki, fgiunchedi, gerritbot, Addshore, Aklapper, Wikidata-bugs,
aude, Mbch331
Nuria added a comment.
> a way to expose a stream of events in a defined format that can be consumed
> easily by a range of clients.
This talks about consumption, not production but I do not want to get too deep
into that cause I really I do not think we are discussing w
Nuria added a subscriber: Nuria.
Nuria added a comment.
Wait, one thing is limn, other (i know, confusing) the limn-data repositories.
those are not tied to limn necessarily, they are just poorly named.
TASK DETAIL
https://phabricator.wikimedia.org/T112506
EMAIL PREFERENCES
https
Nuria added a comment.
> @Ottomata, main reason would be the ability to work with $simple_queue,
> $binary_kafka, $amazon_queue and so on without changes in MW code. This isn't
> so theoretical. We'll want a lighter-weight queue for testing, developers and
> third party users
Nuria added a comment.
> EventLogging: Decode, validate and enqueue JSON events for EL.
mmm..I am not sure who would be the users of this endpoint at this time, do
you have a case for EL that is not served by varnish endpoint?
> Provide edit related events (ex: edit, creation, de
Nuria closed this task as "Resolved".
TASK DETAIL
https://phabricator.wikimedia.org/T119054
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: JAllemandou, Nuria
Cc: Tbayer, gerritbot, Lydia_Pintscher, Aklapper, Addshore, StudiesWorld,
Wik
Nuria added a comment.
@addshore: It is on our backlog but we have several things before it so we
cannot give an ETA. Now, I suggest that 1) you do some ad-hoc querying and get
the data you need to met your end of December deadline. And 2) we can work
together on oozification of this job later
Nuria added a subscriber: Nuria.
Nuria added a comment.
@addshore: Do you have access to cluster 1002 to run querys yourself? Timeline
wise if you need this before end of year it might be faster if you start
working on it while we help you get changes going.
TASK DETAIL
https
Nuria added a comment.
See attached a rough preview of 1 week of wikidata requests per browser per
country via Druid
TASK DETAIL
https://phabricator.wikimedia.org/T130102
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Nuria, Addshore
Nuria added a comment.
FYI that when we have our pageview dataset working on druid you could look at
this data in an easier fashion, now, as i said (other than they many crawlers
for wikidata) browser stats per project are not that significantly different.
TASK DETAIL
https
Nuria added a comment.
@Lydia_Pintscher:
Ah! Sorry, I should have included this:
The data is all on wedrequest table of wmf db on hive for all projects.
https://wikitech.wikimedia.org/wiki/Analytics/Data/Webrequest
You can take a look for wikidata pageviews but from having looked
Nuria added a project: Analytics.
TASK DETAIL
https://phabricator.wikimedia.org/T130102
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Nuria, Addshore, Aklapper, Lydia_Pintscher, D3r1ck01, Izno, JAllemandou,
Wikidata-bugs, aude, Mbch331
Nuria added a comment.
@Lydia_Pintscher: Have you evaluated (by looking at current data) that
wikidata browser stats are significantly different from other projects?
This question comes up often and our browser traffic -when we have looked at
it in the past- doesn't exhibit major
Nuria moved this task to Radar on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T120452
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Yurik, Nuria
Cc: TheDJ
Nuria moved this task to Tasked on the Analytics workboard.
TASK DETAIL
https://phabricator.wikimedia.org/T120452
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Yurik, Nuria
Cc: TheDJ
Nuria added a comment.
The policy doesn't have an specific owner, if that is what you are asking.
Here is is: https://meta.wikimedia.org/wiki/User-Agent_policy
TASK DETAIL
https://phabricator.wikimedia.org/T135164
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel
Nuria added a comment.
I am not sure this requires any works from analytics team. Seems like the
data you need is already available on pageview API, correct?
TASK DETAIL
https://phabricator.wikimedia.org/T132223
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel
Nuria removed projects: Analytics, Pageviews-API.
TASK DETAIL
https://phabricator.wikimedia.org/T132223
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Nuria, Lucie, Addshore, Lydia_Pintscher, Ricordisamoa, Quiddity, Aklapper,
D3r1ck01
Nuria added a comment.
I realize this task did not included the link to browser reports:
https://browser-reports.wmflabs.org/#all-sites-by-os
Please let us know if it can be closed, i assume @Addshore has provided the
data you needed. As I mentioned browser stats are not significantly
Nuria added a comment.
.Please try to notify owner of UA policy. If they add the word "bot" to UA
this would automatically be marked as spider.
TASK DETAIL
https://phabricator.wikimedia.org/T135164
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailp
Nuria moved this task from Incoming to Backlog on the Analytics board.
TASK DETAIL
https://phabricator.wikimedia.org/T135164
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc
Nuria moved this task from Backlog to Q1 (July 2016) on the Analytics board.
TASK DETAIL
https://phabricator.wikimedia.org/T135164
WORKBOARD
https://phabricator.wikimedia.org/project/board/11/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Nuria edited projects, added Analytics; removed Analytics-Kanban.
TASK DETAIL
https://phabricator.wikimedia.org/T120452
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Milimetric, Nuria
Cc: Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid
Nuria moved this task from Q1 (July 2016) to Q2 (October 2016) on the Analytics board.
TASK DETAILhttps://phabricator.wikimedia.org/T135164WORKBOARDhttps://phabricator.wikimedia.org/project/board/11/EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc
Nuria added a comment.
Comments from the project standpoint:
@Addshore: we will not be adding new features to Pageview API until we have finished our scaling project and added counting of pageviews for wikis for which it is not happening (ex: outreachwiki)
Any feature additions will happen
Nuria added a comment.
This is again another instance of bot traffic that slips by, this UA might not be causing trouble now but there will be others. Merging into parent taskTASK DETAILhttps://phabricator.wikimedia.org/T135164EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel
Nuria added a comment.
ping @AddshoreTASK DETAILhttps://phabricator.wikimedia.org/T160825EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Lydia_Pintscher, Addshore, matej_suchanek, Aklapper, QZanden, D3r1ck01, Izno, JAllemandou, Wikidata-bugs
Nuria added a comment.
F4079411: Screen Shot 2016-05-30 at 10.28.41 AM.pngTASK DETAILhttps://phabricator.wikimedia.org/T130102EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Addshore, Aklapper, Lydia_Pintscher, QZanden, D3r1ck01, Izno
Nuria added a comment.
Will be closing this task as http://pivot.wikimedia.org ( to which wikimedia de has access) provides this data.TASK DETAILhttps://phabricator.wikimedia.org/T130102EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Addshore
Nuria closed subtask T130102: [Task] dashboard showing browser usage distribution for Wikidata as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T108931EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Ricordisamoa, Abraham
Nuria closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T130102EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Addshore, Aklapper, Lydia_Pintscher, QZanden, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aud
Nuria added a comment.
Also, have in mind that browser usage is really not that different per project and overall, the overall info should be sufficient to take triage decisions: https://analytics.wikimedia.org/dashboards/browsers/#desktop-site-by-osTASK DETAILhttps://phabricator.wikimedia.org
Nuria assigned this task to Ottomata.
TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: Anomie, Aklapper, Smalyshev, QZanden, EBjune, merbst, Salgo60, Avner, debt, Gehel, D3r1ck01, Jonas
Nuria added a comment.
Moving to radar as it doesn't seem there are any actionables for analytics.TASK DETAILhttps://phabricator.wikimedia.org/T170400EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Lydia_Pintscher, Jan_Dittrich, Aklapper
Nuria added a subscriber: Smallyen03.Nuria added a comment.
@Smallyen03 : the idea of the tags is to be able to split webrequest dataset into "partitions" that make subsequent querying more effective. So tags have to be coarse, so this one sounds good: "Tag for a request containi
Nuria added a comment.
ping @Smalyshev is this still a need? Maybe we should set up a short 30 minute sync upTASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: Nuria, Anomie, Aklapper
Nuria added a comment.
From meeting:
@Smalyshev can consume from either kafka or event stream once we add the ability to consume from a given point in time, this is what is mean by "seekable" (on new kafka cluster, next quarter, Q1) .
Keeping data for longer than 7 days is no
Nuria added a comment.
As far as I understand you need to publish not only queries to service but also query results (is this correct @Smalyshev?) analyzing those will produce the metric counts @AndrewSu and @leila are interested on. This requires a schema definition of what a query result
Nuria added a comment.
To incentivize them to contribute, we have to give them even better metrics of community usage/impact that they can give to funders
Understood, as I said we are willing to help in any way we can, seems like a great objective. My main point is that if we come up
Nuria added a comment.
@Smalyshev @AndrewSu please take a look at other metric definitions we have. once you decide on a metric definition please be so kind as to document it in beta: https://meta.wikimedia.org/wiki/Research:Standard_metrics#Newly_registered_user
This helps a lot to quantify what
Nuria added a comment.
If @Smalyshev thinks this would be a good idea and can develop the instrumentation for the metrics and own the metric definition (together with "gene wiki") we can help on the project as needed, seems to me that things like these could be computed with the infrast
Nuria edited projects, added Analytics; removed Analytics-Kanban.
TASK DETAILhttps://phabricator.wikimedia.org/T173850EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Ottomata, gerritbot, Mattflaschen-WMF, Liuxinyu970226, WMDE-leszek, Anomie, Aklapper
Nuria added a comment.
@chelsyx That makes sense, thank you.
I was also trying to make a meta point though: since prior work and statistics exist for commons it will be worth documenting ( on meta?) these numbers and why/how they differ with other numbers community might have access to. I know
Nuria added a comment.
Is the user versus bot percentage overall? I am not sure that is of value to quantify usage as of 2017, right? See timeseries of uploads by bots/users at https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm (scroll down)
Most recent monthly numbers
Nuria added a comment.
Are there any docs we can look at with metrics?TASK DETAILhttps://phabricator.wikimedia.org/T177354EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, NuriaCc: Nuria, Aklapper, mpopov, chelsyx, Abit, SandraF_WMF, Ramsey-WMF
Nuria added a subscriber: mforns.Nuria added a comment.
@Smalyshev: Take a look at information we keep on pageview hourly, for long time keeping we need to remove PII and we neither store detail timestamps or sessionIds as we want to avoid session reconstruction precisely. So probably if we round
Nuria added a comment.
@Smalyshev Ok, we aim to have the cluster handling all prod traffic by end of next quarter, until then it will be mirroing data which i think should be sufficient for you to get started in the wdqs consumer? Correct me if I am wrong.TASK DETAILhttps
Nuria added a comment.
@Smalyshev Please, 45 minutes with me and @Ottomata would do?TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: gerritbot, JAllemandou, Pchelolo, Ladsgroup, Nuria
Nuria added a comment.
@Smalyshev We like to default to public if possible, the more eyes on the data the more useful it can be.TASK DETAILhttps://phabricator.wikimedia.org/T143819EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: mforns, PokestarFan
Nuria added a comment.
I got same doing:
/home/otto/kafkacat -Q -b kafka-jumbo1003.eqiad.wmnet -t eqiad.mediawiki.revision-create:0:1512687299 -Xdebug=allTASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences
Nuria added a comment.
Nice, Can @Smalyshev check whether consuming from these topics as set would work for his purposes?TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: gerritbot
Nuria added a comment.
@Ottomata Could @Smalyshev do a test on consuming from the new cluster though with teh understanding it is not yet productionized to make sure it fits the use cases?TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org
Nuria closed subtask T187296: Increase kafka event retention to 31 as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: gerritbot, JAllemandou, Pchelolo, Ladsgroup, Nur
Nuria closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T187296EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, NuriaCc: mforns, elukey, Ottomata, Aklapper, Nuria, Ladsgroup, Pchelolo, JAllemandou, Smalyshev,
Nuria added a comment.
Ping @Smalyshev now that you have a reliable stream on the new kafka cluster (that supports time-based consumption) is there any other blockers on your end ?TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel
Nuria added a comment.
Please have in mind that metrics for commons exist is https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaCOMMONS.htm , let's make sure those are looked at when this work is taking place.TASK DETAILhttps://phabricator.wikimedia.org/T174519EMAIL PREFERENCEShttps
Nuria added a subscriber: JAllemandou.Nuria added a comment.
I think notes look good.
@mforns main point that I missed is that we probably also want to remove geolocation from dataset #1, I see that from your sumup you did.
Remaining item is sanitization of sparql queries and on that I think we
Nuria added a comment.
Nice! Thank you for documenting.TASK DETAILhttps://phabricator.wikimedia.org/T174519EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, Liuxinyu970226, Capt_Swing, Ramsey-WMF, SandraF_WMF, Abit, chelsyx, mpopov, debt
Nuria closed this task as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T191022EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Jonas, NuriaCc: Smalyshev, Nuria, gerritbot, JAllemandou, Jonas, Aklapper, Gaboe420, Versusxo, Majestic
Nuria added a comment.
Bot did not accepted cookies, user agent was changing slightly, in 1000 records when this event is happening 995 are part of event and of those about 200 are unqiue user agents. Still the IP is teh same and the volumes of requests so high that I am wondering how
Nuria added a comment.
yes , please, I listed issue on dataset page: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Unique_Devices#Changes_and_Known_Problems_with_Dataset
We do not yet have annotations in wikistats (we will at the end of quarter) but when we do this is a good one
Nuria added a parent task: T138207: [Open question] Improve bot identification at scale.
TASK DETAILhttps://phabricator.wikimedia.org/T199517EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, NuriaCc: Nuria, Aklapper, Lydia_Pintscher, JAllemandou
Nuria reopened this task as "Stalled".
TASK DETAILhttps://phabricator.wikimedia.org/T199517EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, NuriaCc: Nuria, Aklapper, Lydia_Pintscher, JAllemandou, Addshore, Lahi, Gq86, GoranSMilovanovi
Nuria added a comment.
F23734550: Screen Shot 2018-07-13 at 12.43.07 PM.png
It coincides with a spike of pageviews from thailand, that seems like a bot accessing teh desktop size, will investigate a bit as to whether this bot was accepting cookies.TASK DETAILhttps://phabricator.wikimedia.org
Nuria added a comment.
@Jonas: do you want all requests to www.wikidata.org to be included, correct? Do you care about request to wikidata query service or anything else about the request at hand?TASK DETAILhttps://phabricator.wikimedia.org/T191022EMAIL PREFERENCEShttps
Nuria added subscribers: tstarling, bd808.Nuria added a comment.
Pinging @bd808 and @Fjalapeno and @tstarling per above comment.TASK DETAILhttps://phabricator.wikimedia.org/T209031EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: bd808, tstarling
Nuria added a comment.
Added annotation for this event to wikidata unique devices data on wikistats: http://localhost:5000/dist/#/wikidata.org/reading/unique-devices/normal|line|All|~totalTASK DETAILhttps://phabricator.wikimedia.org/T199517EMAIL PREFERENCEShttps://phabricator.wikimedia.org
Nuria added a comment.
Misc is no longer in service, all requests have been migrated to 'text'TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Nuria, mpopov, chelsyx, Aklapper, Addshore, Smalyshev
Nuria reassigned this task from Nuria to mpopov.
TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopov, NuriaCc: Ottomata, elukey, Nuria, mpopov, chelsyx, Aklapper, Addshore, Smalyshev, Lydia_Pintscher
Nuria added a comment.
Assigned to @mpopov Again, our apologies that the data sources are hardcoded like this. As I mentioned on our meeting abetter path to go forward would be using the tags for wdqs to identify the requests: https://github.com/wikimedia/analytics-refinery-source/blob/master
Nuria added a comment.
Having missed most of goals this quarter due to our mw woes i think this might need to be moved to next quarter (q4?)TASK DETAILhttps://phabricator.wikimedia.org/T209655EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: Ottomata
Nuria removed a project: Analytics-Legal.
TASK DETAILhttps://phabricator.wikimedia.org/T193728EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NuriaCc: ChristianKl, Alsee, Aklapper, Huji, ArthurPSmith, SimonPoole, Scott_WorldUnivAndSch, Micru, lisong, Lofhi
Nuria added a comment.
You would need a reconstruction that is property-aware, the current one knows
only about pages and revisions. So, with different parameters for what the
reconstruction is doing yes, possible.
TASK DETAIL
https://phabricator.wikimedia.org/T217324
EMAIL PREFERENCES
Nuria added a comment.
@Smalyshev ah, i see what you mean now but I am still of the opinion that the
user should report the query that failed. On our end we can run it and retrieve
the stack trace. Our 500 page could include helpful link to phabricator to
report query that failed.
Maybe I
Nuria added a comment.
@Smalyshev if we configure the error logger to print requests and stack
traces (however deep) we can have alarming on them which would give us a
measure of errors (maybe we already have this). Relying on users to report
stack traces does not seem like it would give
Nuria added a comment.
@Ramsey-WMF Could we possibly get a bit more structured use cases?
Are those documented somewhere besides this ticket so we can see how this use case fits on the big picture? Is there any UI that goes with this case?TASK DETAILhttps://phabricator.wikimedia.org/T215967EMAIL
Nuria moved this task from Incoming to Smart Tools for Better Data on the Analytics board.Nuria triaged this task as "High" priority.
TASK DETAILhttps://phabricator.wikimedia.org/T215616WORKBOARDhttps://phabricator.wikimedia.org/project/board/11/EMAIL PREFERENCEShttps://phabricator.wik
Nuria added a comment.
@bmansurov I think you need to consider also couple more things: a list of links can be very lengthy, do we have a limit for how much this field should occupy? Are links url encoded? (we probably want them to be so).TASK DETAILhttps://phabricator.wikimedia.org/T214706EMAIL
Nuria added a comment.
@bmansurov ah I think I understand what you meant, now sorry: if mediawiki cannot generate the diff you are interested on at the time the page is edited you need to consume an event that happens later in the chain, ya, makes sense.TASK DETAILhttps
Nuria added a comment.
Clarifying: ChnageProp consumes EventBus data just like EventStreams consumes EventBus data. So you cannot "use" changeprop rather you will be sending events to EventBus (soon to be called EventGate) and consuming them from elsewhere and in turn exposing them to
Nuria added a comment.
@Samwalton9 we still need to see if urls are url encoded or not and hook publishing to one of the mediawiki events (I think @bmansurov is doing this with @Pchelolo .help?) Once events are flowing and looking OK they can be set to be published to the outside world.TASK
Nuria moved this task from Incoming to Radar on the Analytics board.Nuria raised the priority of this task from "Normal" to "Needs Triage".
TASK DETAILhttps://phabricator.wikimedia.org/T214706WORKBOARDhttps://phabricator.wikimedia.org/project/board/11/EM
Nuria added a comment.
Not actively working on this now.TASK DETAILhttps://phabricator.wikimedia.org/T189744EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, NuriaCc: Nuria, Jonas, EBernhardson, gerritbot, Lydia_Pintscher, daniel, Aklapper, Smalyshev
Nuria renamed this task from "Provision sparql endpoint for SDC. Requirements
from Product Team." to "Provision search endpoint for SDC. Requirements from
Product Team.".
TASK DETAIL
https://phabricator.wikimedia.org/T221921
EMAIL PREFERENCES
https://phabricator.wi
Nuria added a comment.
@abian : this is still not happening on a recurrent schedule yet.
TASK DETAIL
https://phabricator.wikimedia.org/T209655
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: abian, leila, Ottomata, Nuria
Nuria closed subtask T161731: Create reliable change stream for specific wiki
as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T145712
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Lucas_Werkmeister_WMDE, Liuxinyu970226
Nuria closed this task as "Resolved".
TASK DETAIL
https://phabricator.wikimedia.org/T161731
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ottomata, Nuria
Cc: gerritbot, JAllemandou, Pchelolo, Ladsgroup, Nuria, Anomie, Aklapper,
Nuria added subscribers: Fjalapeno, Nuria.
Nuria added a comment.
pinging @Fjalapeno from your comments the other day I understand Wikidata is
going to use cassandra for these use cases at the end? cc @Addshore
TASK DETAIL
https://phabricator.wikimedia.org/T220823
EMAIL PREFERENCES
Nuria created this task.
Nuria added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service.
TASK DESCRIPTION
TASK DETAIL
https://phabricator.wikimedia.org/T221921
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: Smalyshev
Nuria closed subtask T227905: Public Data Review Needed as Resolved.
TASK DETAIL
https://phabricator.wikimedia.org/T208567
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: GoranSMilovanovic, Nuria
Cc: GoranSMilovanovic, Aklapper, WMDE-leszek, Lea_WMDE
Nuria added a comment.
yes, you can use
https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-hive/src/main/java/org/wikimedia/analytics/refinery/hive/GetHostPropertiesUDF.java
to get the "project/family"
TASK DETAIL
https://phabricator.wikimedia.org/T236
Nuria added a comment.
Ya, =1 to joseph, Special:blah urls (other than Special:Search) should not
have been counted as pageviews and since a fix on July they no longer are.
TASK DETAIL
https://phabricator.wikimedia.org/T236895
EMAIL PREFERENCES
https://phabricator.wikimedia.org
Nuria added a comment.
So this query needs to remove the is_pageview=true line:
https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/WikidataArticlePlaceholderMetrics.scala#L90
TASK DETAIL
https
Nuria added a comment.
@Addshore : disclaimer: I know next to nothing about this but how are you
taking into account that the revision is the last one for the page? That is, a
page might have had a structured data item in a prior revision and from its
most current revision that structured
Nuria added a comment.
So, per my comment above, I think the number of items is actually smaller
than the one @Addshore has computed but more wise folks can correct me if I am
wrong.
TASK DETAIL
https://phabricator.wikimedia.org/T238878
EMAIL PREFERENCES
https
Nuria updated the task description.
TASK DETAIL
https://phabricator.wikimedia.org/T238878
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Nuria
Cc: kzimmerman, mpopov, Ramsey-WMF, Abit, Nuria, 4748kitoko, darthmon_wmde,
DannyS712, Nandana, JKSTNK
Nuria added a comment.
Restricted Application added a project: Structured-Data-Backlog.
I see this ticket is resolved but the dumps on commons have version
version="0.10" since from this ticket i gather that the dumps that contain
those slots are version=11 , are those being produ
1 - 100 of 147 matches
Mail list logo
