[Wikidata-bugs] [Maniphest] [Merged] T149456: Wikidata description change doesn't trigger mobile-sections update

2017-01-31 Thread Pchelolo
Pchelolo added a subscriber: GWicke.Pchelolo merged a task: T148079: Narrow down on-wikidata-description-change reaction. TASK DETAILhttps://phabricator.wikimedia.org/T149456EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: GWicke, Niedzielski

[Wikidata-bugs] [Maniphest] [Commented On] T149114: Reconsider wikidata support in the REST API

2016-10-25 Thread Pchelolo
Pchelolo added a comment. According to sampled request log, the only requests to REST API on wikidata are made by #changeprop and #mobile-content-service (see https://logstash.wikimedia.org/goto/155383470f2b3307f82b91b34ad29889 ) which suggests there's no real users of this API. SO, +1

[Wikidata-bugs] [Maniphest] [Updated] T149456: Wikidata description change doesn't trigger mobile-sections update

2016-11-01 Thread Pchelolo
Pchelolo added a comment. @daniel Your input on this would be much appreciated. A more generic question is whether it's possible to detect what exactly was changed in the wikidata item from the RevisionInsertComplete hook? If there was a way, we could consider moving that detection

[Wikidata-bugs] [Maniphest] [Changed Project Column] T149114: Reconsider wikidata support in the REST API

2016-10-26 Thread Pchelolo
Pchelolo moved this task from next to doing on the Services board.Pchelolo edited projects, added Services (doing); removed Services (next). TASK DETAILhttps://phabricator.wikimedia.org/T149114WORKBOARDhttps://phabricator.wikimedia.org/project/board/69/EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T149114: Reconsider wikidata support in the REST API

2016-10-26 Thread Pchelolo
Pchelolo added a comment. I think we've got general agreement on this. Some code changes needs to be done here: Make #changeprop be able to ignore wikidata Delete wikidata from #restbase domain list TASK DETAILhttps://phabricator.wikimedia.org/T149114EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T175316: Very large jobs posted by Wikidata

2017-09-13 Thread Pchelolo
Pchelolo added a comment. Here's an example of a very large event: https://people.wikimedia.org/~ppchelko/large_event It's not an event itself, it's a log message from #eventbus but the event is embedded in the log message.TASK DETAILhttps://phabricator.wikimedia.org/T175316EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T175316: Very large jobs posted by Wikidata

2017-09-14 Thread Pchelolo
Pchelolo added a comment. @Pchelolo actually, can you confirm how many entries there were in the "pages" parameter? With the latest patches deployed, there should be no more than 20. Perhaps this is an old job getting retried, because it failed earlier? There's definitely more then

[Wikidata-bugs] [Maniphest] [Commented On] T175316: Very large jobs posted by Wikidata

2017-09-14 Thread Pchelolo
Pchelolo added a comment. After the patch was deployed the situation improved a lot, but we've got a 5 Mb event today: https://people.wikimedia.org/~ppchelko/event 5 Mb is not critically large, we can increase the limit in Kafka to 8 Mb I think, but maybe we should do some more improvements

[Wikidata-bugs] [Maniphest] [Closed] T175316: Very large jobs posted by Wikidata

2017-09-19 Thread Pchelolo
Pchelolo closed this task as "Resolved".Pchelolo added a comment. Verified that the jobs are now small enough to fit in our new infrastructure. Resolving.TASK DETAILhttps://phabricator.wikimedia.org/T175316EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailp

[Wikidata-bugs] [Maniphest] [Commented On] T175316: Very large jobs posted by Wikidata

2017-09-14 Thread Pchelolo
Pchelolo added a comment. In T175316#3608922, @GWicke wrote: Looks like adding the JSON_UNESCAPED_UNICODE flag should do it: http://php.net/manual/en/function.json-encode.php We use JsonFormatter class to encode JSON, so the solution is a bit different, but the above patch takes care

[Wikidata-bugs] [Maniphest] [Commented On] T175316: Very large jobs posted by Wikidata

2017-09-08 Thread Pchelolo
Pchelolo added a comment. In T175316#3591889, @GWicke wrote: @Pchelolo, based on our previous conversation about this I am assuming that the bulk of the task is a very large list of pages. Is this correct? Ye, in the actual event the params.pages array contains millions and millions

[Wikidata-bugs] [Maniphest] [Commented On] T172832: Investigate use-cases for delayed job executions

2017-09-07 Thread Pchelolo
Pchelolo added a comment. I wrote a little script to run through a sample of events in the job topics we have in prod right now and here's.a list of job types that had the releaseTimestamp set: mediawiki.job.cdnPurge mediawiki.job.cirrusSearchCheckerJob mediawiki.job.cirrusSearchElasticaWrite

[Wikidata-bugs] [Maniphest] [Created] T175316: Very large jobs posted by Wikidata

2017-09-07 Thread Pchelolo
Pchelolo created this task.Pchelolo added projects: Services (doing), Wikidata. TASK DESCRIPTIONAfter we've started posting the job events into #eventbus and Kafka, we've noticed that some of them were rejected by Kafka because of MESSAGE_SIZE_TOO_LARGE error. The limit was increased from 1 Mb

[Wikidata-bugs] [Maniphest] [Edited] T175316: Very large jobs posted by Wikidata

2017-09-07 Thread Pchelolo
Pchelolo updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...``` There's another example that is **44 Mb is size** serialized. Kafka is capable of handling that, but it's not great in dealing with very large messages, so we can't increase the cap indefinitely. Maybe there's

[Wikidata-bugs] [Maniphest] [Closed] T178997: AddUsagesForPageJob doesn't really report execution status

2017-12-07 Thread Pchelolo
Pchelolo closed this task as "Resolved".Pchelolo added a comment. This is not happening any more, thank you for resolving.TASK DETAILhttps://phabricator.wikimedia.org/T178997EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: thiemowmde,

[Wikidata-bugs] [Maniphest] [Commented On] T161731: Create reliable change stream for specific wiki

2017-12-07 Thread Pchelolo
Pchelolo added a comment. Hm, actually if I just try to consume from that topic (any topic actually) with -F "%T" that should give me message timestamps it gives -1 as well. I suppose that the problem is that we're actually producing these messages into Kafka 0.9 and perhaps not

[Wikidata-bugs] [Maniphest] [Commented On] T161731: Create reliable change stream for specific wiki

2017-12-05 Thread Pchelolo
Pchelolo added a comment. In T161731#3814596, @Smalyshev wrote: @Ottomata thanks, I can connect to the hosts above, but still not sure how to control the starting point. I'll try to look around for clients that can do this. Java client has offsetsForTimes implemented and supports seek

[Wikidata-bugs] [Maniphest] [Updated] T161731: Create reliable change stream for specific wiki

2017-12-05 Thread Pchelolo
Pchelolo added a comment. Indeed. Created T182163 for updating kafkacat but it is a bit more complex then simply installing new version - it depends on newer librdkafka, so we firs should upgrade and test that.TASK DETAILhttps://phabricator.wikimedia.org/T161731EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Created] T178997: AddUsagesForPageJob doesn't really report execution status

2017-10-25 Thread Pchelolo
Pchelolo created this task.Pchelolo added a project: Wikidata.Herald added a subscriber: Aklapper. TASK DESCRIPTIONEven though the documentation says the job returns a bool reporting execution success/failure, it actually returns nothingTASK DETAILhttps://phabricator.wikimedia.org/T178997EMAIL

[Wikidata-bugs] [Maniphest] [Updated] T178997: AddUsagesForPageJob doesn't really report execution status

2017-10-25 Thread Pchelolo
Pchelolo edited subscribers, added: daniel; removed: Aklapper.Pchelolo added a project: Services (watching). TASK DETAILhttps://phabricator.wikimedia.org/T178997EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: daniel, Pchelolo, GoranSMilovanovic

[Wikidata-bugs] [Maniphest] [Commented On] T178997: AddUsagesForPageJob doesn't really report execution status

2017-10-25 Thread Pchelolo
Pchelolo added a comment. Same happens with UpdateConstraintsTableJobTASK DETAILhttps://phabricator.wikimedia.org/T178997EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: gerritbot, daniel, Pchelolo, Lahi, Lordiis, GoranSMilovanovic, Adik2382

[Wikidata-bugs] [Maniphest] [Updated] T187296: Increase kafka event retention to 31

2018-06-12 Thread Pchelolo
Pchelolo added a project: Services (watching). TASK DETAILhttps://phabricator.wikimedia.org/T187296EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, PcheloloCc: mforns, elukey, Ottomata, Aklapper, Nuria, Ladsgroup, Pchelolo, JAllemandou, Smalyshev

[Wikidata-bugs] [Maniphest] [Commented On] T187296: Increase kafka event retention to 31

2018-06-12 Thread Pchelolo
Pchelolo added a comment. If we add a new mediawiki topic, we need to remember to run this command for it. Or implement T157092 :)TASK DETAILhttps://phabricator.wikimedia.org/T187296EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ottomata, PcheloloCc: mforns

[Wikidata-bugs] [Maniphest] [Reopened] T178997: AddUsagesForPageJob doesn't really report execution status

2017-10-27 Thread Pchelolo
Pchelolo reopened this task as "Open".Pchelolo added a comment. That gerrit was a workaround, the real issue is that the jobs don't follow the Job contract. Reopening.TASK DETAILhttps://phabricator.wikimedia.org/T178997EMAIL PREFERENCEShttps://phabricator.wikimedia.org/sett

[Wikidata-bugs] [Maniphest] [Commented On] T187296: Increase kafka event retention to 14 or 21 days

2018-02-16 Thread Pchelolo
Pchelolo added a comment. Currently on kafka-main machines the disk utiliation is really low, so I think we can easily do it without kafka-jumbo. dsh --group kafka_eqiad --remoteshell ssh -- 'df -h' | grep '/srv' /dev/md17.2T 737G 6.1T 11% /srv /dev/md17.2T 736G 6.1T 11

[Wikidata-bugs] [Maniphest] [Updated] T187296: Increase kafka event retention to 14 or 21 days

2018-02-16 Thread Pchelolo
Pchelolo added a comment. Currently on kafka-main machines the disk utiliation is really low, so I think we can easily do it without kafka-jumbo. dsh --group kafka_eqiad --remoteshell ssh -- 'df -h' | grep '/srv' /dev/md17.2T 737G 6.1T 11% /srv /dev/md17.2T 736G 6.1T 11

[Wikidata-bugs] [Maniphest] [Changed Project Column] T200594: Add client identifier to requests sent from Kartotherian to WDQS

2018-07-31 Thread Pchelolo
Pchelolo moved this task from Backlog to watching on the Services board.Pchelolo edited projects, added Services (watching); removed Services. TASK DETAILhttps://phabricator.wikimedia.org/T200594WORKBOARDhttps://phabricator.wikimedia.org/project/board/69/EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Changed Project Column] T199096: Add support for wikidata summaries in the /page/summary/ endpoint

2018-07-09 Thread Pchelolo
Pchelolo moved this task from Backlog to watching on the Services board.Pchelolo edited projects, added Services (watching); removed Services. TASK DETAILhttps://phabricator.wikimedia.org/T199096WORKBOARDhttps://phabricator.wikimedia.org/project/board/69/EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T199096: Add support for wikidata summaries in the /page/summary/ endpoint

2018-07-09 Thread Pchelolo
Pchelolo added a comment. From the #services side there are 2 things to consider: Invalidation. Right now we do not have summaries for wikidata, so new rules will need to be added to #changeprop for cache invalidation. In case we include information like number of sitelinks etc, that's

[Wikidata-bugs] [Maniphest] [Closed] T172832: Investigate use-cases for delayed job executions

2018-04-12 Thread Pchelolo
Pchelolo closed this task as "Resolved".Pchelolo edited projects, added Services (done); removed Epic, Services (doing).Pchelolo added a comment. We've already found all the current use-cases and implemented support for delayed jobs, so resolving.TASK DETAILhttps://phabricator.wik

[Wikidata-bugs] [Maniphest] [Created] T192085: PHP Fatal in AffectedPagesFinder::getChangedAspects

2018-04-12 Thread Pchelolo
Pchelolo created this task.Pchelolo added projects: MediaWiki-extensions-WikibaseClient, Services (watching), Wikidata, JobRunner-Service.Herald added a subscriber: Aklapper. TASK DESCRIPTIONTrying to execute a ChangeNotification job with the following parameters: {"database":&qu

[Wikidata-bugs] [Maniphest] [Commented On] T207817: WDQS Updater ran into issue and stopped working

2018-10-24 Thread Pchelolo
Pchelolo added a comment. Maybe jackson just can't parse this microsecond stuff? Maybe milliseconds are fine? Judging by the error message, it cripples on the . after seconds. @Smalyshev Also please note we're only including microseconds and not milliseconds because of the HHVM incompatibility

[Wikidata-bugs] [Maniphest] [Commented On] T207817: WDQS Updater ran into issue and stopped working

2018-10-24 Thread Pchelolo
Pchelolo added a comment. Judging by the quick skim of the Jackson code it supports DateTimeFormatter.ISO_INSTANT format that should support fractional seconds parsing. However, this incident raises a question whether the change applied to EventBus was indeed backward-compatible. We've checked

[Wikidata-bugs] [Maniphest] [Commented On] T207817: WDQS Updater ran into issue and stopped working

2018-10-24 Thread Pchelolo
Pchelolo added a comment. We could rollback https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventBus/+/468482/ - we can workaround it in change-prop and the issue fixed by that is much rarer, but eventually, we'd want to include millisecond resolution into the dt of the events.TASK

[Wikidata-bugs] [Maniphest] [Commented On] T204031: Deploy regular running of wikidata constraint checks using the job queue

2019-01-16 Thread Pchelolo
Pchelolo added a comment. I've noticed some errors in the JobQueue with the message like 'Cannot instantiate job 'constraintsRunCheck': bad spec!' Here's the job: { "database": "wikidatawiki", "mediawiki_signature": "5de8d10b6415ec7910a231f5234dd

[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty

2018-11-27 Thread Pchelolo
Pchelolo added a comment. Do you need the events for the last month to be replayed?TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mobrovac, PcheloloCc: Stashbot, gerritbot, MisterSynergy, mobrovac

[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty

2018-11-26 Thread Pchelolo
Pchelolo added a comment. Not to be worried. We have all the failed events stored since 2018-04-18. If needed, I will fetch all the missing page deletes tomorrow.TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty

2018-11-26 Thread Pchelolo
Pchelolo added a comment. Yes, the fix has not been deployed yet. Asked to put it on Euro Mid-day SWATTASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: TerraCodes, Liuxinyu970226, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T204031: Deploy regular running of wikidata constraint checks using the job queue

2018-09-12 Thread Pchelolo
Pchelolo added projects: WMF-JobQueue, Services (watching).Pchelolo added a comment. Just to make sure - jobs now MUST be JSON-serializable, so please keep that in mind from the very beginning (no PHP classes in the job). Plus please ping #services when deploying it if the anticipated rate

[Wikidata-bugs] [Maniphest] [Commented On] T204031: Deploy regular running of wikidata constraint checks using the job queue

2019-01-22 Thread Pchelolo
Pchelolo added a comment. Default low volume queue concurrency is 50 per https://github.com/wikimedia/mediawiki-services-change-propagation-jobqueue-deploy/blob/master/scap/vars.yaml#L101 So getting this to 100% without hitting the limit should be no problem. However, those 50 slots are shared

[Wikidata-bugs] [Maniphest] [Commented On] T204031: Deploy regular running of wikidata constraint checks using the job queue

2019-01-22 Thread Pchelolo
Pchelolo added a comment. Change-Prop now has a separate rule for this job, so it doesn't interfere with low-traffic jobs. The concurrency is set to 30, which should be enough, however after deploying 100% we need to monitor the lag and increase concurrency if we see the job is steadily

[Wikidata-bugs] [Maniphest] [Commented On] T214706: Surface link changes as a stream

2019-02-07 Thread Pchelolo
Pchelolo added a comment. We're getting closer. The last part of the puzzle would be to emit the event publically via event streams. In order to do that, we need to add the mediawiki.page-links-change topic to the list of exposed topics in puppet for production and for deployment-prepTASK

[Wikidata-bugs] [Maniphest] [Closed] T214706: Surface link changes as a stream

2019-02-14 Thread Pchelolo
Pchelolo edited projects, added Services (done), Core Platform Team Kanban (Done with CPT); removed Patch-For-Review, Core Platform Team Kanban (Blocked Externally), Services (blocked).Pchelolo closed this task as "Resolved".Pchelolo claimed this task.Pchelolo added a comment. MW trai

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo added a comment. Are links url encoded? (we probably want them to be so). I agree with @Nuria on this. The 'meta.uri' is encoded, we need to be consistent. Whether the LinksUpdate in MW has encoded or not encoded links - needs to be tested.TASK DETAILhttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo added a comment. The task at hand is very easy. There's LinksUpdateComplete hook in MW core, it gets the LinksUpdate which contains the list of external links and methods to get links insertions and removals. Writing code in EventBus extension to use the hook is simple. The important part

[Wikidata-bugs] [Maniphest] [Updated] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo edited projects, added Services (blocked); removed Services.Pchelolo added a comment. Moving to services-blocked until #research comes up with the schema for the event.TASK DETAILhttps://phabricator.wikimedia.org/T214706EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo added a comment. @bmansurov look at the event schema for properties change I've linked, it contains all of the metadata you need and it's generated from the same hook. Will such a schema work for you if we replace removed_properties with removed_links and added_properties with added_links

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo added a comment. @Pchelolo yes, replacing "properties" with "links" should do it. I don't think we need the full list of links, just the changes. Perfect. Just make a change request in gerrit with a new schema. I can point you to where to write the code for emitting t

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-01-25 Thread Pchelolo
Pchelolo added a comment. How should we handle the situation with long list of links? Is that really a problem? Should we create multiple events? You should not worry about that unless you encounter a list of links that's > 4mb long :)TASK DETAILhttps://phabricator.wikimedia.org/T214706EM

[Wikidata-bugs] [Maniphest] [Commented On] T214706: How to surface link changes as a stream?

2019-02-05 Thread Pchelolo
Pchelolo added a comment. We have discovered that we would need to update LinkUpdates class in the core to support this functionality, so it will take a little bit more time then we anticipated, but we're making progress.TASK DETAILhttps://phabricator.wikimedia.org/T214706EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T205045: Exception from LinksUpdate: Deadlock found in database query (from Wikibase\Client\Usage\Sql\EntityUsageTable::addUsages)

2019-06-27 Thread Pchelolo
Pchelolo added a comment. In #changeprop <https://phabricator.wikimedia.org/tag/changeprop/> for job queue, we have a concept of concurrency - how many jobs of the same type can be running in parallel. `wikibase-addUsagesForPage` had a special rule with concurrency 30, so if this p

[Wikidata-bugs] [Maniphest] [Commented On] T205045: Exception from LinksUpdate: Deadlock found in database query (from Wikibase\Client\Usage\Sql\EntityUsageTable::addUsages)

2019-06-27 Thread Pchelolo
Pchelolo added a comment. OMG! ❤️ TASK DETAIL https://phabricator.wikimedia.org/T205045 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Pchelolo Cc: Joe, mobrovac, Pchelolo, Michael, Lydia_Pintscher, jcrespo, Ladsgroup, hoo

[Wikidata-bugs] [Maniphest] [Updated] T198343: Replace all calls to Revision::getRevisionText()

2019-09-18 Thread Pchelolo
Pchelolo added a comment. The last usage of the method within core (apart from maintenance scripts) is in ApiQueryDeletedrevs which was deprecated since 1.25. Quick search reveals that it's still used in pywikibot ( there's a ticket about that T75370 <https://phabricator.wikimedia.

[Wikidata-bugs] [Maniphest] [Commented On] T228343: Huge delay for RecentChanges in test.wikidata

2019-07-22 Thread Pchelolo
Pchelolo added a comment. On the JobQueue side, nothing specific is observed neither on graphs nor logs for `recentChangesUpdate`. TASK DETAIL https://phabricator.wikimedia.org/T228343 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc

[Wikidata-bugs] [Maniphest] [Unblock] T198342: Remove all usages of the 'text' flag in calls to Revision::getQueryInfo() and RevisionStore::getQueryInfo().

2019-09-19 Thread Pchelolo
Pchelolo closed subtask T233356: Flow: remove dependency on pre-MCR database schema as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198342 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Daimona, Nikerabbit, BPirkle

[Wikidata-bugs] [Maniphest] [Unblock] T198341: Remove all references to the rev_text_id and ar_text_id fields

2019-09-19 Thread Pchelolo
Pchelolo closed subtask T233356: Flow: remove dependency on pre-MCR database schema as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: holger.knust, Pchelolo Cc: BPirkle, tstarling

[Wikidata-bugs] [Maniphest] [Closed] T233353: EntitySchema: remove dependency on pre-MCR database schema

2019-09-27 Thread Pchelolo
Pchelolo closed this task as "Resolved". Pchelolo moved this task from Doing to Done on the Core Platform Team Workboards (Purple) board. TASK DETAIL https://phabricator.wikimedia.org/T233353 WORKBOARD https://phabricator.wikimedia.org/project/board/4181/ EMAIL PREFERENC

[Wikidata-bugs] [Maniphest] [Unblock] T198341: Remove all references to the rev_text_id and ar_text_id fields

2019-09-27 Thread Pchelolo
Pchelolo closed subtask T233353: EntitySchema: remove dependency on pre-MCR database schema as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: holger.knust, BPirkle

[Wikidata-bugs] [Maniphest] [Unblock] T198342: Remove all usages of the 'text' flag in calls to Revision::getQueryInfo() and RevisionStore::getQueryInfo().

2019-09-23 Thread Pchelolo
Pchelolo closed subtask T233358: SpamBlacklist: remove dependency on pre-MCR database schema as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198342 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Daimona, Nikerabbit, BPirkle

[Wikidata-bugs] [Maniphest] [Closed] T198343: Replace all calls to Revision::getRevisionText()

2019-10-07 Thread Pchelolo
Pchelolo closed this task as "Resolved". Pchelolo moved this task from Blocked Externally to Done on the Core Platform Team Workboards (Clinic Duty Team) board. Pchelolo claimed this task. Pchelolo added a comment. Translate extension is done, all usages has been either remove

[Wikidata-bugs] [Maniphest] [Unblock] T198341: Remove all references to the rev_text_id and ar_text_id fields

2019-10-07 Thread Pchelolo
Pchelolo closed subtask T228675: Remove direct access to the text table from the Translate extension. as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: holger.knust

[Wikidata-bugs] [Maniphest] [Unblock] T198343: Replace all calls to Revision::getRevisionText()

2019-10-07 Thread Pchelolo
Pchelolo closed subtask T228675: Remove direct access to the text table from the Translate extension. as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198343 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Pchelolo, Clarakosi

[Wikidata-bugs] [Maniphest] [Unblock] T198342: Remove all usages of the 'text' flag in calls to Revision::getQueryInfo() and RevisionStore::getQueryInfo().

2019-10-07 Thread Pchelolo
Pchelolo closed subtask T228675: Remove direct access to the text table from the Translate extension. as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198342 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Daimona, Nikerabbit

[Wikidata-bugs] [Maniphest] [Unblock] T198341: Remove all references to the rev_text_id and ar_text_id fields

2019-10-07 Thread Pchelolo
Pchelolo closed subtask T198343: Replace all calls to Revision::getRevisionText() as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: holger.knust, BPirkle, tstarling

[Wikidata-bugs] [Maniphest] [Unblock] T198341: Remove all references to the rev_text_id and ar_text_id fields

2019-10-07 Thread Pchelolo
Pchelolo closed subtask T198342: Remove all usages of the text flag in calls to Revision::getQueryInfo() and RevisionStore::getQueryInfo(). as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T198341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Closed] T198342: Remove all usages of the 'text' flag in calls to Revision::getQueryInfo() and RevisionStore::getQueryInfo().

2019-10-07 Thread Pchelolo
Pchelolo closed this task as "Resolved". Pchelolo moved this task from Blocked to Done on the Core Platform Team Workboards (Purple) board. Pchelolo claimed this task. Pchelolo added a comment. With the completion of the Translate extension, all references has been either remove

[Wikidata-bugs] [Maniphest] [Commented On] T244039: Api tests: Hard deprecate $this->doLogin, remove calls in favor of passing a user where needed

2020-02-14 Thread Pchelolo
Pchelolo added a comment. All uses have been eliminated. TASK DETAIL https://phabricator.wikimedia.org/T244039 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: DannyS712, Pchelolo Cc: Pchelolo, DannyS712, Aklapper, Daimona, Un1tY, Hook696

[Wikidata-bugs] [Maniphest] [Edited] T244927: Hard deprecate Title::quickUserCan

2020-02-12 Thread Pchelolo
Pchelolo updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T244927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: DannyS712, Pchelolo Cc: Aklapper, DannyS712, Liuxinyu970226, Un1tY, keithbrianpadilla, Hook696, Adidsone1

[Wikidata-bugs] [Maniphest] [Commented On] T247058: Deployment strategy and hardware requirement for new Flink based WDQS updater

2020-03-06 Thread Pchelolo
Pchelolo added a comment. Yeah, @Gehel analysis is correct - change-prop is pretty simple and doesn't support any of the advanced features pointed in the task. We have never really needed any of those, we mostly rely on the fact that systems updated by change-prop are idempotent and don't

[Wikidata-bugs] [Maniphest] [Unblock] T238972: switch xml/sql (and adds-changes) dumps to use 0.11 schema with content from multiple slots

2020-04-06 Thread Pchelolo
Pchelolo closed subtask T240213: Write integration tests for XML dumps with multiple MCR slots per revision as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T238972 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ArielGlenn, Pchelolo Cc

[Wikidata-bugs] [Maniphest] [Updated] T245535: API "Content-Type" header warning

2020-03-26 Thread Pchelolo
Pchelolo edited projects, added Core Platform Team Workboards (External Code Reviews); removed Core Platform Team. TASK DETAIL https://phabricator.wikimedia.org/T245535 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pablo-WMDE, Pchelolo Cc

[Wikidata-bugs] [Maniphest] [Block] T238972: switch xml/sql (and adds-changes) dumps to use 0.11 schema with content from multiple slots

2020-03-27 Thread Pchelolo
Pchelolo reopened subtask T240213: Write integration tests for XML dumps with multiple MCR slots per revision as Open. TASK DETAIL https://phabricator.wikimedia.org/T238972 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ArielGlenn, Pchelolo Cc

[Wikidata-bugs] [Maniphest] [Unblock] T238972: switch xml/sql (and adds-changes) dumps to use 0.11 schema with content from multiple slots

2020-03-27 Thread Pchelolo
Pchelolo closed subtask T240213: Write integration tests for XML dumps with multiple MCR slots per revision as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T238972 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ArielGlenn, Pchelolo Cc

[Wikidata-bugs] [Maniphest] [Commented On] T248459: Math extension randomly fails in gate-and-submit for Wikibase

2020-04-01 Thread Pchelolo
Pchelolo added a comment. These are caused by math extension not being able to connect to RESTBase from CI. I think this is a separate issue from the wiki base one. I somehow don't believe the problem is in RESTBase itself - if math rendering was failing in RESTBase with such frequency

[Wikidata-bugs] [Maniphest] [Updated] T248459: Math extension randomly fails in gate-and-submit for Wikibase

2020-04-02 Thread Pchelolo
Pchelolo edited projects, added Core Platform Team Workboards (Clinic Duty Team); removed Core Platform Team. TASK DETAIL https://phabricator.wikimedia.org/T248459 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: WDoranWMF, Pchelolo

[Wikidata-bugs] [Maniphest] [Commented On] T251457: LoadBalancer: Transaction spent [n] second(s) in writes, exceeding the limit of [n]

2020-04-30 Thread Pchelolo
Pchelolo added a comment. @daniel I don't think this is that lock showing up here. Reading the code, it seems like if that lock was taken, we should've seen a bunch more queries in this transaction, like some deletes, read from the archive table etc. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Commented On] T251457: LoadBalancer: Transaction spent [n] second(s) in writes, exceeding the limit of [n]

2020-04-30 Thread Pchelolo
Pchelolo added a comment. Looking at the requestId of one of these errors, the following log messages are found: https://logstash.wikimedia.org/goto/3e387a16e01ae0f02e72b413d6691539 The interesting one there is: Sub-optimal transaction on DB(s) [10.64.16.34 (frwiki) (TRX#7ad8fc

[Wikidata-bugs] [Maniphest] [Commented On] T251457: LoadBalancer: Transaction spent [n] second(s) in writes, exceeding the limit of [n]

2020-04-30 Thread Pchelolo
Pchelolo added a comment. The other associated log actually gives more insight on which lock is making the transaction timeout: Expectation (writeQueryTime <= 1) by MediaWiki::main not met (actual: 3.366758108139): query-m: SELECT GET_LOCK('X', N) AS lockstatus [TRX#7ad8fc]

[Wikidata-bugs] [Maniphest] [Commented On] T251457: LoadBalancer: Transaction spent [n] second(s) in writes, exceeding the limit of [n]

2020-04-30 Thread Pchelolo
Pchelolo added a comment. Complete stack trace: Expectation (writeQueryTime <= 1) by MediaWiki::main not met (actual: 3.366758108139): query-m: SELECT GET_LOCK('X', N) AS lockstatus [TRX#7ad8fc] #0 /srv/mediawiki/php-1.35.0-wmf.30/includes/libs/rdbms/TransactionProfiler.

[Wikidata-bugs] [Maniphest] [Commented On] T251457: LoadBalancer: Transaction spent [n] second(s) in writes, exceeding the limit of [n]

2020-04-30 Thread Pchelolo
Pchelolo added a comment. It's PageEditStash. See my comment above. TASK DETAIL https://phabricator.wikimedia.org/T251457 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Agusbou2015, Pchelolo, daniel, DannyS712, Jdforrester-WMF

[Wikidata-bugs] [Maniphest] T259625: Contributor sets page description

2020-09-11 Thread Pchelolo
Pchelolo added a comment. Setting the description <https://www.wikidata.org/w/api.php?action=help=wbsetdescription> in Wikidata requires a language parameter. We can infer the language from the domain, however for wikis with language variants the user should be specifying the vari

[Wikidata-bugs] [Maniphest] T259624: Reader gets page description

2020-09-09 Thread Pchelolo
Pchelolo set the point value for this task to "1". TASK DETAIL https://phabricator.wikimedia.org/T259624 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: eprodromou, Pchelolo Cc: Aklapper, eprodromou, Naike, Akuckartz, apaskulin, dar

[Wikidata-bugs] [Maniphest] T259625: Contributor sets page description

2020-09-09 Thread Pchelolo
Pchelolo set the point value for this task to "1". TASK DETAIL https://phabricator.wikimedia.org/T259625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: eprodromou, Pchelolo Cc: Pchelolo, Aklapper, eprodromou, Naike, Akuckartz,

[Wikidata-bugs] [Maniphest] T259617: Local Description API

2020-09-09 Thread Pchelolo
Pchelolo set the point value for this task to "5". TASK DETAIL https://phabricator.wikimedia.org/T259617 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: eprodromou, Pchelolo Cc: kaldari, Pchelolo, daniel, Aklapper, eprodromou, Naike,

[Wikidata-bugs] [Maniphest] T259626: Contributor deletes page description

2020-09-09 Thread Pchelolo
Pchelolo set the point value for this task to "1". TASK DETAIL https://phabricator.wikimedia.org/T259626 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: eprodromou, Pchelolo Cc: Aklapper, eprodromou, Naike, Akuckartz, apaskulin, dar

[Wikidata-bugs] [Maniphest] T259624: Reader gets page description

2020-09-09 Thread Pchelolo
Pchelolo claimed this task. Pchelolo moved this task from Tasks Ready for Estimation to Doing on the Platform Team Workboards (Green) board. TASK DETAIL https://phabricator.wikimedia.org/T259624 WORKBOARD https://phabricator.wikimedia.org/project/board/4180/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T259626: Contributor deletes page description

2020-09-09 Thread Pchelolo
Pchelolo moved this task from Tasks Ready for Estimation to Doing on the Platform Team Workboards (Green) board. Pchelolo claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T259626 WORKBOARD https://phabricator.wikimedia.org/project/board/4180/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T259625: Contributor sets page description

2020-09-09 Thread Pchelolo
Pchelolo moved this task from Tasks Ready for Estimation to Doing on the Platform Team Workboards (Green) board. Pchelolo claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T259625 WORKBOARD https://phabricator.wikimedia.org/project/board/4180/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T259624: Reader gets page description

2020-09-09 Thread Pchelolo
Pchelolo added a comment. What about language variants? I think we will support language variants via `accept-language` header. TASK DETAIL https://phabricator.wikimedia.org/T259624 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc

[Wikidata-bugs] [Maniphest] T259622: System Administrator disables Wikidata fallback for article descriptions

2020-09-15 Thread Pchelolo
Pchelolo moved this task from Tasks Ready for Estimation to Doing on the Platform Team Workboards (Green) board. Pchelolo claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T259622 WORKBOARD https://phabricator.wikimedia.org/project/board/4180/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T263493: Reader gets appropriate article short description in search results

2020-10-18 Thread Pchelolo
Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo moved this task from Documentation to PM Sign-off on the Platform Team Workboards (Green) board. Pchelolo added a comment. I don't think there's anything to document here. TASK DETAIL https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T263493: Reader gets appropriate article short description in search results

2020-10-06 Thread Pchelolo
Pchelolo moved this task from Ready to Deploy to User Story Review on the Platform Team Workboards (Green) board. Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo added a comment. This is deployed and ready. TASK DETAIL https://phabricator.wikimedia.org/T263493

[Wikidata-bugs] [Maniphest] T259622: System Administrator disables Wikidata fallback for article descriptions

2020-10-06 Thread Pchelolo
Pchelolo moved this task from Ready to Deploy to User Story Review on the Platform Team Workboards (Green) board. Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo added a comment. This is deployed and ready. TASK DETAIL https://phabricator.wikimedia.org/T259622

[Wikidata-bugs] [Maniphest] T263493: Reader gets appropriate article short description in search results

2020-10-14 Thread Pchelolo
Pchelolo added a comment. Now wmf.11 was redeployed, so now it should be ok to test. TASK DETAIL https://phabricator.wikimedia.org/T263493 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pchelolo Cc: Aklapper, daniel, Pchelolo, kaldari

[Wikidata-bugs] [Maniphest] T259625: Contributor sets page description

2020-08-19 Thread Pchelolo
Pchelolo added a comment. - https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Short_descriptions notes that's it's recommended to mention the source of the description in the edit summary. Should we support optionally providing the page summary? - The template on enwiki <ht

[Wikidata-bugs] [Maniphest] T259622: System Administrator disables Wikidata fallback for article descriptions

2020-08-20 Thread Pchelolo
Pchelolo added a comment. WikiBase client, that implements Action API for descriptions uses `allowLocalShortDesc` config variable. Whatever config we introduce mustn't duplicate that one. TASK DETAIL https://phabricator.wikimedia.org/T259622 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T259617: Local Description API

2020-08-20 Thread Pchelolo
Pchelolo added a comment. Perhaps the implementation of this API should go into an extension as well, like `HorriblyHackyWikimediaFeatures` extension? So that we do not pollute the core with this until we have proper backend support for page descriptions from the data model level upwards

[Wikidata-bugs] [Maniphest] T259625: Contributor sets page description

2020-09-29 Thread Pchelolo
Pchelolo moved this task from Blocked to User Story Review on the Platform Team Workboards (Green) board. Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo added a comment. Done: curl -XPUT -H 'Content-Type: application/json' -H 'Authorization: Bearer $TOKEN' -H

[Wikidata-bugs] [Maniphest] T259626: Contributor deletes page description

2020-09-29 Thread Pchelolo
Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo moved this task from Blocked to User Story Review on the Platform Team Workboards (Green) board. Pchelolo added a comment. See T259625#6502392 <https://phabricator.wikimedia.org/T259625#6502392> for some handy CURLs

[Wikidata-bugs] [Maniphest] T259624: Reader gets page description

2020-09-29 Thread Pchelolo
Pchelolo reassigned this task from Pchelolo to eprodromou. Pchelolo added a comment. Done: enwiki: curl https://api.wikimedia.org/core/v1/wikipedia/en/page/San_Francisco/description {"description":"Consolidated city-county in California, US"} For lang-

  1   2   >