[Wikidata-bugs] [Maniphest] [Commented On] T251497: Adapt munging process for SDoC

2020-05-28 Thread dcausse
dcausse added a comment. I think that schema:ImageObject should be kept since we may have AudioObject and VideoObject TASK DETAIL https://phabricator.wikimedia.org/T251497 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse

[Wikidata-bugs] [Maniphest] [Commented On] T252091: RFC: Site-wide edit rate limiting with PoolCounter

2020-05-28 Thread dcausse
dcausse added a comment. In T252091#6171258 <https://phabricator.wikimedia.org/T252091#6171258>, @tstarling wrote: > Load is average queue size, if you take the currently running batch as being part of the queue. WDQS currently does not monitor the queue size. I gather (afte

[Wikidata-bugs] [Maniphest] [Edited] T254232: New clusters for SPARQL Endpoint for Commons

2020-06-02 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T254232 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Gehel, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] [Commented On] T243270: Test commons RDF dumps on sdcquery.wmflabs.org

2020-06-08 Thread dcausse
dcausse added a comment. We're waiting for https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/569260 to be deployed and will do another round of test. TASK DETAIL https://phabricator.wikimedia.org/T243270 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-06-15 Thread dcausse
dcausse added a subtask: T255399: Prepare wdqs1009 to run the streaming updater . TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill, darthmon_wmde

[Wikidata-bugs] [Maniphest] [Created] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-15 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION RDF Data is flowing in the wdqs_streaming_upater_test topic on the jumbo cluster. We should

[Wikidata-bugs] [Maniphest] [Updated] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-15 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] [Updated] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-15 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Blissjay007, Oblanco79, Alter-paule, Beast1978, CBogen

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-15 Thread dcausse
dcausse added a subscriber: Ottomata. dcausse added a comment. @Ottomata could we increase the retention period on the topic wdqs_streaming_updater_test on the jumbo cluster? We need extra time as this is first time we assemble all components together and we may run out of time to start

[Wikidata-bugs] [Maniphest] [Edited] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-15 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Blissjay007, Oblanco79, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, Daryl

[Wikidata-bugs] [Maniphest] [Closed] T230588: Wikidata Query Service is swapping items and properties

2020-06-17 Thread dcausse
dcausse added subscribers: RKemper, Zbyszko, dcausse. dcausse closed this task as "Resolved". dcausse moved this task from Waiting to Done on the Discovery-Search (Current work) board. dcausse reassigned this task from Zbyszko to RKemper. dcausse added a comment. @agray indeed, @R

[Wikidata-bugs] [Maniphest] [Claimed] T255657: Strange result in Wikidata query

2020-06-17 Thread dcausse
dcausse claimed this task. dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T255657 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Lucas_Werkmeister_WMDE, Aklapper, Epidosis, CBogen

[Wikidata-bugs] [Maniphest] [Lowered Priority] T255657: Strange result in Wikidata query

2020-06-17 Thread dcausse
dcausse lowered the priority of this task from "High" to "Medium". dcausse added a comment. Still trying to figure out what has happened. - tested the munger locally and it does not produce this - checked the dump that were to load the data and it looked OK - re

[Wikidata-bugs] [Maniphest] [Triaged] T255753: Wikidata Query Servie MWAPI calls with bd:serviceParam wikibase:limit "once" . executes 2 API calls instead of 1

2020-06-21 Thread dcausse
dcausse added a project: Discovery-Search (Current work). dcausse triaged this task as "Medium" priority. dcausse moved this task from All WDQS-related tasks to Small Bugs on the Wikidata-Query-Service board. TASK DETAIL https://phabricator.wikimedia.org/T255753 WORKBOA

[Wikidata-bugs] [Maniphest] [Triaged] T255658: PHP Warning: {"type":"error", "message":"cirrussearch-backend-error", "params":[]} [Called from Wikibase\Search\Elastic\EntitySearchElastic::getRankedSea

2020-06-21 Thread dcausse
dcausse moved this task from needs triage to Wikidata Search on the Discovery-Search board. dcausse triaged this task as "Medium" priority. dcausse added a comment. I think that it's not a new problem, I had the impression as well that it was already reported. This issue

[Wikidata-bugs] [Maniphest] [Updated] T252826: Whitelist x-wikimedia-debug header field (currently not allowed by Access-Control-Allow-Headers in preflight response)

2020-06-22 Thread dcausse
dcausse removed a project: Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T252826 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Tgr, RLazarus, Aklapper, mobrovac, Jseddon, Naike, CBogen, eprodromou, darthmon_wmde

[Wikidata-bugs] [Maniphest] [Updated] T249989: Disable native OpenSearch suggestions in Wikibase wikis

2020-06-22 Thread dcausse
dcausse removed a project: Discovery-Search. dcausse added a comment. Removing Discovery-Search as this is purely UI related. As noted in the description (T190454#4092077 <https://phabricator.wikimedia.org/T190454#4092077>) the ultimate goal would be to unify all these autocomplet

[Wikidata-bugs] [Maniphest] [Commented On] T198078: Extra xsd:decimal triples for some(?) globe coordinate values

2020-06-24 Thread dcausse
dcausse added a comment. Data has been reloaded, please reopen if you encounter the problem again. TASK DETAIL https://phabricator.wikimedia.org/T198078 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, dcausse Cc: dcausse, Mahir256, Gehel

[Wikidata-bugs] [Maniphest] [Edited] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-29 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Ottomata, Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, darthmon_wmde

[Wikidata-bugs] [Maniphest] [Edited] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-29 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Ottomata, Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, darthmon_wmde

[Wikidata-bugs] [Maniphest] [Assigned] T255399: Prepare wdqs1009 to run the streaming updater

2020-06-30 Thread dcausse
dcausse triaged this task as "Medium" priority. dcausse assigned this task to RKemper. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: Ottomata, Aklapper, dcausse, A

[Wikidata-bugs] [Maniphest] [Triaged] T248451: [WDQS Streaming Updater] Custom parallelism configuration for Streaming Updater

2020-07-01 Thread dcausse
dcausse triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T248451 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Zbyszko, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, dar

[Wikidata-bugs] [Maniphest] [Updated] T248451: [WDQS Streaming Updater] Custom parallelism configuration for Streaming Updater

2020-07-01 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T248451 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Zbyszko, Alter-paule, Beast1978, CBogen, Un1tY, Hook696

[Wikidata-bugs] [Maniphest] [Claimed] T248451: [WDQS Streaming Updater] Custom parallelism configuration for Streaming Updater

2020-07-01 Thread dcausse
dcausse claimed this task. dcausse moved this task from In Progress to Needs review on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T248451 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Updated] T251515: Automate data reload for SPARQL Endpoint for Commons

2020-07-01 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T251515 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles, dcausse Cc: Zbyszko, Aklapper, Lea_Lacroix_WMDE, Gehel, Alter-paule, Beast1978

[Wikidata-bugs] [Maniphest] [Closed] T246237: Extract some statistics on the use of the isBlank() function in wdqs query logs

2020-07-01 Thread dcausse
dcausse closed this task as "Resolved". dcausse added a comment. closing, will reopen if more details are needed TASK DETAIL https://phabricator.wikimedia.org/T246237 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JAllemandou,

[Wikidata-bugs] [Maniphest] [Unblock] T244341: Stop using blank nodes for encoding SomeValue and OWL constraints in WDQS

2020-07-01 Thread dcausse
dcausse closed subtask T246237: Extract some statistics on the use of the isBlank() function in wdqs query logs as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T244341 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse

[Wikidata-bugs] [Maniphest] [Created] T256875: The streaming updater should support page deletions

2020-07-01 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When an item is deleted the streaming updater should produce a message instructing the consumer to

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-01 Thread dcausse
dcausse added a subtask: T256875: The streaming updater should support page deletions. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] [Updated] T256875: The streaming updater should support page deletions

2020-07-01 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] [Created] T256882: The streaming updater should support page undeletes

2020-07-01 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When a deleted items is restored after a delete the streaming updater should produce an event with

[Wikidata-bugs] [Maniphest] [Updated] T256882: The streaming updater should support page undeletes

2020-07-01 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T256882 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-01 Thread dcausse
dcausse added a subtask: T256882: The streaming updater should support page undeletes. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] [Edited] T256882: The streaming updater should support page undeletes

2020-07-01 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T256882 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-02 Thread dcausse
dcausse added a subtask: T256946: Add a custom watermark generator that supports low rate streams. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup

[Wikidata-bugs] [Maniphest] [Updated] T256946: Add a custom watermark generator that supports low rate streams

2020-07-02 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T256946 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen

[Wikidata-bugs] [Maniphest] [Created] T256946: Add a custom watermark generator that supports low rate streams

2020-07-02 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When mixing incoming streams with different rate (e.g. one high volume stream and another with

[Wikidata-bugs] [Maniphest] [Created] T256948: Evaluate using LinkedDataFragments/Server.js instead of custom blazegraph integration

2020-07-02 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION The way we provide the LDF service is not scalable (T161240 <https://phabricator.wikimedia.

[Wikidata-bugs] [Maniphest] [Created] T256949: The streaming updater should support suppressed deletes

2020-07-02 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When an item is deleted the streaming updater should produce a message instructing the consumer to

[Wikidata-bugs] [Maniphest] [Updated] T256949: The streaming updater should support suppressed deletes

2020-07-02 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-02 Thread dcausse
dcausse added a subtask: T256949: The streaming updater should support suppressed deletes. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] [Edited] T256948: Evaluate using LinkedDataFragments/Server.js instead of custom blazegraph integration

2020-07-02 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T256948 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Edited] T256949: The streaming updater should support suppressed deletes

2020-07-02 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Edited] T256949: The streaming updater should support suppressed deletes

2020-07-02 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Edited] T256949: The streaming updater should support suppressed deletes

2020-07-02 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Commented On] T256949: The streaming updater should support suppressed deletes

2020-07-07 Thread dcausse
dcausse added a comment. @Bugreporter this task is about propagating the delete when an oversighter deletes a page with the //Suppress data from administrators as well as others// checkbox activated: F31919412: suppressed_delete.png <https://phabricator.wikimedia.org/F31919

[Wikidata-bugs] [Maniphest] [Created] T257325: Upgrade to flink 1.11

2020-07-07 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Update our dependencies to flink 1.11 and verify that everything works as expected

[Wikidata-bugs] [Maniphest] [Updated] T257325: Upgrade to flink 1.11

2020-07-07 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T257325 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] [Updated] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-07 Thread dcausse
dcausse added a subtask: T257325: Upgrade to flink 1.11. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill, darthmon_wmde, Iamamz3, Smalyshev

[Wikidata-bugs] [Maniphest] [Commented On] T248449: [WDQS Streaming Updater] Add error handling for Streaming Updater

2020-07-07 Thread dcausse
dcausse added a comment. This task should also try to uncover improper error handling in the shared code used by the steaming updater. One example is WikibaseRepository: when hitting a 404 while fetch RDF data it simply ignores this and returns an empty list of statements, this is far from

[Wikidata-bugs] [Maniphest] [Triaged] T257336: Request increased quota for wikidata-query Cloud VPS project

2020-07-08 Thread dcausse
dcausse triaged this task as "High" priority. dcausse added a comment. raising prio as this is blocking T251488 <https://phabricator.wikimedia.org/T251488> TASK DETAIL https://phabricator.wikimedia.org/T257336 EMAIL PREFERENCES https://phabricator.wikimedia.or

[Wikidata-bugs] [Maniphest] [Updated] T257336: Request increased quota for wikidata-query Cloud VPS project

2020-07-08 Thread dcausse
dcausse added a parent task: T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS. TASK DETAIL https://phabricator.wikimedia.org/T257336 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, RKemper, dcausse

[Wikidata-bugs] [Maniphest] [Updated] T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS

2020-07-08 Thread dcausse
dcausse added a subtask: T257336: Request increased quota for wikidata-query Cloud VPS project. TASK DETAIL https://phabricator.wikimedia.org/T251488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Smalyshev, Ramsey-WMF, Cparle

[Wikidata-bugs] [Maniphest] [Commented On] T221917: Create RDF dump of structured data on Commons

2020-07-09 Thread dcausse
dcausse added a comment. should `latest-full.*` be deleted to avoid confusion? TASK DETAIL https://phabricator.wikimedia.org/T221917 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ArielGlenn, dcausse Cc: DD063520, D063520, CBogen, nettrom_WMF

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-13 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION 18:28:30 - should break the pipeline *** FAILED *** 18:28:30 Unsatisfied expectation

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-13 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T257795 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-13 Thread dcausse
dcausse added a subtask: T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: revi, Mholloway, Ladsgroup, Multichill

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-13 Thread dcausse
dcausse added a comment. In: val from: Future[Iterable[Statement]] = fetchAsync(item, fromRev) val to: Future[Iterable[Statement]] = fetchAsync(item, revision) val awaiting: Future[Unit] = from flatMap { fromIt => to map { toIt => generateAndSendDiff(op

[Wikidata-bugs] [Maniphest] T257341: Add ORES article quality predictions to the WDQS

2020-07-13 Thread dcausse
dcausse triaged this task as "Low" priority. dcausse added a comment. As we try to split the graph as much as we can I think the proper approach to this would be store this data into a dedicated graph exposed through its own sparql endpoint and connected to wdqs through sparql

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-14 Thread dcausse
dcausse added a comment. Wanting the Executor to free of any previous task is perhaps something we want to reconsider. Ideally when we fetch 2 revisions at the same time if one fetch fails we want to fail (interrupt) the second fetch as soon as possible. Interrupting a thread while http

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-14 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T257795 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] T255657: Strange result in Wikidata query (full URLs given instead of identifiers)

2020-07-20 Thread dcausse
dcausse removed dcausse as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T255657 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Nikki, Lucas_Werkmeister_WMDE, Aklapper, Epidosis, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T166427: Measure maturity/quality of Wikidata items in a way it can be queried

2020-07-20 Thread dcausse
dcausse added a comment. As we try to split the graph as much as we can I think the proper approach to this would be store this data into a dedicated graph exposed through its own sparql endpoint and connected to wdqs through sparql federation. In general I would advise against using the

[Wikidata-bugs] [Maniphest] T257795: GenerateEntityDiffPatchOperationUnitTest fails randomly

2020-07-21 Thread dcausse
dcausse moved this task from Scaling to Current work on the Wikidata-Query-Service board. dcausse added a project: Discovery-Search (Current work). dcausse triaged this task as "Medium" priority. dcausse assigned this task to Mstyles. TASK DETAIL https://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse renamed this task from "RDF TTL format for Structured Data on Commons is broken" to "RDF dumps for Structured Data on Commons are broken". dcausse triaged this task as "High" priority. dcausse added projects: Wikidata-Query-Service, Wikidata. dcausse updat

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Addshore, Zbyszko, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse added a comment. I wonder if these IRIs should not be `http` instead of `https` because the `Concept URI` link on the file page refers to `http://commons.wikimedia.org/entity/M123`. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse added a comment. I think the `Concept URI` link should match what's in the dump. T226453#5436469 <https://phabricator.wikimedia.org/T226453#5436469> suggests if it these URIs for commons are not yet too widespread `https` could be used. TASK DET

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Addshore, Zbyszko, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Addshore, Zbyszko, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Addshore, Zbyszko, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-21 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Addshore, Zbyszko, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696

[Wikidata-bugs] [Maniphest] T240328: Slow indexing of Lexemes for wbsearchentities

2020-07-22 Thread dcausse
dcausse added a comment. Sense lookups are not supported by WikibaseLexemeCirrusSearch so I suppose that they use the wb_term mysql table. I thought that we combined mysql+elastic lookups so that when an ID is searched mysql can respond but looking at WikibaseLexemeCirrusSearch it was

[Wikidata-bugs] [Maniphest] T240328: Slow indexing of Lexemes for wbsearchentities

2020-07-22 Thread dcausse
dcausse triaged this task as "Medium" priority. dcausse edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T240328 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ T

[Wikidata-bugs] [Maniphest] T255399: Prepare wdqs1009 to run the streaming updater

2020-07-22 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T255399 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: Ottomata, Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-22 Thread dcausse
dcausse added a subtask: T258507: v: prefix not correctly prefixed in Wikibase when using entitysource config and extra prefixes. TASK DETAIL https://phabricator.wikimedia.org/T258474 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc

[Wikidata-bugs] [Maniphest] T258507: v: prefix not correctly prefixed in Wikibase when using entitysource config and extra prefixes

2020-07-22 Thread dcausse
dcausse added a parent task: T258474: RDF dumps for Structured Data on Commons are broken. TASK DETAIL https://phabricator.wikimedia.org/T258507 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Silvan_WMDE, dcausse Cc: dcausse, Addshore

[Wikidata-bugs] [Maniphest] T258508: WDQS | Prefix showing http instead of https

2020-07-22 Thread dcausse
dcausse added a comment. The whole system has been designed from scratch to support wikidata which is using `http` and it's likely that some parts of the code still boldly assumes that `http` is being used for IRIs. Looking at what you pasted I see at least two issues: - th

[Wikidata-bugs] [Maniphest] T258625: Querying WCQS should allow me to use prefixes for MediaInfo items

2020-07-22 Thread dcausse
dcausse created this task. dcausse added projects: Wikidata-Query-Service, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION sparql select * { sdc:M80570743 ?p ?o . } limit 10 AC: The list of prefixes supported should be (esp. sdc, sdcv, sdcref

[Wikidata-bugs] [Maniphest] T258625: Querying WCQS should allow me to use prefixes for MediaInfo items

2020-07-22 Thread dcausse
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T258625 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86

[Wikidata-bugs] [Maniphest] T258627: Autocompletion for MediaInfo items on WMCS

2020-07-22 Thread dcausse
dcausse created this task. dcausse added projects: Wikidata-Query-Service, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a WMCS user when writing a sparql query and when I start entering the `sdc:` prefix I'd like to have the same autocompletion fe

[Wikidata-bugs] [Maniphest] T258474: RDF dumps for Structured Data on Commons are broken

2020-07-23 Thread dcausse
dcausse claimed this task. dcausse moved this task from Incoming to To Be Deployed on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T258474 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T258508: WDQS | Prefix showing http instead of https

2020-07-23 Thread dcausse
dcausse added a comment. @Headingtona thanks this helps a lot, so I believe there is either a misconfiguration of your wikibase setup or a bug in the way Wikibase generates the RDF output. Could you paste the part of the configuration you changed? The fact that your wikibase setup is unable

[Wikidata-bugs] [Maniphest] T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction

2020-07-23 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION In order to prepare the switch to RichAsyncFunction it might be preferable to switch to

[Wikidata-bugs] [Maniphest] T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction

2020-07-23 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T258683 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-23 Thread dcausse
dcausse added a subtask: T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: tfmorris, revi, Mholloway, Ladsgroup

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-23 Thread dcausse
dcausse added a subtask: T258684: Use RichAsyncFunction for GenerateEntityDiffPatchOperation. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: tfmorris, revi, Mholloway, Ladsgroup

[Wikidata-bugs] [Maniphest] T258684: Use RichAsyncFunction for GenerateEntityDiffPatchOperation

2020-07-23 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T258684 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen

[Wikidata-bugs] [Maniphest] T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction

2020-07-23 Thread dcausse
dcausse edited parent tasks, added: T258684: Use RichAsyncFunction for GenerateEntityDiffPatchOperation; removed: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T258683 EMAIL PREFERENCES https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] T258684: Use RichAsyncFunction for GenerateEntityDiffPatchOperation

2020-07-23 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Doing blocking calls in a classic operation is causing nasty side-effects, the most annoying one

[Wikidata-bugs] [Maniphest] T258684: Use RichAsyncFunction for GenerateEntityDiffPatchOperation

2020-07-23 Thread dcausse
dcausse added a subtask: T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction. TASK DETAIL https://phabricator.wikimedia.org/T258684 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen, Akuckartz

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-23 Thread dcausse
dcausse removed a subtask: T258683: Switch GenerateEntityDiffPatchOperation to a RichFunction. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: tfmorris, revi, Mholloway, Ladsgroup

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-07-23 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Change propagation streams are per DC which means that events for the same kind of input may be

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-07-23 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen

[Wikidata-bugs] [Maniphest] T244590: EPIC: Rework the WDQS updater as an event driven application

2020-07-23 Thread dcausse
dcausse added a subtask: T258687: The streaming updater should read its events from multiple DC streams. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: tfmorris, revi, Mholloway

[Wikidata-bugs] [Maniphest] T258685: Changes to RDF formatter URI should be visible in query service

2020-07-23 Thread dcausse
dcausse added a comment. It might be feasible but it's not clear yet how hard it would be. Listing possible solutions (from the least desirable to the ideal): - reloading the dump is obviously a way to mitigate the issue, we want to make reloads more frequent but given the ti

[Wikidata-bugs] [Maniphest] T258709: Add nfs mount point to labstore for wdqs servers (wikibase dumps access)

2020-07-23 Thread dcausse
dcausse created this task. dcausse added projects: Wikidata-Query-Service, Data-Services. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION To workaround T222349 <https://phabricator.wikimedia.org/T222349> we could p

[Wikidata-bugs] [Maniphest] T258739: wdqs admins should have access to nginx logs, jstack on wdqs machines

2020-07-24 Thread dcausse
dcausse added a comment. I could not use this command last time and it did not work I think because the jvm was too busy. I need root to use jstack -F which I think has more chances to attach the jvm. TASK DETAIL https://phabricator.wikimedia.org/T258739 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] T258507: v: prefix not correctly prefixed in Wikibase when using entitysource config and extra prefixes

2020-07-26 Thread dcausse
dcausse added a comment. The is "somehow" affecting commons as well eventhough the RDF on commons remains "parseable". Here is a extract of https://commons.wikimedia.org/wiki/Special:EntityData/M211196.ttl?flavor=dump to illustrate the issue: @prefix rdf: <

[Wikidata-bugs] [Maniphest] T251497: Adapt munging process for SDoC

2020-07-27 Thread dcausse
dcausse added a comment. @Jheald we perhaps don't need to have both `schema:MediaObject` and `wikibase:Mediainfo`? TASK DETAIL https://phabricator.wikimedia.org/T251497 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Jheald, dc

<    1   2   3   4   5   6   7   8   9   10   >