[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-14 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Gehel Cc: Thadguidry, dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-14 Thread Maintenance_bot
Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Maintenance_bot Cc: Thadguidry, dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde,

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-14 Thread gerritbot
gerritbot added a comment. Change 626156 **merged** by jenkins-bot: [wikidata/query/rdf@master] Support multiple DC streams per topic https://gerrit.wikimedia.org/r/626156 TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-09 Thread gerritbot
gerritbot added a comment. Change 626156 had a related patch set uploaded (by DCausse; owner: DCausse): [wikidata/query/rdf@master] Support multiple DC streams per topic https://gerrit.wikimedia.org/r/626156 TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-09 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, gerritbot Cc: Thadguidry, dcausse, Aklapper, Alter-paule, Beast1978, CBogen, Un1tY,

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-08 Thread dcausse
dcausse claimed this task. dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T258687 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-07 Thread Gehel
Gehel moved this task from Scaling to Current work on the Wikidata-Query-Service board. Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T258687 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-02 Thread dcausse
dcausse added a comment. Thanks for the advise, most of the streaming process of this application is keyed indeed. I haven't thought about using the CEP api provided by flink it sounds very flexible. As for this particular task I'm hoping that recent flink 1.11 that unifies periodic and

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-09-02 Thread Thadguidry
Thadguidry added a comment. @dcausse Dunno if this might help but could a simple window help or where you use KeyedProcessFunction on a KeyedStream? If the stream is unkeyed (or initially

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-07-23 Thread dcausse
dcausse added a parent task: T244590: EPIC: Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen,

[Wikidata-bugs] [Maniphest] T258687: The streaming updater should read its events from multiple DC streams

2020-07-23 Thread dcausse
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Change propagation streams are per DC which means that events for the same kind of input may be