dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata.
TASK DESCRIPTION Change propagation streams are per DC which means that events for the same kind of input may be found in multiple kafka topics. The flink pipeline should support this setup and there must be a way to pass or infer the many topics that may contain the data the application must consume. One simple way to do this is to accept multiple topics, e.g. the option `rev_create_topic` could accept multiple topics (`eqiad.mediawiki.revision-create` and `codfw.mediawiki.revision-create`). note: in the existing setup only one stream is likely to be active and thus the watermarking must be properly setup to support idle streams. AC: - events sent to `codfw.mediawiki.revision-create` must be read TASK DETAIL https://phabricator.wikimedia.org/T258687 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
