[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Gehel Cc: Ottomata, dcausse, Aklapper, Gehel, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, Maintenance_bot Cc: Ottomata, dcausse, Aklapper, Gehel, Invadibot, MPhamWMF,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 727021 **merged** by Gehel: [operations/cookbooks@master] sre.wdqs: Add kafka position transfer to wdqs cookbooks https://gerrit.wikimedia.org/r/727021 TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 723214 **merged** by jenkins-bot: [operations/software/spicerack@master] Added spicerack.kafka with offset transfer function https://gerrit.wikimedia.org/r/723214 TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 727021 had a related patch set uploaded (by ZPapierski; author: ZPapierski): [operations/cookbooks@master] [WIP] Add kafka position transfer to wdqs cookbooks https://gerrit.wikimedia.org/r/727021 TASK DETAIL

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 721857 **merged** by Gehel: [operations/puppet@production] Add kafka clusters' brokers to spicerack config https://gerrit.wikimedia.org/r/721857 TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Ottomata added a comment. Just making sure yall have considered using consumer offset sync as supported by MirrorMaker 2 . Upgrading to MM2 would be a big deal, but we haven't done it because no one has needed it! If there

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 723214 had a related patch set uploaded (by ZPapierski; author: ZPapierski): [operations/software/spicerack@master] Added spicerack.kafka with offset transfer function https://gerrit.wikimedia.org/r/723214 TASK DETAIL

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, gerritbot Cc: dcausse, Aklapper, Gehel, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

gerritbot added a comment. Change 721857 had a related patch set uploaded (by ZPapierski; author: ZPapierski): [operations/puppet@production] Add kafka clusters' brokers to spicerack config https://gerrit.wikimedia.org/r/721857 TASK DETAIL

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Zbyszko claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko Cc: dcausse, Aklapper, Gehel, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Zbyszko updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko Cc: dcausse, Aklapper, Gehel, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

dcausse added a comment. Note that there's a WIP patch to ease the transfer of kafka offsets (while transferring a journal from one node to another) that will no longer maintained in the triple store. The solution is up for

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

MPhamWMF triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Aklapper, Gehel, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

MPhamWMF set the point value for this task to "8". TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Aklapper, Gehel, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Zbyszko updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko Cc: Aklapper, Gehel, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

MPhamWMF moved this task from All WDQS-related tasks to Current work on the Wikidata-Query-Service board. MPhamWMF added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T276469 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As an operator, I want to have automation or documented procedures to standard operations so that I

[Wikidata-bugs] [Maniphest] T276469: Cookbooks and / or operation procedures are adapted for the new Flink based WDQS Streaming Updater

Gehel added a parent task: T244590: [Epic] Rework the WDQS updater as an event driven application. TASK DETAIL https://phabricator.wikimedia.org/T276469 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, Gehel, MPhamWMF,