[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-04-04 Thread Maintenance_bot
Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, Maintenance_bot Cc: Addshore, Sjoerddebruin, Aklapper, Tarrow, Astuthiodit_1, karapayneWM

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-04-04 Thread gerritbot
gerritbot added a comment. Change 764830 **merged** by Bking: [operations/puppet@production] Revert "Temp remove codfw from wikidata updateQueryServiceLag check" https://gerrit.wikimedia.org/r/764830 TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https:

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-03-17 Thread Gehel
Gehel closed this task as "Resolved". Gehel claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Addshore, Sjoerddebruin, Aklapper, Tarrow, Fernandobacasegua34, Astuthiodit

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-28 Thread Gehel
Gehel added a parent task: T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00). TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-28 Thread Gehel
Gehel moved this task from Incoming to Current work on the Wikidata-Query-Service board. Gehel added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T302330 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://p

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-23 Thread ItamarWMDE
ItamarWMDE added a project: Wikidata-Campsite (Team A Hearth 🏰🔥). TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ItamarWMDE Cc: Addshore, Sjoerddebruin, Aklapper, Tarrow, Fernandobacasegua34, 786,

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-23 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Addshore, Sjoerddebruin, Aklapper, Tarrow, Fernandobacasegua34, 786, Suran38, Biggs657, k

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-23 Thread gerritbot
gerritbot added a comment. Change 764830 had a related patch set uploaded (by Addshore; author: Addshore): [operations/puppet@production] Revert \"Temp remove codfw from wikidata updateQueryServiceLag check\" https://gerrit.wikimedia.org/r/764830 TASK DETAIL https://phabricator.w

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-23 Thread Addshore
Addshore lowered the priority of this task from "Unbreak Now!" to "High". Addshore added a comment. No longer UBN TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Addshore, Sjoerdde

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Maintenance_bot
Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Addshore, Sjoerddebruin, Aklapper, Tarrow, karapayneWMDE, Invadibot, MPhamW

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Addshore
Addshore added a comment. F34962209: image.png TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Addshore, Sjoerddebruin, Aklapper, Tarr

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Tarrow
Tarrow added a comment. We might want to tweak the script so that normal (non-puppet) deployers can fix these things (e.g. they read from mw-config or something) TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/email

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Stashbot
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2022-02-22T19:25:32Z] T302330 `ryankemper@cumin1001:~$ sudo -E cumin '*mwmaint*' 'run-puppet-agent'` (getting https://gerrit.wikimedia.org/r/c/operations/puppet/+/764875 out) TAS

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread gerritbot
gerritbot added a comment. Change 764875 **merged** by Ryan Kemper: [operations/puppet@production] Temp remove codfw from wikidata updateQueryServiceLag check https://gerrit.wikimedia.org/r/764875 TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://p

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Addshore
Addshore triaged this task as "Unbreak Now!" priority. Addshore added a comment. Marking as UBN, as this affects editing and user experience on wikidata TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreference

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Sjoerddebruin, Aklapper, Tarrow, Fernandobacasegua34, 786, Suran38, Biggs657, karapayneWM

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread gerritbot
gerritbot added a comment. Change 764875 had a related patch set uploaded (by Addshore; author: Addshore): [operations/puppet@production] Temp remove codfw https://gerrit.wikimedia.org/r/764875 TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phab

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Tarrow
Tarrow added a comment. Looks like altering this to not read from codfw prometheus might be sensible https://gerrit.wikimedia.org/g/operations/puppet/+/b81506884faa046e4103412f1928ece934a157dc/modules/profile/manifests/mediawiki/maintenance/wikidata.pp#15 TASK DETAIL https://phabricator.wik

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Lydia_Pintscher
Lydia_Pintscher added a project: Wikidata-Query-Service. TASK DETAIL https://phabricator.wikimedia.org/T302330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lydia_Pintscher Cc: Sjoerddebruin, Aklapper, Tarrow, karapayneWMDE, Invadibot, MPhamWMF, ma

[Wikidata-bugs] [Maniphest] T302330: Wikidata MaxLag above 10 for 1hr

2022-02-22 Thread Tarrow
Tarrow created this task. Tarrow added a project: Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION I saw: [1] Firing Labels alertname = Max Lag above 10 for 1 hour metric = wikibase-queryservice runbook = https://wikitech.wikimedia.org/wiki/W