[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Gehel closed this task as "Resolved". Gehel claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 680095 **merged** by Ryan Kemper: [operations/cookbooks@master] wdqs: int can't take in float as string https://gerrit.wikimedia.org/r/680095 TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

MPhamWMF moved this task from All WDQS-related tasks to Current work on the Wikidata-Query-Service board. MPhamWMF added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T280108 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 680095 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper): [operations/cookbooks@master] wdqs: int can't take in float as string https://gerrit.wikimedia.org/r/680095 TASK DETAIL https://phabricator.wikimedia.org/T280108

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:46:40Z] T280108 T267927 Manually re-enabled and ran puppet on `wdqs1005` (had closed the tmux pane which terminated the

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:33:28Z] T280108 T267927 Data transfers completed successfully; small issue with new `wait_for_updater` logic is

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

RKemper added a comment. 100.0% (2/2) success ratio (>= 100.0% threshold) for command: 'systemctl start ...art wdqs-updater'. 100.0% (2/2) success ratio (>= 100.0% threshold) of nodes successfully executed all commands. [1/1000, retrying in 600.00s] Attempt to run

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T16:21:44Z] T280108 T267927 Current wdqs transfers in progress: `wqds1004`->`wdqs1005`, `wdqs2008`->`wdqs2001` TASK DETAIL

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T16:17:13Z] T280108 T267927 Merged https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/679702 and ran puppet-agent on

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 679702 **merged** by Ryan Kemper: [operations/cookbooks@master] wdqs: result is list not dictionary https://gerrit.wikimedia.org/r/679702 TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

RKemper added a comment. (For future reference) import logging logging.basicConfig(level=logging.DEBUG) from spicerack import Spicerack spicerack = Spicerack(verbose=True, dry_run=True) def site(host): """Hacky way to get the site in which a host is

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 679702 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper): [operations/cookbooks@master] wdqs: result is list not dictionary https://gerrit.wikimedia.org/r/679702 TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

RKemper added a comment. Tested out the new cookbook, there's at least one small issue: Exception raised while executing cookbook sre.wdqs.data-transfer: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/spicerack/_menu.py", line 234, in run

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T06:33:53Z] !log T280108 T267927 `data-transfer` to `wdqs1004` was successful; cookbook failed due to a newly introduced

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T04:14:14Z] T280108 T267927 `wdqs2008` (source) caught up on lag, xfering to `wdqs1004`: `sudo -i cookbook

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T04:06:00Z] T280108 T267927 Merged https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/679320, will verify correct

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 679320 **merged** by Ryan Kemper: [operations/cookbooks@master] WDQS: Wait for updater to catchup during data transfer. https://gerrit.wikimedia.org/r/679320 TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

gerritbot added a comment. Change 679320 had a related patch set uploaded (by Gehel; author: Gehel): [operations/cookbooks@master] WDQS: Wait for updater to catchup during data transfer. https://gerrit.wikimedia.org/r/679320 TASK DETAIL https://phabricator.wikimedia.org/T280108

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Gehel added a parent task: T267927: Reload wikidata journal from fresh dumps. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314,

[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag

Gehel created this task. Gehel added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As a user of WDQS, I only want non-lagged servers to be pooled. As an operator of WDQS I don't want