[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel closed subtask T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: jbond,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-17T00:14:54Z] T267927 `sudo run-puppet-agent` and `sudo pool` on `wdqs2003` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-17T00:08:09Z] T267927 Reload of `wdqs2003` complete TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T17:48:15Z] T267927 Transferring from `wdqs2008`->`wdqs2003` to resolve the data corruption on `wdqs2003` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. We can't proceed any further on this ticket until we address https://phabricator.wikimedia.org/T280382, because the short-term fix for disk space will require re-imaging the affected `wdqs` hosts which will require starting the `data-transfer` process over (not the

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T17:03:46Z] T267927 Pooled `wdqs1007`, `wdqs2003`, `wdqs1008`, `wdqs2004` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T17:00:36Z] T267927 Following data transfers complete: `wdqs1004`->`wdqs1007`, `wdqs2001`->`wdqs2003`, `wdqs1003`->`wdqs1008`, `wdqs2008`->`wdqs2004` TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T03:26:41Z] T267927 Pooled `wdqs2001` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T03:22:22Z] T267927 Pooled `wdqs1006` and `wdqs2002` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T03:09:50Z] T267927 kicked off next round of `data-transfer`s: `wdqs1004`->`wdqs1007`, `wdqs2001`->`wdqs2003`, `wdqs1003`->`wdqs1008`, `wdqs2008`->`wdqs2004` TASK

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-16T03:05:21Z] T267927 Last round of `data-transfer`s finished successfully, proceeding to next round TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:56:19Z] T267927 WDQS kicked off next round of `data-transfer`s: `wdqs1004`->`wdqs1006`, `wdqs2001`->`wdqs2002`, `wdqs2008`->`wdqs1003` TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:48:47Z] T267927 pooled `wdqs1005` (all caught up on lag) TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:46:40Z] T280108 T267927 Manually re-enabled and ran puppet on `wdqs1005` (had closed the tmux pane which terminated the

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T22:33:28Z] T280108 T267927 Data transfers completed successfully; small issue with new `wait_for_updater` logic is

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T16:21:44Z] T280108 T267927 Current wdqs transfers in progress: `wqds1004`->`wdqs1005`, `wdqs2008`->`wdqs2001` TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T16:17:13Z] T280108 T267927 Merged https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/679702 and ran puppet-agent on

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. Since we've seeded to `wdqs1004.eqiad.wmnet` we can start using that to transfer to other eqiad public nodes. Still need to seed to the other 3 (codfw public, codfw internal, eqiad internal) before we'll be at the peak "velocity". TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T06:33:53Z] !log T280108 T267927 `data-transfer` to `wdqs1004` was successful; cookbook failed due to a newly introduced

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Alter-paule,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T04:14:14Z] T280108 T267927 `wdqs2008` (source) caught up on lag, xfering to `wdqs1004`: `sudo -i cookbook

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-15T04:06:00Z] T280108 T267927 Merged https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/679320, will verify correct

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. I was having trouble understanding what the restart status of blazegraph has to do with the transfer of `/srv/wdqs/wikidata.jnl`. (As an aside, the point that running multiple simultaneous transfers from the same source is bottlenecked by outgoing bandwidth anyway

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a comment. Multiple transfers from the same source host is not supported. The various cookbook instances don't synchronize, so blazegraph is likely to be restarted before all transfers are completed, leading to corruption of data. Note that in any case, this would not speed up

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. [EQIAD PUBLIC] DONE sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1004.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-14T09:12:03Z] T267927 depooled `wdqs1004` following data transfer (catching up on lag), current round of data transfers is done so there shouldn't be any left to depool

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-04-14T09:06:28Z] T267927 depool `wdqs2001` following data transfer (catching up on lag) TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: jbond, Zbyszko, Gehel,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-03-29T09:16:24Z] T267927 `sudo -i cookbook sre.wdqs.data-reload wdqs2008.codfw.wmnet --task-id T267927 --reload-data wikidata --reason 'T267927: Reload wikidata jnl from

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-03-27T06:10:03Z] T267927 `sudo https_proxy=webproxy.codfw.wmnet:8080 wget https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2 -O

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 675003 **merged** by Ryan Kemper: [operations/cookbooks@master] wdqs: only disable puppet during blazegraph rstrt https://gerrit.wikimedia.org/r/675003 TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 672383 **abandoned** by Jbond: [operations/puppet@production] C:query_service: Add paramter to control if we manage services Reason: https://gerrit.wikimedia.org/r/672383 TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 672384 **abandoned** by Jbond: [operations/puppet@production] P:query_service: add ability to disable managing services Reason: https://gerrit.wikimedia.org/r/672384 TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 675003 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper): [operations/cookbooks@master] wdqs: only disable puppet during blazegraph rstrt https://gerrit.wikimedia.org/r/675003 TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. `wdqs1009`, `wdqs1010`, and `wdqs2008` are done, so we need to `data-transfer` to the remaining instances. It looks like `wdqs1010` is warning (not critical) for disk space on `/srv`. Glancing at the box it's still got 44G left but is at 96% utilization:

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-03-24T19:56:58Z] T267927 Host key is missing for `wdqs2008` leading to `data-transfer` cookbook failing, looking into resolving TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-03-24T19:42:09Z] T267927 Re-enabledpuppet on `wdqs2008` and ran puppet agent TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

jbond added a comment. A drafted a couple of changes (see above) which try to add a hiera parameter to control this, not sure if i got everything but i think it should be a good start TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 672384 had a related patch set uploaded (by Jbond; owner: John Bond): [operations/puppet@production] P:query_service: add ability to disable managing services https://gerrit.wikimedia.org/r/672384 TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, gerritbot Cc: ayounsi, jbond, Zbyszko, Gehel, Aklapper, dcausse, MPhamWMF, maantietaja,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

gerritbot added a comment. Change 672383 had a related patch set uploaded (by Jbond; owner: John Bond): [operations/puppet@production] C:query_service: Add paramter to control if we manage services https://gerrit.wikimedia.org/r/672383 TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

jbond added a comment. fyi I noticed the following comment in the reload cook book # FIXME: this cookbook is expected to run for weeks, we don't want to disable puppet for that long #but we want to ensure that the various services aren't restarted by puppet along the way.

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

jbond added a subscriber: ayounsi. jbond added a comment. In T267927#6863396 , @RKemper wrote: > Note: Puppet is still disabled on `wdqs2008` while the reload runs. It occurred to me that I'm not sure if puppet actually **needs** to be

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. Note: Puppet is still disabled on `wdqs2008` while the reload runs. It occurred to me that I'm not sure if puppet actually **needs** to be disabled during data reloads or if that's just a precaution we've historically taken - any insight here @Gehel? TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-26T04:23:31Z] T267927 [WDQS Data Reload] `sudo -i cookbook sre.wdqs.data-reload wdqs2008.codfw.wmnet --task-id T267927 --reload-data wikidata --reason 'T267927: Reload

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. Downtimed `wdqs2008` until `2021-03-04 21:56:59` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Zbyszko, Gehel, Aklapper, dcausse, MPhamWMF, CBogen,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-25T19:16:19Z] T267927 Downloading dumps: `sudo https_proxy=webproxy.codfw.wmnet:8080 wget https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2 -O

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-25T18:59:18Z] T267927 Manual puppet run got `wdqs2008` present in puppetdb again. Now being blocked by lack of host key for `wdqs2008` present on `cumin2001`, so I'm

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-25T18:50:05Z] T267927 Trying to kick off data reload on `wdqs2008` from `cumin2001` fails because of `spicerack.remote.RemoteError: No hosts provided`. Doing some

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

ayounsi added a comment. Thanks, thinking a bit more about it, it might be better to put whatever blocks those reloads in Puppet behind a Hiera option. So puppet can stay enabled while not interfering with those actions. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subscriber: Zbyszko. Gehel added a comment. Data load on wdqs2008 has resulted (again) in what looks like a corrupted journal. This will need to be restarted, but I'll wait on feedback from @dcausse / @Zbyszko to see if we have any short term ideas on how to improve this.

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a comment. loading the dump is completed on wdqs1009, restarting the updater TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: ayounsi, Gehel, Aklapper, dcausse,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a comment. In T267927#6855514 , @ayounsi wrote: >> The last Puppet run was at Tue Feb 16 13:22:35 UTC 2021 (11219 minutes ago). Puppet is disabled. T267927 : Reload wikidata jnl

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

ayounsi added a comment. > The last Puppet run was at Tue Feb 16 13:22:35 UTC 2021 (11219 minutes ago). Puppet is disabled. T267927 : Reload wikidata jnl from fresh dumps - gehel@cumin2001 - T267927

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-24T04:10:51Z] T267927 [WDQS Data Reload] Running `/srv/deployment/wdqs/wdqs/loadData.sh -n wdq -d /srv/wdqs/munged/ -s 864` on `ryankemper@wdqs2008` tmux session

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a comment. data reload started on wdqs2008: gehel@cumin2001:~$ sudo -i cookbook sre.wdqs.data-reload --task-id T267927 --reload-data wikidata --reason 'T267927: Reload wikidata jnl from fresh dumps' wdqs2008.codfw.wmnet --depool --proxy-server

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a comment. Data reload failed on both wdqs1009 and wdqs1010. wdqs1009: the host crashed, data reload continued manually, starting from chunk 407 (`/srv/deployment/wdqs/wdqs/loadData.sh -n wdq -d /srv/wdqs/munged/ -s 407`). wdqs1010: looks like a disk error,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel closed subtask T272120: Deleted item still gets shown in WDQS query results as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-09T18:42:04Z] T267927 [WDQS Data Reload] `sudo cookbook sre.wdqs.data-reload wdqs1010.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --reason 'T267927: Reload

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-09T18:41:53Z] T267927 [WDQS Data Reload] Small typo in previous SAL log message, see subsequent SAL line for correction: TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-09T18:40:57Z] T267927 [WDQS Data Reload] `sudo cookbook sre.wdqs.data-reload wdqs1010.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --reason

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-09T18:40:25Z] T267927 [WDQS Data Reload] `sudo cookbook sre.wdqs.data-reload wdqs1009.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --skolemize --reason

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-09T18:37:40Z] T267927 [WDQS Data Reload] Clearing old wikidata journal file to free disk space before beginning data reload:`sudo systemctl status wdqs-blazegraph && sudo

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. Still waiting for the latest dumps to be downloaded (few more hours), then need to reboot WDQS hosts as part of https://phabricator.wikimedia.org/T274213, then can do the actual data-reload TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-05T23:35:16Z] T267927 Re-downloading latest dumps (main database, lexeme) in tmux session `downloads_dumps` on `ryankemper@wdqs1009.eqiad.wmnet` TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. `sudo cookbook sre.wdqs.data-reload wdqs1009.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --skolemize --reason 'T267927: Reload wikidata jnl from fresh dumps' --task-id T267927 ` is failing with: - OUTPUT of 'test -f

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-02-05T22:42:01Z] T267927 `sudo cookbook sre.wdqs.data-reload wdqs1009.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --skolemize --reason 'T267927: Reload

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Mbch331 added a subtask: T273937: Deleted lexeme is available in the Wikidata Query Service. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Mbch331 Cc: Aklapper, dcausse, MPhamWMF,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse added a comment. I pre-fetched the dumps required for the reload on wdqs1010 & wdqs1009. - `wdqs1009` needs to be reloaded using `--reuse-downloaded-dump --reload-data wikidata --skolemize` - `wdqs1010` with `--reuse-downloaded-dump --reload-data wikidata` TASK DETAIL

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T272120: Deleted item still gets shown in WDQS query results. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF, CBogen,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T268408: Query returns outdated results . TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T267175: SPARQL-Query shows entries, which should be filter out; number of entries in result set might change when executed repeatedly (possible caching/indexing problem). TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T267924: WDQS Updater (based on recent changes) missed some updates. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF, CBogen,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T264042: Query service returns weird Commons links when asking for P50 (author) statement value. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc:

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T266211: SPARQL: bad "instance of" statements pointing to EntityData. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel added a subtask: T255657: Strange result in Wikidata query (full URLs given instead of identifiers). TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, Gehel Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper added a comment. We'll want to reload these this Friday, because the latest dumps should be available thursday evening. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc:

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

CBogen set the point value for this task to "3". TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: CBogen Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Addshore closed subtask T267644: Update Wikidata unit conversion config (normalized quantities) as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Aklapper, dcausse,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse added a subtask: T267644: Update Wikidata unit conversion config (normalized quantities). TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, CBogen, Akuckartz,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

Gehel triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,

[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps

dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION As wdqs user I want to query a dataset that is coherent with the state of wikidata. During