[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bking, Maintenance_bot Cc: Andrew, bking, akosiaris, Hokwelum, Volans, ayounsi, cmooney,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 870714 **abandoned** by Bking: [operations/puppet@production] query_service: Allow query hosts to rsync data from clouddumps Reason: Our use case for dumps is small enough at this time; will revisit if needed

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

bking closed this task as "Resolved". bking added a comment. Thanks @Hokwelum ! Since rsync is not rate-limited, we don't need to bother with the original topic of this ticket. The WDQS-specific rsync work will continue in T327689 . TASK DETAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

bking claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bking Cc: Andrew, bking, akosiaris, Hokwelum, Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel, Aklapper,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

fnegri moved this task from Kanban to Watching on the cloud-services-team board. fnegri edited projects, added cloud-services-team; removed cloud-services-team (Kanban). TASK DETAIL https://phabricator.wikimedia.org/T222349 WORKBOARD https://phabricator.wikimedia.org/project/board/2773/

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

fnegri edited projects, added cloud-services-team; removed cloud-services-team (FY2022/2023-Q3). Restricted Application edited projects, added cloud-services-team (Kanban); removed cloud-services-team. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

fnegri moved this task from Doing? (legacy column) to FY2022/2023-Q3 on the cloud-services-team board. fnegri edited projects, added cloud-services-team (FY2022/2023-Q3); removed cloud-services-team. TASK DETAIL https://phabricator.wikimedia.org/T222349 WORKBOARD

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

fnegri edited projects, added cloud-services-team; removed cloud-services-team (Kanban). fnegri moved this task from Kanban to Doing? (legacy column) on the cloud-services-team board. TASK DETAIL https://phabricator.wikimedia.org/T222349 WORKBOARD

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Hokwelum added a subscriber: Andrew. Hokwelum added a comment. In T222349#8485437 , @bking wrote: > Per above patches and discussion with @akosiaris , we are going to try using the rsync protocol instead of NFS. Do the clouddumps

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 870714 had a related patch set uploaded (by Bking; author: Bking): [operations/puppet@production] query_service: Allow query hosts to rsync data from clouddumps https://gerrit.wikimedia.org/r/870714 TASK DETAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

bking added subscribers: akosiaris, bking. bking added a comment. Per above patches and discussion with @akosiaris , we are going to try using the rsync protocol instead of NFS. Do the clouddumps servers also rate-limit rsync? TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Hokwelum, Volans, ayounsi, cmooney, EBernhardson,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 867646 **merged** by Bking: [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/867646 TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Hokwelum, Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel, Aklapper,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 867646 had a related patch set uploaded (by Bking; author: Ebernhardson): [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/867646 TASK DETAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

jbond edited projects, added netops; removed SRE. Restricted Application added a project: Infrastructure-Foundations. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: jbond Cc: Hokwelum, Volans,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Hokwelum, Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 835596 **merged** by Ryan Kemper: [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/835596 TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel, Aklapper, Jersione,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 835596 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper): [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/835596 TASK DETAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 832543 **merged** by Ryan Kemper: [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/832543 TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel, Aklapper, Jersione,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

gerritbot added a comment. Change 832543 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson): [operations/puppet@production] Mount labstore to wcqs/wdqs instance for dumps reload https://gerrit.wikimedia.org/r/832543 TASK DETAIL

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Gehel removed Gehel as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T222349 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: Volans, ayounsi, cmooney, EBernhardson, Bstorm, ArielGlenn, Gehel, Aklapper, fnegri,

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

ArielGlenn added a comment. Note that the checksum files for those dumps are available for download as well, since they are provided along with the main dump output files to all mirrors. Someone from WMCS will probably need to look at this (again) if the discussion is being re-opened.

[Wikidata-bugs] [Maniphest] T222349: Do not rate limit dumps from internal network

Volans added subscribers: cmooney, ayounsi, Volans. Volans reopened this task as "Open". Volans added a comment. Restricted Application added a project: wdwb-tech. I'm re-opening this as a follow up from a chat in this CR