[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus closed this task as "Resolved". RLazarus added a comment. We can't really test this without draining a DC, but we'll find out at the next switchover whether this is still a problem, and if so we can open a new task. Resolving with bold optimism in the meantime. TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: ayounsi, Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus closed subtask T288867: Rewrite mw-warmup.js in Python as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: ayounsi, Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus added a subtask: T288867: Rewrite mw-warmup.js in Python. TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: ayounsi, Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus added a comment. POSTs will probably have to wait for the Python rewrite, but then they'll be easy. Can you recommend specific requests? TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: ayounsi, Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus added a comment. In that case it sounds like yes, we do need cache warming in eqiad before repooling it -- and we'll need to add URLs to warm up s8, per this task. The Python rewrite doesn't have to be related -- for now, the new script will read URLs from the same files as the old script (urls-cluster.txt <https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/mediawiki/files/maintenance/mediawiki-cache-warmup/urls-cluster.txt>) so feel free to add entries there without waiting for me. That might change, but if so I'll coordinate, so in the meantime don't worry about it. TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: ayounsi, Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290989: switchdc cache warmup should include URLs that warmup relevant Wikidata caches
RLazarus added a comment. Agree we don't need it in order to switch the RW site to codfw. Are we also planning to depool eqiad for reads, and serve fully from codfw for any of the switchover period? If so, will we need cache warming in eqiad before repooling? TASK DETAIL https://phabricator.wikimedia.org/T290989 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: Clement_Goubert, RLazarus, Lydia_Pintscher, karapayneWMDE, LSobanski, Krinkle, Addshore, jcrespo, Marostegui, Lucas_Werkmeister_WMDE, Ladsgroup, Legoktm, Aklapper, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, Lmlmljs, 786, Biggs657, Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Krenair, akosiaris ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T288175: Migrate wikibase-dispatch-changes crons to systemd timers
RLazarus added a parent task: T211250: Create a mediawiki::cronjob define. TASK DETAIL https://phabricator.wikimedia.org/T288175 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, RLazarus Cc: Aklapper, Addshore, Ladsgroup, Legoktm, Biggs657, Invadibot, Lalamarie69, maantietaja, Juan90264, Alter-paule, Hazizibinmahdi, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, Djdungti, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, scfc, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T269693: mediawiki_job_wikidata-updateQueryServiceLag failing
RLazarus closed this task as "Resolved". RLazarus added a comment. Yep, the alert has cleared. Thanks! TASK DETAIL https://phabricator.wikimedia.org/T269693 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, RLazarus Cc: dcausse, Dzahn, RLazarus, RKemper, Aklapper, Ramtin0071, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269693: mediawiki_job_wikidata-updateQueryServiceLag failing
RLazarus created this task. RLazarus added projects: Wikidata-Query-Service, Operations. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION This systemd timer runs every minute, but the last time it succeeded was Dec 7, 19:41 UTC. Since then it's been failing consistently; journalctl looks like: Dec 08 16:52:00 mwmaint1002 systemd[1]: Started MediaWiki periodic job wikidata-updateQueryServiceLag. Dec 08 16:52:01 mwmaint1002 mediawiki_job_wikidata-updateQueryServiceLag[131972]: Failed to get lag from prometheus Dec 08 16:52:01 mwmaint1002 systemd[1]: mediawiki_job_wikidata-updateQueryServiceLag.service: Main process exited, code=exited, status=1/FAILURE Dec 08 16:52:01 mwmaint1002 systemd[1]: mediawiki_job_wikidata-updateQueryServiceLag.service: Unit entered failed state. Dec 08 16:52:01 mwmaint1002 systemd[1]: mediawiki_job_wikidata-updateQueryServiceLag.service: Failed with result 'exit-code'. SAL shows @RKemper was reimaging WDQS hosts at the time it started failing, not sure if that's related or coincidence. TASK DETAIL https://phabricator.wikimedia.org/T269693 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: RLazarus, RKemper, Aklapper, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264821: Hourly read spikes against s8 resulting in occasional user-visible latency & error spikes
RLazarus added a subtask: T266040: Large text objects are randomized to cache backends. TASK DETAIL https://phabricator.wikimedia.org/T264821 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: Michael, RhinosF1, Joe, LSobanski, Addshore, Ladsgroup, RLazarus, Marostegui, Aklapper, CDanis, lmata, wkandek, JMeybohm, Akuckartz, darthmon_wmde, Legado_Shulgin, Nandana, jijiki, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Wong128hk, Wikidata-bugs, aude, faidon, Mbch331, Rxy, Jay8g, fgiunchedi, Dzahn ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T262773: Stop archiving the wikidata-bugs mailinglist in pipermail
RLazarus added subscribers: Dzahn, RLazarus. RLazarus added a comment. Any list administrator can disable archiving new messages, at https://lists.wikimedia.org/mailman/admin/wikidata-bugs/archive -- just change "Archive messages?" to "no" and submit. If you'd also like to delete the existing archive, SRE can do that. For the record, we're not talking about very much data at all: rzl@lists1001:~$ sudo du -hs /var/lib/mailman/archives/private/wikidata-bugs{,.mbox} 3.8G/var/lib/mailman/archives/private/wikidata-bugs 1.3G/var/lib/mailman/archives/private/wikidata-bugs.mbox so from purely a storage perspective it's not worth worrying about -- but I don't know if there are any other considerations in terms of mailman's performance. CC @Dzahn to weigh in if he's so inclined. TASK DETAIL https://phabricator.wikimedia.org/T262773 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RLazarus Cc: RLazarus, Dzahn, Peachey88, Lydia_Pintscher, Nemo_bis, Ladsgroup, Legoktm, Aklapper, lmata, Akuckartz, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Wong128hk, Barras, Wikidata-bugs, aude, Jalexander, faidon, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs