[Wikidata-bugs] [Maniphest] T349118: Migrate node-based services in production to node18
elukey added a comment. I filed https://gerrit.wikimedia.org/r/c/mediawiki/services/recommendation-api/+/977751 for the recommendation-api service, if anybody has time to review it I will take care of the build/deploy :) TASK DETAIL https://phabricator.wikimedia.org/T349118 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: KartikMistry, Michael, gmodena, Ottomata, elukey, WDoranWMF, Ahoelzl, MSantos, Jdforrester-WMF, GLima-WMF, Danny_Benjafield_WMDE, BartTerpstra, Isabelladantes1983, Themindcoder, Dominika_Naumiuk, Kappakayala, Mohamed-Awnallah, Adamm71, Jersione, Hellket777, LisafBia6531, Wangombe, Astuthiodit_1, 786, lbowmaker, VPuffetMichel, Arnoldokoth, BTullis, Biggs657, ihurbain, karapayneWMDE, Kritisinghh, Vyaki, bwang, Deek9399, Invadibot, Surya742, Dasgupta002, Deveshprasad577, Atharva17, Tsiruot, sobhanbera, UOzurumba, Bharatkhatri351, Ywats0ns, PallaviPatke, diegodlh, Aishik_Rehman, maantietaja, Juan90264, Tiputini, wkandek, JMeybohm, Alter-paule, Beast1978, Rileych, ItamarWMDE, Un1tY, DAlangi_WMF, Nintendofan885, Akuckartz, Ironie, 50019062, Hook696, darthmon_wmde, Kent7301, joker88john, Tris_T7, CucyNoiD, Nandana, jijiki, Gaboe420, Mihailo, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, chapulina, QZanden, srishakatux, Alfa80, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Jdrewniak, Jason_Quinn, Soum213, Taiwania_Justo, Neuronton, Nizil, Scott_WUaS, Shangkuanlc, Ixocactus, Verdy_p, Wikidata-bugs, Base, aude, Amire80, Mvolz, Lydia_Pintscher, Jsahleen, mxn, Nikerabbit, Arrbee, santhosh, Arlolra, TheDJ, Mbch331, Jay8g, Krenair, Clement_Goubert ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T349118: Migrate node-based services in production to node18
elukey closed subtask T348950: Upgrade change propagation to nodejs18 as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T349118 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Michael, gmodena, Ottomata, elukey, WDoranWMF, Ahoelzl, MSantos, Krinkle, Jdforrester-WMF, Danny_Benjafield_WMDE, BartTerpstra, Isabelladantes1983, Themindcoder, Kappakayala, Mohamed-Awnallah, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, 786, lbowmaker, VPuffetMichel, Arnoldokoth, BTullis, Biggs657, ihurbain, karapayneWMDE, Kritisinghh, Vyaki, bwang, Deek9399, Invadibot, Surya742, Dasgupta002, Deveshprasad577, Atharva17, Tsiruot, sobhanbera, UOzurumba, Bharatkhatri351, Ywats0ns, diegodlh, Aishik_Rehman, maantietaja, Juan90264, wkandek, JMeybohm, Alter-paule, Beast1978, Rileych, ItamarWMDE, Un1tY, DAlangi_WMF, Nintendofan885, Akuckartz, Ironie, Hook696, darthmon_wmde, Kent7301, joker88john, Tris_T7, CucyNoiD, Nandana, jijiki, Gaboe420, Mihailo, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, chapulina, QZanden, srishakatux, Alfa80, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Jdrewniak, Jason_Quinn, Neuronton, Nizil, Scott_WUaS, Shangkuanlc, Ixocactus, Verdy_p, Wikidata-bugs, Base, aude, Amire80, Mvolz, Lydia_Pintscher, Jsahleen, mxn, Arrbee, santhosh, KartikMistry, Arlolra, TheDJ, Mbch331, Jay8g, Krenair, Clement_Goubert ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T349118: Migrate node-based services in production to node18
elukey updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T349118 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Michael, gmodena, Ottomata, elukey, WDoranWMF, Ahoelzl, MSantos, Krinkle, Jdforrester-WMF, Danny_Benjafield_WMDE, BartTerpstra, Isabelladantes1983, Themindcoder, Kappakayala, Mohamed-Awnallah, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, 786, lbowmaker, VPuffetMichel, Arnoldokoth, BTullis, Biggs657, ihurbain, karapayneWMDE, Kritisinghh, Vyaki, bwang, Deek9399, Invadibot, Surya742, Dasgupta002, Deveshprasad577, Atharva17, Tsiruot, sobhanbera, UOzurumba, Bharatkhatri351, Ywats0ns, diegodlh, Aishik_Rehman, maantietaja, Juan90264, wkandek, JMeybohm, Alter-paule, Beast1978, Rileych, ItamarWMDE, Un1tY, DAlangi_WMF, Nintendofan885, Akuckartz, Ironie, Hook696, darthmon_wmde, Kent7301, joker88john, Tris_T7, CucyNoiD, Nandana, jijiki, Gaboe420, Mihailo, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, chapulina, QZanden, srishakatux, Alfa80, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Jdrewniak, Jason_Quinn, Neuronton, Nizil, Scott_WUaS, Shangkuanlc, Ixocactus, Verdy_p, Wikidata-bugs, Base, aude, Amire80, Mvolz, Lydia_Pintscher, Jsahleen, mxn, Arrbee, santhosh, KartikMistry, Arlolra, TheDJ, Mbch331, Jay8g, Krenair, Clement_Goubert ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T343419: Move Wikidata tools to Lift Wing
elukey added a comment. Ack thanks! Then I'd suggest to start with damaging and goodfaith, and then migrate later on when RR Wikidata is ready? TASK DETAIL https://phabricator.wikimedia.org/T343419 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: achou, Arian_Bozorg, Ladsgroup, Lucas_Werkmeister_WMDE, Michael, ItamarWMDE, Aklapper, Lydia_Pintscher, elukey, Danny_Benjafield_WMDE, fbalicchia, isarantopoulos, Astuthiodit_1, karapayneWMDE, Simonmaignan, Invadibot, amy_rc, maantietaja, calbon, Anerka, Akuckartz, Nandana, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Zache, Wikidata-bugs, aude, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T343419: Move Wikidata tools to Lift Wing
elukey added a subscriber: achou. elukey added a comment. @Ladsgroup we have a dedicated Revertrisk model dedicated to Wikidata in staging (more info in T333125 <https://phabricator.wikimedia.org/T333125>), but I think that we just need to do some follow ups and verification before adding it to production. @achou do you have more info if/when the Research team wants to deploy to prod? TASK DETAIL https://phabricator.wikimedia.org/T343419 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: achou, Arian_Bozorg, Ladsgroup, Lucas_Werkmeister_WMDE, Michael, ItamarWMDE, Aklapper, Lydia_Pintscher, elukey, Danny_Benjafield_WMDE, fbalicchia, isarantopoulos, Astuthiodit_1, karapayneWMDE, Simonmaignan, Invadibot, amy_rc, maantietaja, calbon, Anerka, Akuckartz, Nandana, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Zache, Wikidata-bugs, aude, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T343419: Move Wikidata tools to Lift Wing
elukey added a comment. https://github.com/Ladsgroup/Vandalism-dashboard/issues/15 TASK DETAIL https://phabricator.wikimedia.org/T343419 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, Lydia_Pintscher, elukey, Danny_Benjafield_WMDE, fbalicchia, isarantopoulos, Astuthiodit_1, karapayneWMDE, Simonmaignan, Invadibot, amy_rc, maantietaja, calbon, Anerka, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Zache, Wikidata-bugs, aude, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T343419: Move Wikidata tools to Lift Wing
elukey added a project: Item Quality Evaluator. TASK DETAIL https://phabricator.wikimedia.org/T343419 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, Lydia_Pintscher, elukey, Danny_Benjafield_WMDE, fbalicchia, isarantopoulos, Astuthiodit_1, karapayneWMDE, Simonmaignan, Invadibot, amy_rc, maantietaja, calbon, Anerka, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Zache, Wikidata-bugs, aude, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T343419: Move Wikidata tools to Lift Wing
elukey created this task. elukey added projects: Wikidata, Machine-Learning-Team. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION List to be expanded: - https://wdvd.toolforge.org - https://item-quality-evaluator.toolforge.org Some useful docs in https://wikitech.wikimedia.org/wiki/ORES TASK DETAIL https://phabricator.wikimedia.org/T343419 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, Lydia_Pintscher, elukey, Danny_Benjafield_WMDE, fbalicchia, isarantopoulos, Astuthiodit_1, karapayneWMDE, Simonmaignan, Invadibot, maantietaja, calbon, Anerka, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Zache, Wikidata-bugs, aude, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T209611: [Epic] Make ORES scores for wikidata available as a dump
elukey closed subtask T209737: Backfill ORES Hadoop scores with historical data as Declined. TASK DETAIL https://phabricator.wikimedia.org/T209611 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: So9q, Ladsgroup, GoranSMilovanovic, Ottomata, bmansurov, JAllemandou, Aklapper, Mohamed-Awnallah, Busfault, Astuthiodit_1, Atieno, Gethan, BeautifulBold, EChetty, lbowmaker, Suran38, BTullis, karapayneWMDE, Simonmaignan, Invadibot, Theofpa, Ywats0ns, maantietaja, calbon, Peteosx1x, Anerka, NavinRizwi, jannee_e, ItamarWMDE, Akuckartz, 4748kitoko, holger.knust, Nandana, Akovalyov, Lahi, Gq86, Xinbenlv, Vacio, Capankajsmilyo, Fz-29, Lunewa, QZanden, LawExplorer, elukey, _jensen, rosalieper, Mkdw, Scott_WUaS, notconfusing, gnosygnu, terrrydactyl, Wikidata-bugs, aude, Dinoguy1000, Ricordisamoa, Alchimista, He7d3r, Mbch331, jeremyb, Hokwelum ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T200716: Include ORES predictions in RDF export
elukey closed this task as "Declined". elukey added a comment. We are moving to Lift Wing: https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing I am closing old tasks related to ORES since it is being deprecated, please re-open if you feel that any work could be done on Lift Wing. TASK DETAIL https://phabricator.wikimedia.org/T200716 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Jonas, Lucas_Werkmeister_WMDE, hoo, Multichill, Aklapper, PokestarFan, Esc3300, gerritbot, Lydia_Pintscher, Halfak, Smalyshev, Astuthiodit_1, AWesterinen, Gethan, bking, karapayneWMDE, Simonmaignan, Invadibot, MPhamWMF, maantietaja, calbon, Anerka, ItamarWMDE, Akuckartz, ET4Eva, Nandana, Namenlos314, Lahi, Gq86, Xinbenlv, Vacio, GoranSMilovanovic, Fz-29, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, _jensen, rosalieper, Mkdw, Scott_WUaS, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T166427: Measure maturity/quality of Wikidata items in a way it can be queried
elukey closed subtask T200716: Include ORES predictions in RDF export as Declined. TASK DETAIL https://phabricator.wikimedia.org/T166427 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: dcausse, Ls1g, Lydia_Pintscher, gerritbot, Esc3300, PokestarFan, Smalyshev, Halfak, Aklapper, Multichill, Astuthiodit_1, AWesterinen, Gethan, bking, karapayneWMDE, Simonmaignan, Invadibot, MPhamWMF, maantietaja, calbon, Anerka, Hazizibinmahdi, ItamarWMDE, Akuckartz, ET4Eva, Nandana, Namenlos314, Lahi, Gq86, Xinbenlv, Vacio, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Fz-29, QZanden, EBjune, merbst, LawExplorer, Avner, elukey, Gehel, _jensen, rosalieper, Mkdw, Scott_WUaS, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Alchimista, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T332953: Migrate PipelineLib repos to GitLab
elukey updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T332953 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, KartikMistry, santhosh, Martaannaj, sbassett, bking, bd808, Ladsgroup, Krinkle, Legoktm, tstarling, Physikerwelt, dcausse, Jdrewniak, taavi, hnowlan, Michaelcochez, cjming, Jdforrester-WMF, dduvall, Aklapper, thcipriani, Bellucii32, Mohamed-Awnallah, Itsmeduncan, Cleo_Lemoisson, Astuthiodit_1, lbowmaker, TheReadOnly, BTullis, karapayneWMDE, toberto, joanna_borun, Simonmaignan, Invadibot, MPhamWMF, Ywats0ns, Devnull, maantietaja, calbon, Muchiri124, Confetti68, Anerka, CBogen, ItamarWMDE, Nintendofan885, Akuckartz, Otr500, WDoranWMF, Ddurigon, MJL, brennen, Mateo1977, EvanProdromou, Legado_Shulgin, ReaperDawn, Nandana, NebulousIris, Namenlos314, aezell, skpuneethumar, Zylc, Davinaclare77, Abdeaitali, 1978Gage2001, Techguru.pc, Lahi, Operator873, Gq86, Xinbenlv, Vacio, Sharvaniharan, Bsandipan, scblr, Xover, GoranSMilovanovic, SPoore, TBolliger, Chicocvenancio, Hfbn0, QZanden, EBjune, Tbscho, Taquo, LawExplorer, catalandres, Eginhard, Avner, Zppix, JJMC89, TerraCodes, DDJJ, _jensen, rosalieper, Agabi10, PEarleyWMF, RuyP, Liudvikas, Scott_WUaS, Pchelolo, Karthik_sripal, Izno, Wong128hk, Luke081515, Bsadowski1, Niharika, Wikidata-bugs, Jitrixis, aude, Bawolff, Capt_Swing, Dbrant, Dinoguy1000, Gryllida, Lydia_Pintscher, faidon, Grunny, ssastry, scfc, Alchimista, Arlolra, csteipp, Mbch331, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T332953: Migrate PipelineLib repos to GitLab
elukey updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T332953 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, KartikMistry, santhosh, Martaannaj, sbassett, bking, bd808, Ladsgroup, Krinkle, Legoktm, tstarling, Physikerwelt, dcausse, Jdrewniak, taavi, hnowlan, Michaelcochez, cjming, Jdforrester-WMF, dduvall, Aklapper, thcipriani, Bellucii32, Mohamed-Awnallah, Itsmeduncan, Cleo_Lemoisson, Astuthiodit_1, lbowmaker, TheReadOnly, BTullis, karapayneWMDE, toberto, joanna_borun, Simonmaignan, Invadibot, MPhamWMF, Ywats0ns, Devnull, maantietaja, calbon, Muchiri124, Confetti68, Anerka, CBogen, ItamarWMDE, Nintendofan885, Akuckartz, Otr500, WDoranWMF, Ddurigon, MJL, brennen, Mateo1977, EvanProdromou, Legado_Shulgin, ReaperDawn, Nandana, NebulousIris, Namenlos314, aezell, skpuneethumar, Zylc, Davinaclare77, Abdeaitali, 1978Gage2001, Techguru.pc, Lahi, Operator873, Gq86, Xinbenlv, Vacio, Sharvaniharan, Bsandipan, scblr, Xover, GoranSMilovanovic, SPoore, TBolliger, Chicocvenancio, Hfbn0, QZanden, EBjune, Tbscho, Taquo, LawExplorer, catalandres, Eginhard, Avner, Zppix, JJMC89, TerraCodes, DDJJ, _jensen, rosalieper, Agabi10, PEarleyWMF, RuyP, Liudvikas, Scott_WUaS, Pchelolo, Karthik_sripal, Izno, Wong128hk, Luke081515, Bsadowski1, Niharika, Wikidata-bugs, Jitrixis, aude, Bawolff, Capt_Swing, Dbrant, Dinoguy1000, Gryllida, Lydia_Pintscher, faidon, Grunny, ssastry, scfc, Alchimista, Arlolra, csteipp, Mbch331, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T314835: wdqs space usage on thanos-swift
elukey added a comment. root@thanos-fe1001:/home/elukey# source /etc/swift/account_AUTH_wdqs.env root@thanos-fe1001:/home/elukey# swift list rdf-streaming-updater-codfw rdf-streaming-updater-codfw+segments rdf-streaming-updater-eqiad rdf-streaming-updater-eqiad+segments rdf-streaming-updater-staging thanos-swift updater updater+segments updater-zbyszko updater-zbyszko-v2 root@thanos-fe1001:/home/elukey# swift stat rdf-streaming-updater-codfw | egrep 'Objects|Bytes' Objects: 854053 Bytes: 2383274480337 root@thanos-fe1001:/home/elukey# swift stat rdf-streaming-updater-eqiad | egrep 'Objects|Bytes' Objects: 2605 Bytes: 60359444619 root@thanos-fe1001:/home/elukey# swift stat rdf-streaming-updater-codfw+segments | egrep 'Objects|Bytes' Objects: 3742134 Bytes: 18993738619543 root@thanos-fe1001:/home/elukey# swift stat rdf-streaming-updater-eqiad+segments | egrep 'Objects|Bytes' Objects: 10275 Bytes: 40649663164 So the `rdf-streaming-updater-codfw+segments` bucket/container seems to be storing around 18T if I made the conversion correctly, storing ~4M objects. root@thanos-fe1001:/home/elukey# swift list rdf-streaming-updater-codfw+segments | head commons/checkpoints//shared/XXX/1 (redacted just in case) It seems containing a ton of checkpoints. TASK DETAIL https://phabricator.wikimedia.org/T314835 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, bking, Ottomata, dcausse, LSobanski, dr0ptp4kt, fgiunchedi, Aklapper, Raineydaz, Astuthiodit_1, AWesterinen, MatthewVernon, BTullis, karapayneWMDE, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, Muchiri124, CBogen, ItamarWMDE, Akuckartz, Di3sel1975, Chambersjay, RhinosF1, Legado_Shulgin, ReaperDawn, Nandana, Namenlos314, Conradrock, Davinaclare77, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T266702: Move WDQS UI to microsites
elukey closed this task as "Resolved". elukey claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T266702 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, WMDE-leszek, dcausse, Ladsgroup, BBlack, Gehel, Dzahn, Addshore, Aklapper, 786, Suran38, Biggs657, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, lmata, Juan90264, Muchiri124, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, RhinosF1, joker88john, Legado_Shulgin, ReaperDawn, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Davinaclare77, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, Mahir256, QZanden, EBjune, merbst, LawExplorer, Salgo60, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, faidon, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T266702: Move WDQS UI to microsites
elukey added a comment. Found the issue: diff --git a/maint.html b/maint.html index 703e17a..e63c70e 100644 --- a/maint.html +++ b/maint.html @@ -1 +1 @@ -Error 503 Service Unavailable503 Service UnavailableThe service is temporary down for maintenance. Please check back a bit later. \ No newline at end of file +Error 503 Service Unavailable503 Service UnavailableThe service is temporary down for maintenance. Please check back a bit later.. I assume this was a test, I am going to checkout the correct file so puppet will not complain anymore. TASK DETAIL https://phabricator.wikimedia.org/T266702 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, WMDE-leszek, dcausse, Ladsgroup, BBlack, Gehel, Dzahn, Addshore, Aklapper, 786, Suran38, Biggs657, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, lmata, Juan90264, Muchiri124, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, RhinosF1, joker88john, Legado_Shulgin, ReaperDawn, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Davinaclare77, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, Mahir256, QZanden, EBjune, merbst, LawExplorer, Salgo60, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, faidon, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T266702: Move WDQS UI to microsites
elukey reopened this task as "Open". elukey added a comment. Restricted Application added a project: wdwb-tech. Hi folks, on miscweb1002 I see the following in puppet: Feb 3 07:40:34 miscweb1002 puppet-agent[31666]: (/Stage[main]/Profile::Microsites::Query_service/Git::Clone[wikidata/query/gui-deploy]/Exec[git_pull_wikidata/query/gui-deploy]/returns) executed successfully (corrective) It is repeated for each puppet run and it causes the host to pop up in our alerts :) TASK DETAIL https://phabricator.wikimedia.org/T266702 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, WMDE-leszek, dcausse, Ladsgroup, BBlack, Gehel, Dzahn, Addshore, Aklapper, 786, Suran38, Biggs657, joanna_borun, Invadibot, Lalamarie69, MPhamWMF, Devnull, maantietaja, lmata, Juan90264, Muchiri124, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, RhinosF1, joker88john, Legado_Shulgin, ReaperDawn, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Davinaclare77, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Hfbn0, Mahir256, QZanden, EBjune, merbst, LawExplorer, Salgo60, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, faidon, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T288231: Deploy the wdqs streaming updater to production
elukey added a comment.
Hi folks, I was trying to figure out why we had so many UNKNOWNs related to
wdqs nodes in icinga and I noticed this task. The new streaming updater metrics
seem to have a typo:
`wdqs_streaming_updater_kafka_stream_consumer_lag_Value`
Meanwhile this is the puppet config:
query =>
"scalar(wdqs_streaming_updater_kafka_stream_consumer_lag{instance=\"${::hostname}:9101\"})",
TASK DETAIL
https://phabricator.wikimedia.org/T288231
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: elukey
Cc: elukey, Zbyszko, RKemper, Aklapper, dcausse, Suran38, Biggs657, Invadibot,
Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen,
Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana,
Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan,
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst,
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas,
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)
elukey added a comment. I think we should be fine from now on, I wouldn't add more complexity to what we have :) TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek, GoranSMilovanovic, Invadibot, maantietaja, Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)
elukey added a comment. @GoranSMilovanovic sure! During the migration of the hosts where Hive Server/Metastore runs to Debian Buster, we encountered a lot of problems with the only available java lib for mysql, namely the one containing the `org.mariadb.jdbc.Driver` JDBC driver. We have now reverted back to the old mysql driver, manually porting the missing debian packages from Stretch to Buster, and now sqoop needs to run without the extra --driver option. So this option caused problems due to us trying to figure out how to upgrade our systems following Debian best practices, but hopefully now we should be good (at least until Debian Bullseye, the new version, will be out). I have opened https://issues.apache.org/jira/browse/HIVE-25020 to Hive upstream to investigate further, but it will be a longer topic to solve I am afraid. Lemme know if you want to know more. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek, GoranSMilovanovic, Invadibot, maantietaja, Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)
elukey closed this task as "Resolved". elukey claimed this task. elukey added a comment. No issues from our side, going to close, please reopen if necessary! TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek, GoranSMilovanovic, Invadibot, maantietaja, Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281063: Wikidata Concepts Monitor: some datasets are empty
elukey closed subtask T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again) as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T281063 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic, elukey Cc: elukey, WMDE-leszek, GoranSMilovanovic, Manuel, Aklapper, MisterSynergy, Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)
elukey added a comment. Nice! Let's keep it open since I want to understand if we need to use `--driver com.mysql.jdbc.Driver` or not, it will have some impact also for Analytics, thanks a lot for bringing this up and sorry for the trouble! TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek, GoranSMilovanovic, Invadibot, maantietaja, Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)
elukey added a comment. Hi Goran! We moved the hive server nodes to Debian Buster recently (T231067 <https://phabricator.wikimedia.org/T231067>) and we had a problem with Hive and Mariadb, highlights in https://issues.apache.org/jira/browse/HIVE-25020. As a quick workaround you should be able to unblock your queries just removing `--driver org.mariadb.jdbc.Driver`, can you try to see if it works? The first error is unrelated, it is due to the fact that you have created the /tmp/wmde directory as `analytics-privatedata` (and by default permissions allow only user + group, not others) and your username can't read files in there. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek, GoranSMilovanovic, Invadibot, maantietaja, Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
elukey added a comment. I see https://issues.apache.org/jira/browse/BIGTOP-3382 to add 1.11.1, but it seems that the work will be done after Bigtop 1.5. We could think about contributing to upstream if we feel the need to have that included to Bigtop 1.5! TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
elukey added a comment. root@apt1001:/srv/wikimedia# reprepro lsbycomponent flink flink | 1.6.4-1 | stretch-wikimedia | thirdparty/bigtop14 | amd64 flink | 1.6.4-1 | buster-wikimedia | thirdparty/bigtop14 | amd64 This is Bigtop 1.4 version, the last upstream, but they are about to release 1.5 so flink will probably be more up to date. TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T246004: Spike: Can/should Swift be used as Flink checkpoint backend?
elukey added a comment. elukey@stat1004:~$ telnet thanos-swift.discovery.wmnet 443 Trying 10.2.2.54... Connected to thanos-swift.discovery.wmnet. Escape character is '^]'. ^] :) TASK DETAIL https://phabricator.wikimedia.org/T246004 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, elukey Cc: elukey, EBernhardson, JMeybohm, fgiunchedi, CBogen, #analytics, dcausse, Gehel, Zbyszko, Aklapper, JAllemandou, Smalyshev, Iamamz3, Ottomata, Alter-paule, NavinRizwi, Beast1978, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, DannyS712, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T246004: Spike: Can/should Swift be used as Flink checkpoint backend?
elukey added a comment.
term swift {
from {
destination-address {
/* swift.svc.codfw */
10.2.1.27/32;
/* swift.svc.eqiad */
10.2.2.27/32;
}
protocol tcp;
destination-port 443;
}
then accept;
}
Should I swap the above addresses with 10.2.2.54:443 and 10.2.1.54:443 ?
TASK DETAIL
https://phabricator.wikimedia.org/T246004
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Zbyszko, elukey
Cc: elukey, EBernhardson, JMeybohm, fgiunchedi, CBogen, #analytics, dcausse,
Gehel, Zbyszko, Aklapper, JAllemandou, Smalyshev, Iamamz3, Ottomata,
NavinRizwi, Akuckartz, darthmon_wmde, DannyS712, Nandana, Namenlos314, Lahi,
Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst,
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll,
Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T246004: Spike: Can/should Swift be used as Flink checkpoint backend?
elukey added a comment. Context in T219544 <https://phabricator.wikimedia.org/T219544> - IIRC your team uses a specific oozie util to push data on swift in the coordinator's definition: https://github.com/wikimedia/wikimedia-discovery-analytics/commit/02fddbfc949b665457354b910bef43da9c421412 (see swift_upload occurrences) There seems also to be some airflow code but I don't know much about it. TASK DETAIL https://phabricator.wikimedia.org/T246004 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, elukey Cc: elukey, EBernhardson, JMeybohm, fgiunchedi, CBogen, #analytics, dcausse, Gehel, Zbyszko, Aklapper, JAllemandou, Smalyshev, Iamamz3, Ottomata, NavinRizwi, Akuckartz, darthmon_wmde, DannyS712, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T255282: mw1384 is misbehaving
elukey closed this task as "Resolved". elukey claimed this task. elukey added a comment. Error seem to be gone after the php-fpm restart, but I forgot to dump the status of APC :( My bet is on APC corruption or something similar, will reopen in case it re-happen. TASK DETAIL https://phabricator.wikimedia.org/T255282 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, elukey, Addshore, Ladsgroup, wkandek, JMeybohm, darthmon_wmde, Legado_Shulgin, Nandana, jijiki, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Wong128hk, Wikidata-bugs, aude, faidon, Jdforrester-WMF, Mbch331, Rxy, Jay8g, fgiunchedi, Dzahn ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T255282: mw1384 is misbehaving
elukey added a project: serviceops. TASK DETAIL https://phabricator.wikimedia.org/T255282 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, elukey, Addshore, Ladsgroup, wkandek, JMeybohm, darthmon_wmde, Legado_Shulgin, Nandana, jijiki, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Wong128hk, Wikidata-bugs, aude, faidon, Jdforrester-WMF, Mbch331, Rxy, Jay8g, fgiunchedi, Dzahn ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T255282: mw1384 is misbehaving
elukey added a comment. To add some details - I depooled/repooled doing `scap pull` and it didn't really work. I also depooled, waited hours, repooled and the issue gets back consistently. It may be a single host issue, but not really sure what. TASK DETAIL https://phabricator.wikimedia.org/T255282 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Aklapper, elukey, Addshore, Ladsgroup, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Wong128hk, Wikidata-bugs, aude, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T246343: Service implementation on wdqs200[7-8].codfw.wmnet
elukey added a comment. - data+categories jnl files copied for both hosts via cookbook - restarted nginx to pick up new mapping for categories (after David's suggestion) - 2007's lag seems ok, 2008 still catching up - both hosts are not pooled in LVS/Pybal TASK DETAIL https://phabricator.wikimedia.org/T246343 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, elukey Cc: elukey, Aklapper, Gehel, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T243871: Long query running on dbstore1005:3318
elukey closed this task as "Resolved". elukey claimed this task. elukey added a comment. After a chat with @Marostegui I decided to kill the query, since it seems that some optimization is needed (it has been running for 40+ days). Also it is a select and can be re-done in case :) Thanks a lot @Addshore for the follow up! TASK DETAIL https://phabricator.wikimedia.org/T243871 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Addshore, Ladsgroup, elukey, Aklapper, Marostegui, 4748kitoko, darthmon_wmde, Nandana, Akovalyov, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, JAllemandou, terrrydactyl, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T237319: 502 errors on ATS/8.0.5
elukey added a comment. In T237319#5681384 <https://phabricator.wikimedia.org/T237319#5681384>, @darthmon_wmde wrote: > Is there anything that we can quickly do on wikibase to fix this? > if so, please advise what concretely. > Thanks! @darthmon_wmde hi! So the issue seems to be that URLs like www.wikidata.org/wiki/Special:EntityData/Q38646387.json might lead to a HTTP 304 response with a body, that violates HTTP specs. This is currently not handled by the Apache httpd version that we use on Debian Stretch (namely proactively dropping the body in case it is found) so a fix in our code would definitely help. TASK DETAIL https://phabricator.wikimedia.org/T237319 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: darthmon_wmde, elukey, Addshore, WMDE-leszek, Ladsgroup, CDanis, Joe, Vgutierrez, ema, Nikerabbit, DannyS712, Aklapper, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Wong128hk, Wikidata-bugs, aude, Lydia_Pintscher, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T237319: 502 errors on ATS/8.0.5
elukey added a comment. Thanks, will remember to use -v :) On buster + sury's php7.2-fpm + apache 2.4.38 I cannot repro: root@497a89a9ac42:/# curl localhost/test.php -v * Expire in 0 ms for 6 (transfer 0x5604525d2f50) * Expire in 1 ms for 1 (transfer 0x5604525d2f50) * Expire in 0 ms for 1 (transfer 0x5604525d2f50) * Expire in 1 ms for 1 (transfer 0x5604525d2f50) * Trying 127.0.0.1... * TCP_NODELAY set * Expire in 149997 ms for 3 (transfer 0x5604525d2f50) * Expire in 200 ms for 4 (transfer 0x5604525d2f50) * Connected to localhost (127.0.0.1) port 80 (#0) > GET /test.php HTTP/1.1 > Host: localhost > User-Agent: curl/7.64.0 > Accept: */* > < HTTP/1.1 304 Not Modified < Date: Fri, 22 Nov 2019 07:22:15 GMT < Server: Apache/2.4.38 (Debian) < * Connection #0 to host localhost left intact I thought to use git bisect to nail down the commit that fixes the problem, but then I remembered http://svn.apache.org/r1837056 (plus https://svn.apache.org/r1843242 as follow up for mod_brotli/deflate). The fix handles the 304/204 body problem in the http output filter, so after all the content generators (like mod_proxy_fcgi) and before sending the response to the external client. The change is not small and contained in one file, but we could think about adding it to our current httpd debian stretch version while waiting for Buster for appservers. TASK DETAIL https://phabricator.wikimedia.org/T237319 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: darthmon_wmde, elukey, Addshore, WMDE-leszek, Ladsgroup, CDanis, Joe, Vgutierrez, ema, Nikerabbit, DannyS712, Aklapper, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Wong128hk, Wikidata-bugs, aude, Lydia_Pintscher, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T237319: 502 errors on ATS/8.0.5
elukey added a comment. I have created a Docker image for Debian stretch installing apache2 (same version of the mw app servers) + php7.2-fpm from Sury's repo + the following trivial config: Options Indexes FollowSymLinks AllowOverride None Require all granted SetHandler "proxy:unix:/run/php/php7.2-fpm.sock|fcgi://localhost" and root@75ab41fd13ca:/# cat /var/www/test.php https://phabricator.wikimedia.org/T237319 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: darthmon_wmde, elukey, Addshore, WMDE-leszek, Ladsgroup, CDanis, Joe, Vgutierrez, ema, Nikerabbit, DannyS712, Aklapper, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Wong128hk, Wikidata-bugs, aude, Lydia_Pintscher, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T237319: 502 errors on ATS/8.0.5
elukey added a comment. On mwdebug2001 this seems to be the mod proxy fcgi debug: Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:debug] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(913): [client 10.192.0.98:37452] AH01076: url: fcgi://localhost/srv/mediawiki/docroot/wikidata.org/w/index.php proxyname: (null) proxyport: 0 Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:debug] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(920): [client 10.192.0.98:37452] AH01078: serving URL fcgi://localhost/srv/mediawiki/docroot/wikidata.org/w/index.php Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'UNIQUE_ID' value 'REDACTED' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_AUTHORIZATION' value '' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_CONTENT_TYPE' value '' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_CONTENT_LENGTH' value '' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SCRIPT_URL' value '/wiki/Special:EntityData/Q38646387.json' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SCRIPT_URI' value 'http://www.wikidata.org/wiki/Special:EntityData/Q38646387.json' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'RW_PROTO' value 'http' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_HOST' value 'www.wikidata.org' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_USER_AGENT' value 'curl/7.52.1' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_ACCEPT' value '*/*' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'HTTP_IF_MODIFIED_SINCE' value 'Sat, 30 Mar 2019 07:05:28 GMT' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'PATH' value '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SERVER_SIGNATURE' value '' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SERVER_SOFTWARE' value 'mwdebug2001.codfw.wmnet' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SERVER_NAME' value 'www.wikidata.org' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SERVER_ADDR' value '10.192.0.98' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'SERVER_PORT' value '80' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'REMOTE_ADDR' value '10.192.0.98' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'DOCUMENT_ROOT' value '/srv/mediawiki/docroot/wikidata.org' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid 140373593552640] mod_proxy_fcgi.c(308): [client 10.192.0.98:37452] AH01062: sending env var 'REQUEST_SCHEME' value 'http' Nov 21 12:02:13 mwdebug2001 apache2[4516]: [proxy_fcgi:trace8] [pid 4516:tid
[Wikidata-bugs] [Maniphest] [Commented On] T236895: ArticlePlaceholder dashboard stopped tracking page views
elukey added a comment. Hello! Do you have a coordinator that I can check in Hue? https://hue.wikimedia.org/oozie/list_oozie_coordinator/0015017-190822093211873-oozie-oozi-C/ seems the closest one and it works afaics.. TASK DETAIL https://phabricator.wikimedia.org/T236895 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, Addshore, Aklapper, Lydia_Pintscher, 4748kitoko, darthmon_wmde, DannyS712, Nandana, Akovalyov, Lahi, Gq86, GoranSMilovanovic, QZanden, cmadeo, LawExplorer, _jensen, rosalieper, Jonas, JAllemandou, terrrydactyl, Wikidata-bugs, aude, jayvdb, Ricordisamoa, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888
elukey added subscribers: WMDE-leszek, Ladsgroup. elukey added a comment. Adding @WMDE-leszek and @Ladsgroup since afaics they were/are working on this :) TASK DETAIL https://phabricator.wikimedia.org/T176875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Ladsgroup, WMDE-leszek, ayounsi, akosiaris, Ottomata, elukey, Smalyshev, Gehel, Addshore, Aklapper, darthmon_wmde, ET4Eva, Legado_Shulgin, DannyS712, Nandana, thifranc, AndyTan, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Avner, Zppix, _jensen, rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888
elukey added a comment.
Changed the following: (Cc: @ayounsi )
elukey@re0.cr2-eqiad# show | compare
[edit firewall family inet filter analytics-in4 term wdqs from
destination-address]
10.192.48.65/32 { ... }
+/* wdqs.svc.eqiad.wmnet */
+10.2.2.32/32;
Now I can see telnet reaching the endpoint from stat1007, but getting
connection refused:
elukey@stat1007:~$ telnet wdqs.svc.eqiad.wmnet
Trying 10.2.2.32...
telnet: Unable to connect to remote host: Connection refused
I guess that something more is needed?
TASK DETAIL
https://phabricator.wikimedia.org/T176875
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: elukey
Cc: ayounsi, akosiaris, Ottomata, elukey, Smalyshev, Gehel, Addshore, Aklapper,
darthmon_wmde, ET4Eva, Legado_Shulgin, DannyS712, Nandana, thifranc, AndyTan,
Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Darkminds3113,
Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune,
merbst, LawExplorer, Avner, Zppix, _jensen, rosalieper, Jonas, FloNight,
Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984,
Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888
elukey added a subscriber: ayounsi. elukey added a comment. Not really, I wish myself from the past added more info. I asked to @ayounsi and he didn't come up with a reason not to, so in theory we could try to modify the term on the firewall and see how it goes. TASK DETAIL https://phabricator.wikimedia.org/T176875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: ayounsi, akosiaris, Ottomata, elukey, Smalyshev, Gehel, Addshore, Aklapper, darthmon_wmde, ET4Eva, Legado_Shulgin, Nandana, thifranc, AndyTan, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Avner, Zppix, _jensen, rosalieper, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T223310: Investigate increase in tx bandwidth usage for mc1033
elukey added a comment. Thanks a lot for the deploy! I checked metrics and nothing seems changed :( TASK DETAIL https://phabricator.wikimedia.org/T223310 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Krenair, Tarrow, alaa_wmde, Catrope, kostajh, Joe, Krinkle, aaron, jijiki, Aklapper, akosiaris, elukey, Addshore, E.S.A-Sheild, darthmon_wmde, joker88john, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, WSH1906, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Taiwania_Justo, Jonas, Wong128hk, Wikidata-bugs, aude, Lydia_Pintscher, faidon, Mbch331, Jay8g, fgiunchedi, Dzahn ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T223310: Investigate increase in tx bandwidth usage for mc1033
elukey added a comment. Would it be possible to deploy https://gerrit.wikimedia.org/r/511612 before the weekly train? TASK DETAIL https://phabricator.wikimedia.org/T223310 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: Tarrow, alaa_wmde, Catrope, kostajh, Joe, Krinkle, aaron, jijiki, Aklapper, akosiaris, elukey, Addshore, E.S.A-Sheild, darthmon_wmde, joker88john, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, WSH1906, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Taiwania_Justo, Jonas, Wong128hk, Wikidata-bugs, aude, Lydia_Pintscher, faidon, Mbch331, Jay8g, fgiunchedi, Dzahn ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata
elukey added a comment. Hello everybody, I am investigating https://phabricator.wikimedia.org/T223310, namely the one of the Redis MainStash being constantly hammered by GET to `global:Wikimedia\Rdbms\ChronologyProtector` since after the deployment of 1.34.0-wmf.3. Is it related to https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventBus/+/504819/ by any chance? TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, elukey Cc: elukey, Krinkle, Ottomata, Aklapper, Smalyshev, aaron, E.S.A-Sheild, darthmon_wmde, alaa_wmde, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Vali.matei, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Fix inefficient CacheAwarePropertyInfoStore memcached access pattern
elukey added a comment. From a quick look with memkeys on mc1033, these are the top talkers keys: memcache key calls objsize req/sec bw(kbps) WANCache:v:global:CacheAwarePropertyInfoStore:wikidatawiki:P248 40146392 8921.40 6566.15 WANCache:v:global:CacheAwarePropertyInfoStore:wikidatawiki:P2093 33892385 7531.62 5121.50 WANCache:v:global:CacheAwarePropertyInfoStore:wikidatawiki:P4656 3602982 800.64525.22 WANCache:v:global:CacheAwarePropertyInfoStore:wikidatawiki:P214 35361 150 785.80942.96 Is it something that was changed recently? TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, elukey Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, aude, daniel, Aklapper, hoo, darthmon_wmde, alaa_wmde, joker88john, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, WSH1906, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T119536: wikidata.org/entity/Q12345 should do content negotiation immediately, instead of redirecting to wikidata.org/wiki/Special:EntityData/Q36661 first
elukey removed a project: User-Elukey. TASK DETAIL https://phabricator.wikimedia.org/T119536 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey, PokestarFan, Ladsgroup, gerritbot, Esc3300, thiemowmde, hoo, elf-pavlik, JanZerebecki, Aklapper, StudiesWorld, daniel, alaa_wmde, joker88john, DannyS712, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, WSH1906, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, TomT0m, _jensen, Urbanecm, rosalieper, JEumerus, Ananthsubray, Tulsi_Bhagat, suriyaa, Wong128hk, Luke081515, SimmeD, Wikidata-bugs, Snowolf, aude, Dcljr, Southparkfan, fbstj, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Glaisher, Krenair ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Fix inefficient CacheAwarePropertyInfoStore memcached access pattern
elukey added a comment. The train was deployed yesterday, I can see an awesome improvement (green bar dropping): F28597246: Screen Shot 2019-04-09 at 8.09.44 AM.png <https://phabricator.wikimedia.org/F28597246> Thanks a lot for all the work! TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, aude, daniel, Aklapper, hoo, alaa_wmde, joker88john, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Blocker] T215413: Image Classification Working Group
elukey changed the status of subtask T148843: Remove computational bottlenecks in stats machine via adding a GPU that can be used to train ML models from Stalled to Open. TASK DETAIL https://phabricator.wikimedia.org/T215413 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Miriam, elukey Cc: MoritzMuehlenhoff, CDanis, akosiaris, SandraF_WMF, Fuzheado, PDrouin-WMF, Krenair, d.astrikov, JoeWalsh, Nirzar, dcausse, fgiunchedi, JAllemandou, leila, Capt_Swing, mpopov, Nuria, DarTar, Halfak, Gilles, EBernhardson, dr0ptp4kt, Harej, MusikAnimal, Abit, elukey, diego, Cparle, Ramsey-WMF, Miriam, Isaac, alaa_wmde, Nandana, JKSTNK, Akovalyov, Lahi, Gq86, E1presidente, Anooprao, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, V4switch, LawExplorer, Avner, Silverfish, _jensen, rosalieper, Susannaanas, Wong128hk, Jane023, Wikidata-bugs, Base, matthiasmullie, aude, Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Matanya, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Blocker] T215413: Image Classification Working Group
elukey changed the status of subtask T148843: Remove computational bottlenecks in stats machine via adding a GPU that can be used to train ML models from Open to Stalled. TASK DETAIL https://phabricator.wikimedia.org/T215413 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Miriam, elukey Cc: MoritzMuehlenhoff, CDanis, akosiaris, SandraF_WMF, Fuzheado, PDrouin-WMF, Krenair, d.astrikov, JoeWalsh, Nirzar, dcausse, fgiunchedi, JAllemandou, leila, Capt_Swing, mpopov, Nuria, DarTar, Halfak, Gilles, EBernhardson, dr0ptp4kt, Harej, MusikAnimal, Abit, elukey, diego, Cparle, Ramsey-WMF, Miriam, Isaac, alaa_wmde, Nandana, JKSTNK, Akovalyov, Lahi, Gq86, E1presidente, Anooprao, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, V4switch, LawExplorer, Avner, Silverfish, _jensen, rosalieper, Susannaanas, Wong128hk, Jane023, Wikidata-bugs, Base, matthiasmullie, aude, Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Matanya, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T97368: Fix inefficient CacheAwarePropertyInfoStore memcached access pattern
elukey added a project: User-Elukey. TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T218197: CacheAwarePropertyInfoStore & CachingPropertyInfoLookup should use WANObjectCache instead of BagOStuff
elukey added a comment. Adding also @aaron that knows better what is best (and hopefully can confirm what I am saying). TASK DETAIL https://phabricator.wikimedia.org/T218197 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: aaron, Aklapper, elukey, Addshore, alaa_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Fix inefficient CacheAwarePropertyInfoStore memcached access pattern
elukey added a comment. To summarize what I am currently seeing: - after the last mw deployment/train, the key got renamed and moved to mc1022 (was before on mc1021). This can be seen in https://grafana.wikimedia.org/d/00316/memcache?panelId=45=1=1551860994585=1552057232883 - when the key expires or gets evicted from the memcache LRU, then a bursts of SETs hits mc1022. This matches with mcrouter's timeouts, that eventually lead to the shard being marked as temporary down for all memcached traffic and hence spikes in mediawiki exceptions. - there is a constant high volume of GETs that raises the mc1022's baseline a lot and gets us closer to what the 1G NIC on the host can handle. We haven't observed any sign of packet dropped due to saturation but in theory a normal increase of api/appservers traffic could lead to that. To solve the bursts of SETs we could use WANObjectCache (https://doc.wikimedia.org/mediawiki-core/master/php/classWANObjectCache.html) instead of using BagOfStuff directly, since it offers some nice functionalities to prevent, for example, that all the api/appservers try to SET a key if not there. We did the same with the Mediawiki Translate extension in T203786 <https://phabricator.wikimedia.org/T203786>, solving a lot of problems. TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T97368: Fix inefficient CacheAwarePropertyInfoStore memcached access pattern
elukey added a subscriber: aaron.
elukey added a comment.
Another big SET bursts to mc1022 caused timeouts for the MW appservers..
Don't meant to pressure you guys but is there any plan forward to fix this?
The SET bursts should be due to a key either being evicted or expired:
elukey@mc1022:~$ cat slab_140 | grep -i wikidata
ITEM
wikibase_shared/1_33_0-wmf_20-wikidatawiki-hhvm:CacheAwarePropertyInfoStore
[100621 b; 1552491470 s]
ITEM
wikibase_shared/1_33_0-wmf_20-wikidatawiki-hhvm:WikiPageEntityRevisionLookup:Q56701805
[99611 b; 1552430549 s]
elukey@mc1022:~$ date -d @1552491470
Wed Mar 13 15:37:50 UTC 2019 <= TTL 24h should
match with this being just re-created
elukey@mc1022:~$ date -d @1552430549
Tue Mar 12 22:42:29 UTC 2019
elukey@mc1022:~$ date
Tue Mar 12 15:39:31 UTC 2019
Now I am wondering how the key gets re-set, it might be something to tune in
the usage of the MWWanCache. Adding @aaron as FYI :)
Mcrouter timeouts/TKO registered:
https://grafana.wikimedia.org/d/00549/mcrouter?panelId=9=1=eqiad%20prometheus%2Fops=appserver=All=1552402906212=1552405067164
Slab 140 SET traffic:
https://grafana.wikimedia.org/d/00317/memcache-slabs?panelId=49=1=1552403014459=1552404751954=eqiad%20prometheus%2Fops=memcached=mc1022=All
TASK DETAIL
https://phabricator.wikimedia.org/T97368
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Addshore, elukey
Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot,
Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki,
thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD,
Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo,
Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30,
Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis,
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden,
LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper,
Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Investigate more efficient memcached solution for CacheAwarePropertyInfoStore
elukey added a comment. I found another occurrence of timeouts in mcrouter with SETs for this key, it might be due to TTL expiring or similar. Reducing the traffic volume to memcached is surely a good thing. TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, elukey Cc: elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Investigate more efficient memcached solution for CacheAwarePropertyInfoStore
elukey added a comment. Correction: the last bursts of mcrouter's timeouts seems to match a high set rate for slab 140 on mc1022, that is exactly where the key is: https://grafana.wikimedia.org/d/00317/memcache-slabs?orgId=1=1552231823147=1552232692113=eqiad%20prometheus%2Fops=memcached=mc1022=All=49 Will add more data tomorrow :) TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, elukey Cc: elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Reopened] T97368: Investigate more efficient memcached solution for CacheAwarePropertyInfoStore
elukey reopened this task as "Open". TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, elukey Cc: elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T97368: Investigate more efficient memcached solution for CacheAwarePropertyInfoStore
elukey added a comment. Quick comment: I noticed while checking some memcached metrics that `wikibase_shared/1_33_0-wmf_20-wikidatawiki-hhvm:CacheAwarePropertyInfoStore` generates 30~60MB/s of GET traffic since the last deployment of mediawiki: https://grafana.wikimedia.org/d/00317/memcache-slabs?orgId=1=now-7d=now=eqiad%20prometheus%2Fops=memcached=mc1022=140=60 Confirmed also via memkeys. Given what happened in T204083 <https://phabricator.wikimedia.org/T204083>, is this meant to generate this kind of traffic volume? If so, will it increase over time, stay steady, etc..? I don't see impact on the network interface or correlation with timeouts, but better safe and ask :) TASK DETAIL https://phabricator.wikimedia.org/T97368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore, elukey Cc: elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot, Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki, thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack
elukey added a comment. ge-6/0/25 - mc1019 ge-6/0/26 - mc1020 ge-6/0/27 - mc1021 ge-6/0/28 - mc1022 ge-6/0/29 - mc1023 The above ones are holding the eqiad mediawiki object cache, they are extremely important. IIUC this maintenance will not require any host shutdown, that would mean stopping memcached and wipe the cache (please @ayounsi confirm that :), so a network blip of one at the time will be fine. Please do it one at the time, checking with me or Joe that nothing is exploding on the mediawiki side while the maintenance is ongoing. TASK DETAIL https://phabricator.wikimedia.org/T187960 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Cmjohnson, elukey Cc: thcipriani, akosiaris, Joe, fgiunchedi, hashar, Krinkle, ArielGlenn, jijiki, Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T187960: Rack/cable/configure asw2-a-eqiad switch stack
elukey added a comment. About Analytics nodes: - ge-1/0/7 - kafka-jumbo1001 -> Kafka needs to be stopped ~10/15 minutes beforehand to have a graceful shutdown (if possible) - ge-2/0/12 - kafka-jumbo1002 -> same thing, and it is better to have kafka-jumbo1001 up and running (with Kafka partitions sync recovered) before doing maintenance on 1002. We can survive two brokers down but since we control the maintenance I'd prefer not :) - ge-2/0/17 - analytics1012 - ge-2/0/18 - analytics1013 - ge-2/0/19 - notebook1002 -> all these are decommed nodes, should we move them or just unrack them? - ge-3/0/26 - analytics1052 - ge-3/0/27 - analytics1053 - ge-3/0/28 - analytics1054 - ge-3/0/29 - analytics1055 - ge-3/0/30 - analytics1056 - ge-3/0/31 - analytics1057 - ge-3/0/33 - analytics1059 - ge-3/0/34 - analytics1060 -> all hadoop worker nodes, one at the time would be great - ge-4/0/0 - druid1001 - depool va pybal --> it is not pooled in pybal (druid100[4-6] are), some heads up beforehand to gracefully stop the daemons would be good. - ge-4/0/1 - aqs1004 - depool va pybal -> some heads up beforehand to gracefully stop cassandra would be good. - ge-4/0/6 - analytics1070 -> hadoop worker node, no special needs - ge-4/0/25 - kafka1001 - depool from pybal --> Kafka would need to be gracefully stopped if possible beforehand - ge-6/0/12 - druid1004 - depool via pybal -> heads up beforehand would be nice to gracefully stop daemons - ge-6/0/15 - an-master1001 - status planned -> This is the hadoop master node, extremely important, I need to perform a failover beforehand. - ge-6/0/24 - aqs1007 - depool from pybal -> need some time to gracefully stop cassandra, and please do not do it when aqs1004 is down as well. TASK DETAIL https://phabricator.wikimedia.org/T187960 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Cmjohnson, elukey Cc: thcipriani, akosiaris, Joe, fgiunchedi, hashar, Krinkle, ArielGlenn, jijiki, Addshore, MMiller_WMF, Catrope, elukey, Marostegui, Stashbot, Paladox, gerritbot, Aklapper, BBlack, Cmjohnson, ayounsi, alaa_wmde, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, AndyTan, kostajh, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Soum213, Taiwania_Justo, Thibaut120094, Wong128hk, Wikidata-bugs, aude, Southparkfan, mark, Lydia_Pintscher, Darkdadaah, faidon, Nikerabbit, Arrbee, santhosh, KartikMistry, Jdforrester-WMF, Mbch331, Jay8g, Ltrlg ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T213894: Modify analytics/wmde/scripts repo to work with the new analytics multisource db setup
elukey added a comment. In T213894#4967148, @Ladsgroup wrote: One imporant thing. Ports needs to be addressed too. I completely missed those. 331 + the digit of the section in case of sX. Example: s5 will be accessible to s5-analytics-replica.eqiad.wmnet:3315 3320 for x1. Example: x1-analytics-replica.eqiad.wmnet:3320 3350 for staging You can also use the DNS PTR records (if the DNS library/code supports those type of queries), so you'll get hostname+port without hardcoding anything.TASK DETAILhttps://phabricator.wikimedia.org/T213894EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, elukeyCc: Ottomata, elukey, GoranSMilovanovic, Ladsgroup, gerritbot, Lydia_Pintscher, Lea_WMDE, Addshore, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, tmletzko, Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, KasiaWMDE, Luke081515, kai.nissen, Bmueller, Wikidata-bugs, aude, jayvdb, Tobi_WMDE_SW, JeroenDeDauw, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T213894: Modify analytics/wmde/scripts repo to work with the new analytics multisource db setup
elukey added a parent task: T215589: Migrate users to dbstore100[3-5]. TASK DETAILhttps://phabricator.wikimedia.org/T213894EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: Ottomata, elukey, GoranSMilovanovic, Ladsgroup, gerritbot, Lydia_Pintscher, Lea_WMDE, Addshore, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, tmletzko, Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, KasiaWMDE, Luke081515, kai.nissen, Bmueller, Wikidata-bugs, aude, jayvdb, Tobi_WMDE_SW, JeroenDeDauw, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T213894: Modify analytics/wmde/scripts repo to work with the new analytics multisource db setup
elukey added a comment. @GoranSMilovanovic feel free to ping me anytime on IRC if you have any doubt :) analytics-slave.eqiad.wmnet points to db1108 that contains the log database, it will not change with this move. The only one changing is analytics-store.eqiad.wmnet, that currently points to dbstore1002 that is going to be decommed before April (hopefully). We are going to replace it with several CNAMEs, all listed in https://wikitech.wikimedia.org/wiki/Analytics/Data_access#MariaDB_replicas (basically one for each section + x1 + staging). For the CloudVPS part, I guess that you are using the labsdb replicas, if so those will not change (but I'd need to see what config you guys use in there to be sure).TASK DETAILhttps://phabricator.wikimedia.org/T213894EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: Ottomata, elukey, GoranSMilovanovic, Ladsgroup, gerritbot, Lydia_Pintscher, Lea_WMDE, Addshore, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, tmletzko, Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, KasiaWMDE, Luke081515, kai.nissen, Bmueller, Wikidata-bugs, aude, jayvdb, Tobi_WMDE_SW, JeroenDeDauw, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T204415: Query stats dashboard not updating
elukey added a comment. Yep exactly, cache misc (where query.wikidata.org was hosted) has been migrated to cache text, therefore all the Hive queries (and related) should be using 'webrequest_text' from now on.TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Nuria, elukeyCc: elukey, Nuria, mpopov, chelsyx, Aklapper, Addshore, Smalyshev, Lydia_Pintscher, Akovalyov, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Jonas, Xmlizer, JAllemandou, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T200954: Lost access to archiva
elukey closed this task as "Resolved".elukey claimed this task.elukey added a comment. There you go: elukey@stat1005:/home/smalyshev$ ls -l archiva.txt -rw--- 1 smalyshev root 40 Aug 2 07:16 archiva.txt Please re-open of follow up with me on IRC if it doesn't work :)TASK DETAILhttps://phabricator.wikimedia.org/T200954EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, TerraCodes, Liuxinyu970226, Ottomata, Gehel, Aklapper, Smalyshev, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Lowered Priority] T200954: Lost access to archiva
elukey lowered the priority of this task from "Unbreak Now!" to "Normal".elukey added a comment. Hello! The password changed due to an accidental leak to gerrit. We are working on a long term solution that will allow us to use LDAP with Archiva to authenticate, but for the moment I think that the easiest solution is for me to create a file to stat1005 on your home directory (readable only by your user) so you can get the updated pass. How does it sound?TASK DETAILhttps://phabricator.wikimedia.org/T200954EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, TerraCodes, Liuxinyu970226, Ottomata, Gehel, Aklapper, Smalyshev, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888
elukey added a comment. After a chat with Discovery we ended up refreshing the list of hosts in the Analytics VLAN firewall (that is meant for traffic from the analytics hosts towards production, like stat1005 to wdqs): https://phabricator.wikimedia.org/T198623#4396997 It seems that it is not possible to whitelist only the VIP IP wdqs.svc.eqiad.wmnetTASK DETAILhttps://phabricator.wikimedia.org/T176875EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, Smalyshev, Gehel, Addshore, Aklapper, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Avner, Zppix, Jonas, FloNight, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173710: Job queue is increasing non-stop
elukey added a comment. Status: elukey@terbium:~$ mwscript extensions/WikimediaMaintenance/getJobQueueLengths.php |sort -n -k2 | tail -n 20 euwiki 237 tgwiki 3759 cawiki 4822 enwiktionary 17148 zhwiki 19958 nowiki 21167 wikidatawiki 28257 bewiki 110296 arwiki 132139 ukwiki 132246 dewiki 155322 svwiki 179250 frwiki 214327 hywiki 504377 itwiki 512539 cewiki 593156 enwiki 654998 ruwiki 5274159 commonswiki 8059943 Total 16619065TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T173710: Job queue is increasing non-stop
elukey added a comment. In T173710#3730226, @EBernhardson wrote: It was perhaps noted before, but because of the recursive nature of the refreshLinks and htmlCacheUpdate jobs even if the backlog is being processed it may not look like it, because the jobs are just enqueing new jobs. Will probably take some time to really know what effect things are having. We are basically lagging 2/3 days in executing jobs, but the queue keeps growing and I have no idea if we are facing something like T129517#2128754 or a 'genuine' (recursive) job enqueue explosion due to a template modification or similar. https://gerrit.wikimedia.org/r/#/c/385248 should be already working for commons, but from mwlog1001's runJob.log I can only see stuff like causeAction=unknown causeAgent=unknown (that probably only confirms that no authenticated user/bot is triggering these jobs iteratively).TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173710: Job queue is increasing non-stop
elukey added a comment. In T173710#3720358, @EBernhardson wrote: All jobs have a requestId parameter, which is passed down through the execution chain. This is the same as the reqId field in logstash. Basically this means if the originating request logged anything to logstash, you should be able to find it with the query type:mediawiki reqId:x and looking for the very first message. That assumes of course the initial request logged anything. Thanks! I tried to spot check in logstash but I am able to see only the request that starts from the jobrunner (the one executing the job), not much more .. :( https://gerrit.wikimedia.org/r/#/c/385248 is really really promising, not sure when it will be deployed but it would surely help in finding quickly a massive template change or similar.TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173710: Job queue is increasing non-stop
elukey added a comment. We had some relief after the last change in the configs of the jobrunners, namely the queue started shrinking, but then we got back into the bad behavior in which we have constantly more jobs enqueued vs completed: F10519970: Screen Shot 2017-10-30 at 6.19.11 PM.png I am currently seeing some big rootjobs with timestamp around Oct 27th that keep seeing jobs executed, but I failed to track down what it has generated them. If anybody has any idea about what procedure to follow to track down the root cause of this job queue increase please come forward :)TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173710: Job queue is increasing non-stop
elukey added a comment. In T173710#3717940, @Jack_who_built_the_house wrote: On ruwiki, many editors are complaining about slow updating of pages with their templates. We have a huge job queue, and it keeps growing day by day, while no top-used templates/modules have been changed in the last days. Please tell, is there any advice that could be given to us, as well as other local communities suffering from this? Hi! We are trying to solve the issue from two sides, namely trying to produce less jobs and prioritizing more the consumption of the current backlog (mostly htmlCacheUpdate and RefreshLinks jobs). At the moment I don't think there is any good advice for local communities, we are hoping to reduce the backlog soon but it might take a while :(TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T173710: Job queue is increasing non-stop
elukey added a comment.
Updated status:
elukey@terbium:~$ /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/group1.dblist showJobs.php --group | awk '{if ($3 > 1) print $_}'
cawiki: refreshLinks: 13566 queued; 6 claimed (6 active, 0 abandoned); 0 delayed
commonswiki: refreshLinks: 1991671 queued; 1 claimed (1 active, 0 abandoned); 0 delayed
commonswiki: htmlCacheUpdate: 3760683 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
elukey@terbium:~$ /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/group2.dblist showJobs.php --group | awk '{if ($3 > 1) print $_}'
arwiki: refreshLinks: 120524 queued; 4 claimed (4 active, 0 abandoned); 0 delayed
bewiki: refreshLinks: 34551 queued; 3 claimed (3 active, 0 abandoned); 0 delayed
cewiki: refreshLinks: 142590 queued; 5 claimed (5 active, 0 abandoned); 0 delayed
cewiki: htmlCacheUpdate: 150593 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
dewiki: htmlCacheUpdate: 11027 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
enwiki: refreshLinks: 69933 queued; 4 claimed (4 active, 0 abandoned); 0 delayed
enwiki: htmlCacheUpdate: 127930 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
frwiki: refreshLinks: 41595 queued; 5 claimed (5 active, 0 abandoned); 0 delayed
hywiki: refreshLinks: 95960 queued; 3 claimed (3 active, 0 abandoned); 0 delayed
itwiki: refreshLinks: 240479 queued; 1 claimed (1 active, 0 abandoned); 0 delayed
itwiki: htmlCacheUpdate: 70493 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
ruwiki: refreshLinks: 985639 queued; 2 claimed (2 active, 0 abandoned); 0 delayed
ruwiki: htmlCacheUpdate: 1928674 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
svwiki: refreshLinks: 679490 queued; 8 claimed (8 active, 0 abandoned); 0 delayed
We could try to increment the number of runners dedicated to refreshLinks and htmlCacheUpdate and see if we manage to process the backlog?TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: zhuyifei1999, RP88, Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lahi, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T173710: Job queue is increasing non-stop
elukey added a project: User-Elukey. TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: Jack_who_built_the_house, elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T173710: Job queue is increasing non-stop
elukey added a comment.
Updated list (showjob1.txt contains group1, showjob.txt group2)
elukey@terbium:~$ awk '{if ($3 > 10) print $_}' showjob1.txt
commonswiki: refreshLinks: 1629991 queued; 5 claimed (2 active, 3 abandoned); 0 delayed
commonswiki: htmlCacheUpdate: 2470968 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
elukey@terbium:~$ awk '{if ($3 > 10) print $_}' showjob.txt
arwiki: refreshLinks: 142867 queued; 3 claimed (3 active, 0 abandoned); 0 delayed
itwiki: refreshLinks: 198068 queued; 5 claimed (5 active, 0 abandoned); 0 delayed
ruwiki: refreshLinks: 807144 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
ruwiki: htmlCacheUpdate: 916539 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
svwiki: refreshLinks: 1582067 queued; 4 claimed (4 active, 0 abandoned); 0 delayed
svwiki: htmlCacheUpdate: 186069 queued; 0 claimed (0 active, 0 abandoned); 0 delayed
The job queue is currently around 8/9M elements and the trend doesn't seem to improve.
In T173710#3646384, @Joe wrote:
I think we might be able to add some capacity to processing those jobs on monday, but we probably have either to re-think the approach to the problem or throw more hardware at it.
Just as FYI we added new capacity in T165519 (now we have 19 jobrunners running in eqiad), but in theory we'd need to eventually decom mw[1161-1167] for T177387 to free some rack space and complete T165519 (other jobrunners will eventually be added as part of the task but we need to free rack space first).TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, debt, mxn, Daimona, YOUR1, Tbayer, mobrovac, Nikerabbit, Mholloway, Legoktm, ema, Joe, GWicke, Nemo_bis, Andreasmperu, BBlack, Peachey88, Liuxinyu970226, daniel, Stashbot, Agabi10, Daniel_Mietchen, XXN, Pasleim, Bugreporter, Sjoerddebruin, Magnus, Mr.Ibrahem, gerritbot, EBernhardson, Esc3300, jcrespo, WMDE-leszek, Jdforrester-WMF, Krinkle, aaron, fgiunchedi, Aklapper, Ladsgroup, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, Vali.matei, Avner, Lewizho99, Zppix, Maathavan, Gehel, FloNight, Eevans, Hardikj, Wikidata-bugs, aude, jayvdb, faidon, Mbch331, Jay8g, jeremyb___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T163922: Create a URL rewrite to handle the /data/ path for canonical URLs for machine readable page content
elukey added a comment. In T163922#3639504, @Lucas_Werkmeister_WMDE wrote: Is there any way to find out which URLs are garbled? Can we look for RW_PROTO in all the cached redirects, or something like that? There is a way (https://wikitech.wikimedia.org/wiki/Varnish#One-off_purges_.28bans.29) but it is going to be risky if we don't get the purge pattern right, since we might risk to cut too many objects from the cache. I would avoid it if possible (and wait the normal caching expiry workflow), but we can discuss with the Traffic team another approach if you want.TASK DETAILhttps://phabricator.wikimedia.org/T163922EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: ema, elukey, TerraCodes, Jay8g, Liuxinyu970226, Lucas_Werkmeister_WMDE, Dereckson, akosiaris, fgiunchedi, Stashbot, gerritbot, Ladsgroup, Aklapper, daniel, Lordiis, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, Dcljr, Southparkfan, Jdforrester-WMF, Matanya, Rxy, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T163922: Create a URL rewrite to handle the /data/ path for canonical URLs for machine readable page content
elukey added a subscriber: ema.elukey added a comment. I believe this is only a matter of cleaning up urls that show up garbled, @ema just did it for https://commons.wikimedia.org/data/main/Data:Bundestagswahl2017/wahlkreis46.map via https://wikitech.wikimedia.org/wiki/Multicast_HTCP_purging#One-off_purge)TASK DETAILhttps://phabricator.wikimedia.org/T163922EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: ema, elukey, TerraCodes, Jay8g, Liuxinyu970226, Lucas_Werkmeister_WMDE, Dereckson, akosiaris, fgiunchedi, Stashbot, gerritbot, Ladsgroup, Aklapper, daniel, Lordiis, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, Dcljr, Southparkfan, Jdforrester-WMF, Matanya, Rxy, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Lowered Priority] T163922: Create a URL rewrite to handle the /data/ path for canonical URLs for machine readable page content
elukey lowered the priority of this task from "Unbreak Now!" to "Normal". TASK DETAILhttps://phabricator.wikimedia.org/T163922EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, TerraCodes, Jay8g, Liuxinyu970226, Lucas_Werkmeister_WMDE, Dereckson, akosiaris, fgiunchedi, Stashbot, gerritbot, Ladsgroup, Aklapper, daniel, Lordiis, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, Dcljr, Southparkfan, Jdforrester-WMF, Matanya, Rxy, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T163922: Create a URL rewrite to handle the /data/ path for canonical URLs for machine readable page content
elukey added a comment. All the appservers are now returning the good version of the redirect, I think that some of them are still showing up broken due to caching.TASK DETAILhttps://phabricator.wikimedia.org/T163922EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, TerraCodes, Jay8g, Liuxinyu970226, Lucas_Werkmeister_WMDE, Dereckson, akosiaris, fgiunchedi, Stashbot, gerritbot, Ladsgroup, Aklapper, daniel, Lordiis, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, Dcljr, Southparkfan, Jdforrester-WMF, Matanya, Rxy, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T119536: wikidata.org/entity/Q12345 should do content negotiation immediately, instead of redirecting to wikidata.org/wiki/Special:EntityData/Q36661 first
elukey added a project: User-Elukey. TASK DETAILhttps://phabricator.wikimedia.org/T119536EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, PokestarFan, Ladsgroup, gerritbot, Esc3300, thiemowmde, hoo, elf-pavlik, JanZerebecki, Aklapper, StudiesWorld, daniel, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, Izno, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Southparkfan, fbstj, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T119536: wikidata.org/entity/Q12345 should do content negotiation immediately, instead of redirecting to wikidata.org/wiki/Special:EntityData/Q36661 first
elukey added a comment. Hey @daniel, I have a couple of things to ask: I checked on deployment-mediawiki04/5 and I saw that an old version of the code was cherry picked, namely the solution with two rewrites (before the redirect=force). I removed it from puppetmaster02, I guess that we'd need to cherry pick https://gerrit.wikimedia.org/r/#/c/357985 and test again? It is a bit difficult to get the list of requirements from this task, would you mind to give us a list of redirects/rewrites/etc.. that this new change is supposed to do? One good thing that we could do is to test all the requests in 2) in labs with LogLevel for mod_rewrite to trace8, so apache will tell us exactly all the steps that the rewrite process do. After that we should be reasonably confident to merge :)TASK DETAILhttps://phabricator.wikimedia.org/T119536EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, PokestarFan, Ladsgroup, gerritbot, Esc3300, thiemowmde, hoo, elf-pavlik, JanZerebecki, Aklapper, StudiesWorld, daniel, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, Lewizho99, Maathavan, DatGuy, Devwaker, Urbanecm, JEumerus, Tulsi_Bhagat, Izno, suriyaa, Luke081515, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Southparkfan, fbstj, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Glaisher, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T162245: Enable GC for HHVM CLI (at least for dump runners)
elukey removed a project: User-Elukey.Herald added a subscriber: PokestarFan. TASK DETAILhttps://phabricator.wikimedia.org/T162245EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: PokestarFan, MoritzMuehlenhoff, elukey, gerritbot, daniel, ArielGlenn, Joe, Aklapper, hoo, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, Litanwei, Lewizho99, Zppix, Maathavan, Izno, Wikidata-bugs, aude, Svick, faidon, Mbch331, Jay8g, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T162245: Enable GC for HHVM CLI (at least for dump runners)
elukey added a comment. Tested with @hoo the settings outlined in https://docs.hhvm.com/hhvm/configuration/INI-settings on mwdebug1002. A script with returns "true" but the GC seems not to be effective. I asked the question on the #hhvm channel and it seems that the options were available in 3.12 but they haven't work reliably up to 3.18. Moritz has almost completed the work of building 3.18 so we could wait the new version before enabling GC.TASK DETAILhttps://phabricator.wikimedia.org/T162245EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: elukey, gerritbot, daniel, ArielGlenn, Joe, Aklapper, hoo, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, Salgo60, Lewizho99, Zppix, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Svick, faidon, Mbch331, Jay8g, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T162245: Enable GC for HHVM CLI (at least for dump runners)
elukey added a project: User-Elukey. TASK DETAILhttps://phabricator.wikimedia.org/T162245EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: daniel, ArielGlenn, Joe, Aklapper, hoo, Th3d3v1ls, Hfbn0, QZanden, Salgo60, Zppix, D3r1ck01, Izno, Wikidata-bugs, aude, Svick, faidon, Mbch331, Jay8g, fgiunchedi, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Triaged] T142944: Performance and caching considerations for article placeholders accesses
elukey triaged this task as "Normal" priority. TASK DETAILhttps://phabricator.wikimedia.org/T142944EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: DaBPunkt, BBlack, daniel, Lydia_Pintscher, Joe, Lucie, Aklapper, hoo, Zppix, D3r1ck01, Izno, Wikidata-bugs, aude, jayvdb, Ricordisamoa, faidon, Mbch331, Jay8g, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Triaged] T146468: wdqs - move metric collections to diamond
elukey triaged this task as "Low" priority. TASK DETAILhttps://phabricator.wikimedia.org/T146468EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: elukeyCc: Addshore, Gehel, Aklapper, mschwarzer, Avner, Zppix, debt, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, Deskana, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
