[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Smalyshev added a comment. @Pchelolo No I already updated the affected items manually.TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mobrovac, SmalyshevCc: Stashbot, gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Pchelolo added a comment. Do you need the events for the last month to be replayed?TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mobrovac, PcheloloCc: Stashbot, gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Smalyshev added a comment. Yep, seeing the events in grafana now, so I think it's all good now. Thanks!TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mobrovac, SmalyshevCc: Stashbot, gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2018-11-27T12:39:46Z] Synchronized php-1.33.0-wmf.4/includes/page/WikiPage.php: Convert $archivedRevisionCount to integer - T210013 T210451 (duration: 00m 47s)TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, Akovalyov, jijiki, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
gerritbot added a comment. Change 475893 merged by Mobrovac: [mediawiki/core@wmf/1.33.0-wmf.4] Convert $archivedRevisionCount to integer. https://gerrit.wikimedia.org/r/475893TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, CucyNoiD, Nandana, NebulousIris, thifranc, Akovalyov, jijiki, AndyTan, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30, Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
gerritbot added a comment. Change 475893 had a related patch set uploaded (by Mobrovac; owner: Ppchelko): [mediawiki/core@wmf/1.33.0-wmf.4] Convert $archivedRevisionCount to integer. https://gerrit.wikimedia.org/r/475893TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Smalyshev added a comment. I think I've extracted all I need from the DB tables for now, but I'll double-check and if anything is still missing I check the extracted data, thanks!TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Ottomata added a comment. Oo, I just did the same, or, at least I copied the relevant files. They are on stat1004:/home/otto/eventbus-validation-logs0. Stas said he might have another way so I stopped there.TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: OttomataCc: MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Pchelolo added a comment. Not to be worried. We have all the failed events stored since 2018-04-18. If needed, I will fetch all the missing page deletes tomorrow.TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Smalyshev added a comment. To make the problem worse, looks like breakage started more than 30 days ago - which means we don't have a record of the old deleted entries in recentchanges, so we don't know which items to update :(TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: MisterSynergy, mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Smalyshev added a comment. It does cause pretty severe breakage - all delete updates are missing from WDQS and people are complaining (in fact, have been complaining for a while now, I just now figured out that it's because we don't have any delete events at all!)TASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: mobrovac, TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, Eevans, JAllemandou, jkroll, Hardikj, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T210451: Kafka eqiad.mediawiki.page-delete topic is empty
Pchelolo added a comment. Yes, the fix has not been deployed yet. Asked to put it on Euro Mid-day SWATTASK DETAILhttps://phabricator.wikimedia.org/T210451EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: PcheloloCc: TerraCodes, Liuxinyu970226, Aklapper, Pchelolo, Ottomata, Gehel, Smalyshev, Legado_Shulgin, Nandana, thifranc, Akovalyov, jijiki, AndyTan, Davinaclare77, Qtn1293, Imarlier, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, D3r1ck01, Jonas, Xmlizer, Wong128hk, JAllemandou, mobrovac, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, ArielGlenn, faidon, He7d3r, Jdforrester-WMF, Mbch331, Jay8g, akosiaris, fgiunchedi, jeremyb, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
