[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
Smalyshev added a comment. @ArielGlenn thanks for your help! I'll watch now how it works over next week and then try to switch dump loading to dailies.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
ArielGlenn added a comment. I've run the script manually with the above change applied; results are available in the expected location.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, ArielGlennCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 449994 merged by ArielGlenn: [operations/puppet@production] category diffs: full ts dir is different than dailies ts dir https://gerrit.wikimedia.org/r/449994TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 449994 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn): [operations/puppet@production] category diffs: full ts dir is different than dailies ts dir https://gerrit.wikimedia.org/r/449994TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
ArielGlenn added a comment. This is now deployed; I'll check tomorrow that the dailies ran ok, and we'll know about the fulls over the weekend.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, ArielGlennCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 378355 merged by ArielGlenn: [operations/puppet@production] Generate daily diffs for categories RDF https://gerrit.wikimedia.org/r/378355TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
ArielGlenn added a comment. The dblist fix has been deployed, off to test the actual bash script now. Until now it's all been manual runs across the dblist with direct calls to the maintenance script.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, ArielGlennCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2018-07-19T11:46:15Z] Synchronized dblists/categories-rdf.dblist: SWAT: [[gerrit:445720|Remove labs wikis from the categories-rdf list, dont need them (T198356)]] (duration: 00m 55s)TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, StashbotCc: Stashbot, Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 445720 merged by jenkins-bot: [operations/mediawiki-config@master] Remove labs wikis from the categories-rdf list, don't need them https://gerrit.wikimedia.org/r/445720TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 445720 had a related patch set uploaded (by Zfilipin; owner: Smalyshev): [operations/mediawiki-config@master] Remove labs wikis from the categories-rdf list, don't need them https://gerrit.wikimedia.org/r/445720TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 443449 merged by jenkins-bot: [mediawiki/core@master] Use array_merge to merge indexed arrays in categoryChangesAsRdf.php. https://gerrit.wikimedia.org/r/443449TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 443449 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler): [mediawiki/core@master] Use array_merge to merge indexed arrays in categoryChangesAsRdf.php. https://gerrit.wikimedia.org/r/443449TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
daniel added a comment. @ArielGlen found the bug: In https://phabricator.wikimedia.org/source/mediawiki/browse/master/maintenance/categoryChangesAsRdf.php$223 we have: $tables += $extra_tables; But += does not work as expected on indexed arrays in PHP. using array_merge instead fixes the issue. This never worked. A local test run immediately failed with an exception.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, danielCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
ArielGlenn added a comment. On testwiki and test2wiki I get the following, running for the last seven day interval (all other wikis run ok): Wikimedia\Rdbms\DBQueryError from line 1443 of /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT rc_timestamp,page_title,page_namespace,rc_title,rc_cur_id,pp_propname,cat_pages,cat_subcats,cat_files FROM `recentchanges` FORCE INDEX (new_name_timestamp) LEFT JOIN `page_props` ON (pp_propname = 'hiddencat' AND (pp_page = rc_cur_id)) LEFT JOIN `category` ON ((cat_title = rc_title)) WHERE (rc_timestamp >= '20180625095244') AND (rc_timestamp < '20180702095244') AND rc_namespace = '14' AND rc_new = '0' AND rc_log_type = 'move' AND rc_type = '3' ORDER BY rc_timestamp ASC LIMIT 200 Function: BatchRowIterator::next Error: 1054 Unknown column 'page_title' in 'field list' Can we take care of that before this goes live? Even just removing those two wikis from the categoriedrdf db list would be ok.TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, ArielGlennCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Charlie_WMDE, Smalyshev, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, Avner, Lewizho99, Maathavan, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T198356: Generate daily diffs for recently changed categories
gerritbot added a comment. Change 378355 had a related patch set uploaded (by Smalyshev; owner: Smalyshev): [operations/puppet@production] Generate daily diffs for categories RDF https://gerrit.wikimedia.org/r/378355TASK DETAILhttps://phabricator.wikimedia.org/T198356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: Lydia_Pintscher, daniel, Aklapper, gerritbot, ArielGlenn, EBernhardson, Lea_WMDE, Charlie_WMDE, Smalyshev, Lahi, Gq86, Darkminds3113, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Avner, Gehel, Jonas, FloNight, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
