[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
gerritbot added a comment. Change 425294 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Add maintenance script to clear term_search_key+term_weight https://gerrit.wikimedia.org/r/425294TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: hoo, gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
Lucas_Werkmeister_WMDE added a comment. I’ve made it an option so I wouldn’t have to adjust the existing tests (and also because I prefer --clear-term-weight=true to be the default behavior, even though we’ll use --clear-term-weight=false).TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: hoo, gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
hoo added a comment. In T191631#4126290, @Lucas_Werkmeister_WMDE wrote: Also I wonder, why we even bothering clearing out term_weight… having 0.0 probably has very little/ no benefit compared to just having something in there. Hm, good point… but perhaps it could be confusing for replica db users if some rows still have the term_weight even if they’re not supposed to rely on it? If you want, I can remove it from the script and clear only the term_search_key. IMO the tables are primarily an internal interface, so there's no need for such niceness… not updating the field will also mean less UPDATE load when doing this.TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, hooCc: hoo, gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
Lucas_Werkmeister_WMDE added a comment. Stopping can usually just be done with timeout… but it then needs to be able to pick up the work again. The script prints the current term_row_id after each batch, so you should be able to resume (--from-id) from that. Also I wonder, why we even bothering clearing out term_weight… having 0.0 probably has very little/ no benefit compared to just having something in there. Hm, good point… but perhaps it could be confusing for replica db users if some rows still have the term_weight even if they’re not supposed to rely on it? If you want, I can remove it from the script and clear only the term_search_key.TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: hoo, gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
hoo added a comment. In T191631#4120568, @Lucas_Werkmeister_WMDE wrote: Does the script need an option to stop after a certain amount of work has been done? So far I’m following rebuildTermSqlIndex and adding --from-id and --batch-size options, but no --limit options or anything like that. We probably don't want one script to just run forever, so it will need some way to stop and continue. Stopping can usually just be done with timeout… but it then needs to be able to pick up the work again. Also I wonder, why we even bothering clearing out term_weight… having 0.0 probably has very little/ no benefit compared to just having something in there.TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, hooCc: hoo, gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Lewizho99, Maathavan, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
gerritbot added a comment. Change 425294 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] Add maintenance script to clear term_search_key+term_weight https://gerrit.wikimedia.org/r/425294TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: gerritbot, Lucas_Werkmeister_WMDE, Aklapper, Lahi, Gq86, GoranSMilovanovic, lisong, QZanden, LawExplorer, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T191631: Add maintenance script to wipe term_search_key and term_weight columns
Lucas_Werkmeister_WMDE added a comment. Does the script need an option to stop after a certain amount of work has been done? So far I’m following rebuildTermSqlIndex and adding --from-id and --batch-size options, but no --limit options or anything like that.TASK DETAILhttps://phabricator.wikimedia.org/T191631EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Lucas_Werkmeister_WMDE, Aklapper, Lahi, Gq86, GoranSMilovanovic, lisong, QZanden, LawExplorer, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
