[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-11-02 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Gehel,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-11-02 Thread Gehel
Gehel removed projects: Wikibase wb_terms leftovers 2020, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). Gehel added a comment. Saneitizer is running slower than usual at the moment, this needs to be investigated first. See T266762 and

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-11-02 Thread dcausse
dcausse added a comment. Moving to blocked until we know what's causing T266762 TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc:

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-28 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a comment. To summarize: the sanitizer was turned off on Wikidata because it caused excessive database load; during the wb_terms cleanup hike, we optimized it to not do as much unneeded work (mainly rEWBA01de21628b23: Override

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-26 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE changed the task status from "Stalled" to "Open". Lucas_Werkmeister_WMDE moved this task from Stalled/Waiting to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Lucas_Werkmeister_WMDE added a comment. Alright,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-26 Thread EBernhardson
EBernhardson added a comment. Saneitizer was turned back on last week, everything there is working well and wikidata can be reenabled any time. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-16 Thread WMDE-leszek
WMDE-leszek added a project: Wikibase wb_terms leftovers 2020. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: WMDE-leszek Cc: Lucas_Werkmeister_WMDE, tfmorris, Pintoch, CBogen, EBernhardson,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-15 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE set Due Date to Tue, Oct 27, 12:00 AM. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Lucas_Werkmeister_WMDE, tfmorris, Pintoch, CBogen,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-15 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE Cc: Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-15 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE changed the task status from "Open" to "Stalled". Lucas_Werkmeister_WMDE added a comment. Alright, then we can try turning on the sanitizer on Wikidata either next week (after wmf.14 is rolled out) or the week after that (probably safer). TASK DETAIL

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-15 Thread gerritbot
gerritbot added a comment. Change 633170 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Override EntityHandler::getParserOutputForIndexing https://gerrit.wikimedia.org/r/633170 TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-13 Thread EBernhardson
EBernhardson added a comment. All seems reasonable to me. Note that the saneitizer is globally turned off as of a week ago due to a separate incident. Expecting to turn that back on this week. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-13 Thread gerritbot
gerritbot added a comment. Change 632752 **merged** by jenkins-bot: [mediawiki/core@master] Clarify HTML generation for indexing in ContentHandler https://gerrit.wikimedia.org/r/632752 TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-12 Thread Addshore
Addshore added a comment. In T239931#6536170 , @Lucas_Werkmeister_WMDE wrote: > If we no longer use the HTML in the sanitizer (see the above changes), do we still need a gradual ramp-up when reenabling the sanitizer or is it okay to

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-12 Thread Ladsgroup
Ladsgroup added a comment. Agreed. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc: Lucas_Werkmeister_WMDE, tfmorris, Pintoch, CBogen, EBernhardson, Ladsgroup, Addshore,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-12 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a comment. In T239931#6331040 , @Ladsgroup wrote: > In T239931#6330953 , @CBogen wrote: > >> @addshore do you know if the load has been reduced enough

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-12 Thread gerritbot
gerritbot added a comment. Change 632751 **abandoned** by Lucas Werkmeister (WMDE): [mediawiki/extensions/Jade@master] Override getParserOutputForIndexing to ensure HTML Reason: No longer needed with PS2 of I50f3a530f2, where generating HTML is no longer skipped by default.

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-09 Thread gerritbot
gerritbot added a comment. Change 633170 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Wikibase@master] Override EntityHandler::getParserOutputForIndexing https://gerrit.wikimedia.org/r/633170 TASK DETAIL

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-07 Thread Lucas_Werkmeister_WMDE
Lucas_Werkmeister_WMDE added a comment. The above two changes attempt to implement the optimization outlined in the IRC log in the task description: to stop generating HTML for search indexing by setting the `'generate-html'` hint. (Wikibase seems to respect this hint, as far as I could

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-07 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: tfmorris, Pintoch, CBogen, EBernhardson, Ladsgroup, Addshore, Aklapper, dcausse,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-07 Thread gerritbot
gerritbot added a comment. Change 632752 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/core@master] Don’t generate HTML in getParserOutputForIndexing https://gerrit.wikimedia.org/r/632752 TASK DETAIL

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-07 Thread gerritbot
gerritbot added a comment. Change 632751 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/Jade@master] Override getParserOutputForIndexing to ensure HTML https://gerrit.wikimedia.org/r/632751 TASK DETAIL

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-05 Thread Gehel
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel Cc: tfmorris, Pintoch, CBogen, EBernhardson,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-10-05 Thread CBogen
CBogen merged a task: T264566: Redirected entity still present in search results after 6 months. CBogen added subscribers: Pintoch, tfmorris. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-08-27 Thread CBogen
CBogen triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: CBogen Cc: CBogen, EBernhardson, Ladsgroup, Addshore, Aklapper, dcausse, Wilmanbeno, Akuckartz,

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-07-23 Thread Ladsgroup
Ladsgroup added a comment. In T239931#6330953 , @CBogen wrote: > @addshore do you know if the load has been reduced enough that this is no longer an issue? The load has been reduced (we dropped the old term store and the new one is

[Wikidata-bugs] [Maniphest] T239931: Reduce the impact of the sanitizer on wikidata

2020-07-23 Thread CBogen
CBogen added a comment. @addshore do you know if the load has been reduced enough that this is no longer an issue? TASK DETAIL https://phabricator.wikimedia.org/T239931 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: CBogen Cc: CBogen,