[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup closed subtask T288639: SpamBlacklistHooks::onEditFilterMergedContent causes every edit to be rendered twice as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc:

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Krinkle added a parent task: T277788: Save Timing improvements (2021). TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Krinkle Cc: danshick-wmde, RShigapov, Kormat, Marostegui,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Addshore Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup, tstarling,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore added a comment. > but also since these are post-edit actions, it wouldn't have much impact and the edit saving time but it would help on import and batch edits. At least when the testing on local Wikibase was done this should have an impact on API response times to users, as

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

ReleaseTaggerBot edited projects, added MW-1.37-notes (1.37.0-wmf.19; 2021-08-16); removed MW-1.37-notes (1.37.0-wmf.21; 2021-08-30). TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup added a comment. It helped in the size of binlog on ParserCache servers (https://grafana.wikimedia.org/d/00106/parser-cache?viewPanel=6=1=1629900681358=1629987081359) F34620730: image.png The actual impact on the db is a bit

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Maintenance_bot Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-08-25T17:46:34Z] Synchronized php-1.37.0-wmf.19/extensions/Wikibase/repo/includes/Content/EntityHandler.php: Backport: [[gerrit:714674|Set EntityHandler::generateHTMLOnEdit to false (T285987

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714674 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.19] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/714674 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-08-25T17:14:25Z] Synchronized php-1.37.0-wmf.20/extensions/Wikibase/repo/includes/Content/EntityHandler.php: Backport: [[gerrit:714675|Set EntityHandler::generateHTMLOnEdit to false (T285987

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714675 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.20] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/714675 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 713649 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/713649 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

ReleaseTaggerBot edited projects, added MW-1.37-notes (1.37.0-wmf.21; 2021-08-30); removed MW-1.37-notes (1.37.0-wmf.20; 2021-08-23). TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714675 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.20] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/714675 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714674 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.19] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/714674 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-08-25T10:49:33Z] Synchronized php-1.37.0-wmf.19/includes/Storage/DerivedPageDataUpdater.php: Backport: [[gerrit:714672|Introduce concept of generateHTMLOnEdit() for ContentHandler (T285987

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-08-25T10:47:48Z] Synchronized php-1.37.0-wmf.19/includes/content/ContentHandler.php: Backport: [[gerrit:714672|Introduce concept of generateHTMLOnEdit() for ContentHandler (T285987

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714672 **merged** by jenkins-bot: [mediawiki/core@wmf/1.37.0-wmf.19] Introduce concept of generateHTMLOnEdit() for ContentHandler https://gerrit.wikimedia.org/r/714672 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 713268 **merged** by jenkins-bot: [mediawiki/core@master] Introduce concept of generateHTMLOnEdit() for ContentHandler https://gerrit.wikimedia.org/r/713268 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2021-08-25T10:03:19Z] Synchronized php-1.37.0-wmf.20/includes: Backport: [[gerrit:714671|Introduce concept of generateHTMLOnEdit() for ContentHandler (T285987 )]] (duration: 02m

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.20; 2021-08-23). TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, ReleaseTaggerBot Cc: danshick-wmde, RShigapov, Kormat,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714671 **merged** by jenkins-bot: [mediawiki/core@wmf/1.37.0-wmf.20] Introduce concept of generateHTMLOnEdit() for ContentHandler https://gerrit.wikimedia.org/r/714671 TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714672 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/core@wmf/1.37.0-wmf.19] Introduce concept of generateHTMLOnEdit() for ContentHandler https://gerrit.wikimedia.org/r/714672 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 714671 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/core@wmf/1.37.0-wmf.20] Introduce concept of generateHTMLOnEdit() for ContentHandler https://gerrit.wikimedia.org/r/714671 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 713649 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/extensions/Wikibase@master] Set EntityHandler::generateHTMLOnEdit to false https://gerrit.wikimedia.org/r/713649 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, gerritbot Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 713268 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani): [mediawiki/core@master] [WIP] Introduce concept of generateHTMLOnEdit() for Content https://gerrit.wikimedia.org/r/713268 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup added a subtask: T288639: SpamBlacklistHooks::onEditFilterMergedContent causes every edit to be rendered twice. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc:

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup added a comment. This is wy more complicated than it looks, I try to summarize my findings here: - First, if we fix this, and taking editing the sandbox as an example, the wall time will be reduced by 33% and CPU time by 40% (data

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup claimed this task. Ladsgroup moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Restricted Application added a project: User-Ladsgroup. TASK DETAIL https://phabricator.wikimedia.org/T285987 WORKBOARD

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore moved this task from Prioritized Tech (prioritised from top to bottom) to Wikidata-Campsite-Iteration-∞ on the Wikidata-Campsite board. Addshore edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞); removed Wikidata-Campsite. TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore set the point value for this task to "13". TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup, tstarling,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup, tstarling,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup, tstarling,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: danshick-wmde, RShigapov, Kormat, Marostegui, LSobanski, Ladsgroup,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 704856 **abandoned** by Ladsgroup: [mediawiki/extensions/Wikibase@master] [POC] Set $generateHtml to false in EntityContent Reason: POC is done :D https://gerrit.wikimedia.org/r/704856 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup added a comment. In T285987#7227115 , @Kormat wrote: > In T285987#7215913 , @Ladsgroup wrote: > >> My guess would be that this will remove around a couple tens of millions

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Kormat added a comment. In T285987#7215913 , @Ladsgroup wrote: > My guess would be that this will remove around a couple tens of millions of PC entries from there (we can do a check how many wikidata entries there, let me know if want

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

LSobanski updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: LSobanski Cc: Marostegui, LSobanski, Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

LSobanski updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: LSobanski Cc: Marostegui, LSobanski, Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Marostegui, LSobanski, Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Ladsgroup added subscribers: LSobanski, Marostegui. Ladsgroup added a comment. I just want to say that I think the effect on readers will be negligible, HTML of entity content comparing to wikitext is much faster to produce. Also there is a rather major benefit from this that is not

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel, Krinkle,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

gerritbot added a comment. Change 704856 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup): [mediawiki/extensions/Wikibase@master] [POC] Set $generateHtml to false in EntityContent https://gerrit.wikimedia.org/r/704856 TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel, Krinkle,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Ladsgroup, tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel, Krinkle, Addshore,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore added a project: Wikidata-Campsite. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: tstarling, Lucas_Werkmeister_WMDE, Michael, Tarrow, daniel, Krinkle, Addshore, Aklapper,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

tstarling added a comment. In T285987#7191372 , @Addshore wrote: > Adding #platform_engineering to get some input and thoughts on this topic from mediawiki folks that know

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Lucas_Werkmeister_WMDE added a comment. We need to watch out that link tables etc. are still populated even when we don’t generate HTML. (So if I remember the interfaces correctly, we still need to generate //`ParserOutput`// with all its metadata, just without HTML inside it.) TASK DETAIL

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Krinkle edited projects, added Performance-Team (Radar); removed Performance Issue. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Krinkle Cc: Michael, Tarrow, daniel, Krinkle, Addshore,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore added a project: Platform Engineering. Addshore added a comment. Adding #platform_engineering to get some input and thoughts on this topic from mediawiki folks that know more about something that might be missing, or

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore added a project: Performance Issue. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Tarrow, daniel, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, Akuckartz,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Tarrow, daniel, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Tarrow, daniel, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore added a project: MediaWiki-extensions-WikibaseRepository. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Tarrow, daniel, Krinkle, Addshore, Aklapper, Invadibot, maantietaja,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T285987 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Tarrow, daniel, Krinkle, Addshore, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana, Lahi,

[Wikidata-bugs] [Maniphest] T285987: Do not generate full html parser output at the end of Wikibase edit requests

Addshore created this task. Addshore added projects: Wikidata, wdwb-tech. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION **Introduction** During Wikibase local development we noticed something odd and slow 1. Create some new wikibase items 2. create a final