[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Addshore closed this task as "Resolved". Addshore claimed this task. Addshore added a comment. Restricted Application added a project: User-Addshore. Marking as resolved, as the investigation is done TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Addshore removed a project: Wikidata-Campsite. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Addshore Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper, Invadibot,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

WMDE-leszek edited projects, added Wikidata-Campsite; removed Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: WMDE-leszek Cc: Ladsgroup,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup added a comment. Three things are mistakes: `PrefetchingPropertyTermLookup_prefetchTerms`, `PrefetchingItemTermLookup_prefetchTerms`, `CachedDatabasePropertyLabelResolver_loadProperties`. The other nine are correct. We can either remove calls to the trait (but it might break some

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup added a comment. I'm pretty sure it's not two thirds, the only mistake I found is `CachedDatabasePropertyLabelResolver_loadProperties`, the rest are related to a query but sometimes it's slightly hidden. Let me clean it up. TASK DETAIL https://phabricator.wikimedia.org/T262923

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Lucas_Werkmeister_WMDE added a comment. But I don’t think this is a problem with just `PrefetchingItemTermLookup`… from a quick `git grep`, roughly two thirds of `$this->incrementForQuery()` calls aren’t directly related to a database query, I’d say. If I’m not mistaken, this new

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Lucas_Werkmeister_WMDE added a comment. Though it doesn’t seem to make a big difference, unless I did it wrong (feel free to review in Grafana): F32367793: Screenshot_2020-09-29 Wikibase SQL Term Storage - Grafana.png TASK DETAIL

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

toan added a comment. Suggested way forward: stop incrementing the statsd counter for PrefetchingItemTermLookup::prefetchTerms to get the correct result. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Lucas_Werkmeister_WMDE added a comment. > If it's causing two queries and not one, it should be counted twice since we care about the load. Am I missing something obvious? It’s not necessarily causing two queries, if I understand correctly. `PrefetchingItemTermLookup::prefetchTerms`

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup added a comment. In T262923#6490857 , @toan wrote: > Playing around with a new panel on grafana and looking at how these statsd things are called from the code. > > the statsd measurement of

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

toan added a comment. Playing around with a new panel on grafana and looking at how these statsd things are called from the code. the statsd measurement of PrefetchingItemTermLookup_prefetchTerms is always followed by DatabaseTermIdsResolver_selectTermsViaJoin which both seem to be

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup removed Ladsgroup as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup added a comment. Now we have a dashboard for ratio queries coming from client wikis: https://grafana.wikimedia.org/d/00548/wikibase-sql-term-storage?viewPanel=22=1=30s=now-3h=now Currently, it varies between 60% to 70% TASK DETAIL

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Maintenance_bot Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.9; 2020-09-15); removed MW-1.36-notes (1.36.0-wmf.10; 2020-09-22). TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-09-21T15:51:36Z] Synchronized php-1.36.0-wmf.9/extensions/Wikibase/lib/includes/Store/Sql/Terms/: [[gerrit:628808|Introduce and use StatsdMonitoring trait in term store (T262923

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-09-21T15:50:12Z] Synchronized php-1.36.0-wmf.9/extensions/Wikibase/lib/includes/Store/Sql/Terms/Util/StatsdMonitoring.php: [[gerrit:628808|Introduce and use StatsdMonitoring trait in term store (T262923

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a comment. Change 628808 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.9] Introduce and use StatsdMonitoring trait in term store https://gerrit.wikimedia.org/r/628808 TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, gerritbot Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper, Alter-paule,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a comment. Change 628808 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.9] Introduce and use StatsdMonitoring trait in term store https://gerrit.wikimedia.org/r/628808 TASK DETAIL

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, Maintenance_bot Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.10; 2020-09-22). TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, ReleaseTaggerBot Cc: Ladsgroup, Rosalie_WMDE, toan,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a comment. Change 628316 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Introduce and use StatsdMonitoring trait in term store https://gerrit.wikimedia.org/r/628316 TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup claimed this task. Ladsgroup moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Restricted Application added a project: User-Ladsgroup. TASK DETAIL https://phabricator.wikimedia.org/T262923 WORKBOARD

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a comment. Change 628316 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@master] Introduce and use StatsdMonitoring trait in term store https://gerrit.wikimedia.org/r/628316 TASK DETAIL

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: Ladsgroup, Rosalie_WMDE, toan, Lucas_Werkmeister_WMDE, Aklapper, Alter-paule, Beast1978,

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Ladsgroup added subscribers: toan, Rosalie_WMDE, Ladsgroup. Ladsgroup added a comment. we are doing it with @toan @Rosalie_WMDE TASK DETAIL https://phabricator.wikimedia.org/T262923 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc:

[Wikidata-bugs] [Maniphest] T262923: Investigate sensible DB load group splits for Wikidata / Wikibase

Lucas_Werkmeister_WMDE created this task. Lucas_Werkmeister_WMDE added projects: Wikidata, wdwb-tech-focus, Wikidata-Trailblazing-Exploration, Sustainability (Incident Followup), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). Restricted Application added a subscriber: Aklapper. TASK