[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

WMDE-leszek closed this task as "Resolved". WMDE-leszek moved this task from Test (Verification) to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. WMDE-leszek added a comment. I note the conclusion of the investigation has been the cache is not needed. A follow up task

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Michael moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Michael removed a project: Patch-For-Review. Michael added a comment. As far as I can tell, all associated patches have been either merged or abandoned. TASK

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 605333 **abandoned** by Addshore: [mediawiki/extensions/Wikibase@master] WIP DNM, update EntityRevisionLookup Reason: The cache was removed in https://phabricator.wikimedia.org/T255305, this draft might still be useful for actually using

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619547 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Set caching of CachingEntityRevisionLookup to CACHE_NONE in client https://gerrit.wikimedia.org/r/619547 TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.5; 2020-08-18); removed MW-1.36-notes (1.36.0-wmf.4; 2020-08-11). TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup,

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619597 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@master] Set caching of CachingEntityRevisionLookup to CACHE_NONE in repo https://gerrit.wikimedia.org/r/619597 TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. Now it's deployed across the fleet. Here's my analysis: - I was worried that constant decompressing of values coming from memcached would elevate the CPU usage but it didn't - I was worried that since wikibase has the caches keyed by entity id instead of

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619617 **abandoned** by Ladsgroup: [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.3] Set caching of CachingEntityRevisionLookup to CACHE_NONE in client Reason: wmf.4 is being deployed everywhere instead https://gerrit.wikimedia.org/r/619617 TASK

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-08-12T18:50:11Z] Synchronized php-1.36.0-wmf.4/extensions/Wikibase/client/includes/Store/Sql/DirectSqlStore.php: [[phab:T255305 |Set caching of CachingEntityRevisionLookup

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619616 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.4] Set caching of CachingEntityRevisionLookup to CACHE_NONE in client https://gerrit.wikimedia.org/r/619616 TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. One thing that was mentioned is that we should use getKnownCurrentRevision in revision lookup that has internal caching. TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619617 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.3] Set caching of CachingEntityRevisionLookup to CACHE_NONE in client https://gerrit.wikimedia.org/r/619617 TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619616 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.4] Set caching of CachingEntityRevisionLookup to CACHE_NONE in client https://gerrit.wikimedia.org/r/619616 TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. In repo: - before and after: https://performance.wikimedia.org/xhgui/run/view?id=5f342c3cbb8544a80bdb2b91 and https://performance.wikimedia.org/xhgui/run/view?id=5f342e54bb8544b80b969e7e (the after one is the second request to make the cache warm) TASK

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2020-08-12T18:04:18Z] Synchronized php-1.36.0-wmf.4/extensions/Wikibase/repo/includes/Store/Sql/SqlStore.php: [[phab:T255305 |Set caching of CachingEntityRevisionLookup to

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.4; 2020-08-11). TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, ReleaseTaggerBot Cc: Ladsgroup, Michael, Addshore, Aklapper,

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619615 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.4] Set caching of CachingEntityRevisionLookup to CACHE_NONE in repo https://gerrit.wikimedia.org/r/619615 TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619615 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.4] Set caching of CachingEntityRevisionLookup to CACHE_NONE in repo https://gerrit.wikimedia.org/r/619615 TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619597 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@master] Set caching of CachingEntityRevisionLookup to CACHE_NONE in repo https://gerrit.wikimedia.org/r/619597 TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Ladsgroup added a comment. So my idea for this is to get this cherry-picked (but not merged on master) and deploy it on canaries (10% of traffic) for an hour tomorrow and

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

gerritbot added a comment. Change 619547 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [mediawiki/extensions/Wikibase@master] Set caching of CachingEntityRevisionLookup to CACHE_NONE https://gerrit.wikimedia.org/r/619547 TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. The page id to revision id (Q228 -> 915) happens through a WikiPageEntityMetaDataAccessor which has an implementation that has basic caching: PrefetchingWikiPageEntityMetaDataAccessor so it's not as severe than I thought. TASK DETAIL

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. I confirm that setting the entity revision lookup cache to none just makes it to fallback to the mediawiki's SqlBlobStore cache (checking from debug logs) and SqlBlobStore cache indeed compresses it (619 is revid of the most recent revision of Q228):

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup claimed this task. Ladsgroup moved this task from To Do to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Restricted Application added a project: User-Ladsgroup. TASK DETAIL https://phabricator.wikimedia.org/T255305 WORKBOARD

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Krinkle added a project: Performance-Team (Radar). TASK DETAIL https://phabricator.wikimedia.org/T255305 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Krinkle Cc: Ladsgroup, Michael, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, Akuckartz,

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Ladsgroup added a comment. You can store caches as database entries (in objectcache table) by setting $wgMainCacheType to CACHE_DB. I did it and this is I've got so far: - We actually ourselves store them twice, once under `wikibase_shared/repo:WikiPageEntityRevisionLookup:Q228` and

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Michael added a comment. The investigation should shed light on the following questions: - How complicate would removing the Cache for CachingEntityRevisionLookup be? - Would this be the wok of a few people over a couple of days or should we have a hike/trailblaze for it? TASK

[Wikidata-bugs] [Maniphest] T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more

Addshore renamed this task from "[Investigation] - Consider if CachingEntityRevisionLookup is needed any more" to "[Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more". Addshore updated the task description. TASK DETAIL