Ladsgroup added a comment.
Hope these answers help. In T246415#6494603 <https://phabricator.wikimedia.org/T246415#6494603>, @Michael wrote: >> lots of them actually coming from mediawiki core and we have no control over destination of queries originating from there > > But that means that this is not coming from within Wikibase code, right? Is it referring to requests that originate from the Mediawiki core instance that runs Wikidata.org, like displaying the main page or running a normal wiki search? Take for example, the code that loads an entity to look up some data, it has to hit revision lookup, and revision lookup in core depending on cache hit or miss, might query the database (or not). Also normal core operations like checking recentchanges are also also included in core queries >> we found out most of the reads on s8 comes from clients and not wikidata itself (as they read wikidata's database directly). > > I thought s8 //is// the replica of the "wikidata's database"? It is the replica of wikidata's database but it mostly read by requests going to client wikis, like if you need to reparse the article of Alan Turing in Basque Wikipedia (because you made an edit in it or parser cache expired), it actually hits s8 quite heavily. >> e.g. client/repo is not easy to implement given that most of them hit lib/ instead > > That doesn't look like a feature to me. Shouldn't code in lib/ be handed a db connection to use instead creating its own? Though that might be a refactoring task that exceeds the scope of this ticket. But maybe not? 🤔 That's a good approach but it would require (I assume) rather large refactorings. > **about term store:** > If 56% of the load is term store in total and that load is split roughly 60/40 between clients and repo, then that means that ca. 33% of load comes from client term store requests and 23% from repo term store requests. How do term store requests from commons fit into this? Commons is considered as part of repo, even client requests in wikidata are also considered as part of repo since there is no easy for lib/ code to determine if the request is coming from client code or repo code. So I assume the actual client read is actually much higher (but OTOH, it doesn't matter that much since issues in wikidata client reads affect the repo behavior too as they are the same wiki) > Can client wikis on one hand and repo/wikidata.org on the other hand function without the term store data, maybe by gracefully falling back to QIDs everywhere? If yes, then groups splitting termstore/not termstore seem like they would make sense. But if everything goes down if the term store is down, then a split clients/repo would seem more useful. Does that make sense? In some cases yes but in some cases, they fallback to read from ES (= loading the whole entity) which in turn would bring down everything but fixing this IMO should not be too big. > **about other types of requests:** > Also, there seem to be some requests that maybe don't make up much load but still (sadly) lead to fatal errors if they don't succeed, like sitelinks. (Judging from everything going down when that table was deleted.) > We should probably make sure they are grouped into the "critical cluster" as well? hmm, good point, I think its load was around 6% which wasn't big but not small either. The table you're mentioning is `wb_items_per_site` and is really similar to term store but it's just not big enough to have its own section, I assume we can group it with term store and call it "wikidata secondary data store" or something like that. TASK DETAIL https://phabricator.wikimedia.org/T246415 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc: Michael, Marostegui, Ladsgroup, WMDE-leszek, Aklapper, Addshore, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Iflorez, darthmon_wmde, Kent7301, alaa_wmde, joker88john, CucyNoiD, Nandana, jijiki, Klaas_Z4us_V, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Pablo-WMDE, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, elukey, _jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
