Marostegui added a comment.
In T246415#6477957 <https://phabricator.wikimedia.org/T246415#6477957>, @Ladsgroup wrote: > In T246415#6477773 <https://phabricator.wikimedia.org/T246415#6477773>, @Marostegui wrote: > >> Thank you for all this work Amir, this is really interesting. I am curious about a few things, and if you'd have the time, can you explain further? >> >>> Splitting based on rc table doesn't make sense, let's just drop that. >> >> Why doesn't make sense? > > I mean mostly in the context of having a dedicated node for it (as it's only 1% of the read load, it would make sense if we had 100 replicas) but maybe redirecting most queries to one node (that would also handle lots of other things) would make sense too. > >> Gotcha. We also have to keep in mind that 1 host, in reality means 2 (for HA) and in both DCs. >>> 56% of the total load is term store (and it's heavily cached, this is actually 1% of the actual amount of read that reaches to the database, this is scary). >>> I'm working on to find out what's the load for term store coming from, it's very likely client but let's wait until we can get the actual data >> >> I guess those two are related, but, what do you mean with `the client`? > > Client is any wiki that's not repo (wikidata/commons), like English Wikipedia. Actually during the migration of term store, we found out most of the reads on s8 comes from clients and not wikidata itself (as they read wikidata's database directly). > >> Ah, understood! Thank you :) >>> 6% of all of the write pressure is just this: >>> 95904.00000000001 (6.519%) UPDATE `user` SET `user_editcount` = `user_editcount` + ? WHERE `user_id` = ? AND ( `user_editcount` IS NOT NULL ) >>> In other words, updating user edit count takes 6% of all of writes on master. This is twice as the write pressure of term store!!! I will file a bug for that. >> >> This is quite crazy indeed. Can't this be done thru a queue or something? > > Yup, We have a similar thing for updating site_stats table. I assume we can reuse that. That'd be nice, +1 Thank you again TASK DETAIL https://phabricator.wikimedia.org/T246415 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Marostegui Cc: Marostegui, Ladsgroup, WMDE-leszek, Aklapper, Addshore, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Iflorez, darthmon_wmde, Kent7301, alaa_wmde, joker88john, CucyNoiD, Nandana, jijiki, Klaas_Z4us_V, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Pablo-WMDE, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, elukey, _jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Jay8g
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
