Marostegui added a comment.
In T219145#5088371 <https://phabricator.wikimedia.org/T219145#5088371>, @alaa_wmde wrote: > In T219145#5087634 <https://phabricator.wikimedia.org/T219145#5087634>, @Marostegui wrote: > > > In T219145#5084305 <https://phabricator.wikimedia.org/T219145#5084305>, @alaa_wmde wrote: > > > > > @Marostegui thanks for your update! > > > > > > Postponing until the new master is up and running is very unlikely to be the adjustment we would go for. > > > > > > The more likely one would be to change to migration plan 2 (take from bonfire slides <https://docs.google.com/presentation/d/1FbaD4sv7vMZICD7NJfW3tp2CqtosigB0dNeWbtztpKo/edit#slide=id.g5069c67ff2_0_344>): > > > > > > I don't have access to that. Just requested it. > > > For some reason I can't see a request from you in my mailbox .. but here's the slide screenshot that describes the plan > F28584563: image.png <https://phabricator.wikimedia.org/F28584563> @Addshore gave me access a few hours ago :) > > >>> **on write**: write to new schema and delete from old one >>> **on read** read from both schemas >>> when old wb_terms is empty, read only from new schema and drop wb_terms table eventually >>> >>> This plan will actually not require extra space (not significant increase at least) and will result in reducing the total space usage over time. >> >> You've got any estimations on how much it would require? > > I think it won't require anything significant, I mean nothing other than usual dbms needs for schema/meta info on tables and indexes. > >>> It is hard to estimate the footprint of this plan on performance, which is likely to be a little higher than the original one. If I can think of a way to estimate I will share some numbers soon. >> >> This would be useful to know if you can come with some figures. >> I assume it is possible to stop the migration right away if we see performance issues and then resume once we are on a better shape (ie: we discover the performance degradation is an issue and we decide to fully wait for the new master), right? > > Good question.. I don't think stop here would be possible as in "we stop dealing with new schema immediately and just switch back to write and read from old wb_terms", because we would have already deleted some data from `wb_terms` and moved them over to new schema. So stopping here would mean to "migrate back" in which we: **on write** we write to old schema and delete from new schema, and **on read** we read from both until new schema is empty. That would not release us immediately from the degradation in performance as we would wish. > > ---- > > I think we'll need to wait until we have written some of the necessary code for migration and run some tests to get some numbers. Yes, we definitely need some re-thinking on how to stop the migration or rollback if something arises, which can be a possibility on such complex process. Specially performance-wise, things can go different than original planned. Also, we might have the new master in place if this will take some more time, which is also a good thing in general: more disk space, faster disks, more memory... :-) TASK DETAIL https://phabricator.wikimedia.org/T219145 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: alaa_wmde, Marostegui Cc: jcrespo, Marostegui, JeroenDeDauw, Addshore, Ladsgroup, Aklapper, alaa_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
