alaa_wmde moved this task from Peer Review to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. alaa_wmde added subscribers: Lucas_Werkmeister_WMDE, Ladsgroup, Addshore, hoo. alaa_wmde added a comment.
@Ladsgroup @Addshore @Lucas_Werkmeister_WMDE @hoo For the new store, we are saving terms in a deferred update. This makes uniqueness checks: a) if done during request, they might not catch possible collisions with values that are in deferred updates at the moment but haven't been written to db yet. b) if done as part of the deferred update, they don't have the chance to report back collisions to the user. Our options are: 1. live with (a) and believe that those race conditions are rare. Pretty bad as it introduces uniqueness-breaches defying the whole purpose of those checks. 2. live with (b) and believe that those race conditions are rare. This is pretty bad too, as it won't report collisions with already persisted terms. 3. do both (a) and (b) .. extra load, can solve both issues with (a) and (b) alone, except for when a collision is going to happen between current edit and some deferred update. In that case, we only avoid breaking uniqueness but the user won't know. 4. move saving terms to the request, instead of doing it in a deferred update. not sure how bad that is yet. can we tell already how longs those queries are taking already (from some db stats)? or will we need to setup a separate monitoring to know that? 5. During request A, we store hashed fingerprint in suitable cache, and then deferred update A would remove it when finished. That way, request B can check during request both db and cache (with its hashed fingerprint). Probably the best option of both worlds, but need your opinions. 6. Store hashed fingerprints in a separate table that is used only for uniqueness checks. extra storage, but is it going to be too much extra? if hash size is 512 bits then we would need ( 512 bits for hash + 64 bits for entity id) * 70m ~= 5GB which sounds a lot. 7. ... any other options you see? TASK DETAIL https://phabricator.wikimedia.org/T232040 WORKBOARD https://phabricator.wikimedia.org/project/board/3539/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: alaa_wmde Cc: hoo, Addshore, Ladsgroup, Lucas_Werkmeister_WMDE, Aklapper, alaa_wmde, Hook696, Daryl-TTMG, RomaAmorRoma, 0010318400, E.S.A-Sheild, Iflorez, darthmon_wmde, Meekrab2012, joker88john, DannyS712, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
