-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Mats!
Yes, great presentation! Question: on slide 17, you write that the transaction position in the file is also needed. I'm wondering why this is the case? I'm assuming by "file" you mean the master's serial transaction log? If so, why should the position of the transaction in that log be required by the replicas? Cheers! Jay Mats Kindahl wrote: > Hi Drizzlelites, > > Sorry about that, the presentation was larger than I thought. I have put it up > on http://www.kindahl.net/pub/DynamicReplicationTopologies.pdf instead. > > Best wishes, > Mats Kindahl > > Mats Kindahl wrote: >> Adi Alurkar wrote: >>> Greetings, >>> >>> It was great to meet everybody at Developer day. I have been catching up >>> with all the docs on the protocol and discussing them with our CTO and in >>> the last brainstorm with him we realized that although the proposed protocol >>> offers great enhancements in terms of exposing shard id / QUERY / QUERY_RO >>> in the protocol itself, the following idea would make it even better. >>> >>> To simplify and explain the problem consider the following architecture: >>> (also note that this would apply to sharded DBD scenario as well) >>> >>> Master Slave/Replica >>> M1 ---> R1 >>> ---> R2 >>> ---> R3 >>> ... >>> ---> R<n> >>> >>> * R1, R2 use a persistent storage engine (primarily) to serve as backups to >>> M1 >>> * R3...RN use memory based/memcache storage engines and used to handle most >>> of the "READ ONLY" traffic. >>> * Data is being updated at the M1. >>> * Data is being replicated to each of the replicas R1..RN. >>> * The replicas can be applying the changes at master at different rates and >>> are thus at different consistency state's >>> * The application assumes eventual consistency. This implies the following >>> all request for data made by a thread should only be serviced with data that >>> is at a consistency state higher than or equal to of the previous state. >>> Using timestamp as a crude gauge to measure consistency if thread TH1 >>> requests data at 10:00 all subsequent data request from TH1 should only see >>> data that is consistent as of 10:00 or later. >>> * In reality timestamp is not the correct way to track consistency, which is >>> where we rely on SCN within oracle to achieve consistent reads. The simplest >>> definition of SCN is a increasing counter which determines the consistency >>> state of the database. >>> * C<n> indicates the consistency state.(Needless to say this needs to be >>> serialized and determined at the end of a transaction) >>> >>> Consider the following use case in the above architecture: >>> >>> * all request where QUERY_RO can be proxied to all servers but all QUERY >>> i.e. writes/updates need to be sent to and can be served only by M1. >>> * new thread TH1 updates (either, INSERT/UPDATE/DATE/DDL) the data at the >>> master, it received a C1 as part of the confirmation of the update and is >>> set in the protocol. >>> * all subsequent QUERY_RO requests via TH1 include C1 as part of the request >>> (set in protocol) thus can be served only by servers that have consistency >>> state >=C1. Each satisfied request resets the consistency state in the >>> protocol. >>> * lets assume TH2 performs multiple updates resulting in the M1 to be at >>> C50. R1 is at C45, R2 is at C38, R3 is at R24, etc >>> * TH1 request data (QUERY_RO) and is at C24. The request can be routed to >>> any of the candidate servers which have consistency state >= C24. >>> * a new TH3 requests data (QUERY_RO) with empty consistent state (or 0) and >>> thus can be served by the first available server. >> This only works if there is a single source of data (i.e., as you note >> below, a >> single master). >> >>> As indicated above adding the concept of SCN and exposing it via the >>> protocol would make this a really awesome feature. Some other enhancements >>> that would make the SCN concept even more rock solid include: >>> * Oracle's SCN is tied to the Oracle instance and is not portable from one >>> machine to another, except incase of physical standby. This makes it very >>> difficult to implement partial replication(ability to replicate only certain >>> objects/schemas), support for multi-master multi-slave replicated >>> environment. Not to mention moving a slave from one server to another >>> impossible unless the new master is a physical standby of the original >>> server. >> Exactly. >> >>> * support manual "override" of the SCN for slaves, this would allow the >>> slaves to inherit the SCN from the master per transaction making >>> slave->master promotion easier. >> This just adds to the confusion, since "faking" consistency can be a good >> source >> of problems. The problem is that you assume a total order, but that is not >> necessary. >> >> Use a vector instead (called "version vector" by some), where you have a >> partial >> order, but not a total order on the elements. I have attached a short >> presentation I did for the replication team at MySQL a few years ago, which >> describes this approach. >> >>> * determine SCN from a central source which uses consensus/paxos, etc >> That just adds a single point of failure, which of course can be used, but >> does >> not scale very well, since you then need to replicate the SCN central source >> as >> well, and then you need to have an SCN for every SCN distributed (assuming >> you >> use the same scheme for checking consistency of the SCN server), which is >> kind >> of recursive. >> >>> Does the idea of supporting something like a SCN and exposing it in the >>> protocol to support use case such as above seem valid? >> Absolutely. >> >> However, it would be nice to be able to handle a branching temporal model as >> well, not just a linear temporal model. There were some mention that slaves >> in >> an "alternate future" frequently is a problem, so right now I am thinking of >> ways to handle that without introducing an unnecessary overhead. >> >> Just my few cents, >> Mats Kindahl >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~drizzle-discuss >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~drizzle-discuss >> More help : https://help.launchpad.net/ListHelp > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkn/Ea4ACgkQ2upbWsB4UtGt7gCfQj57BHni2fChIg7Hgw5Jt4Fg BZMAmwTszZIzPr+6mvMa1npJq7hOPZed =ugBx -----END PGP SIGNATURE----- _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
