Jay Pipes wrote: > Hi Mats! > > Yes, great presentation! > > Question: on slide 17, you write that the transaction position in the > file is also needed. I'm wondering why this is the case? I'm assuming > by "file" you mean the master's serial transaction log? If so, why > should the position of the transaction in that log be required by the > replicas?
It just serves as a transaction id. It could just as well be a sequence of numbers assigned when the transactions are committed. /Matz > > Cheers! > > Jay > > Mats Kindahl wrote: >> Hi Drizzlelites, > >> Sorry about that, the presentation was larger than I thought. I have put it >> up >> on http://www.kindahl.net/pub/DynamicReplicationTopologies.pdf instead. > >> Best wishes, >> Mats Kindahl > >> Mats Kindahl wrote: >>> Adi Alurkar wrote: >>>> Greetings, >>>> >>>> It was great to meet everybody at Developer day. I have been catching up >>>> with all the docs on the protocol and discussing them with our CTO and in >>>> the last brainstorm with him we realized that although the proposed >>>> protocol >>>> offers great enhancements in terms of exposing shard id / QUERY / QUERY_RO >>>> in the protocol itself, the following idea would make it even better. >>>> >>>> To simplify and explain the problem consider the following architecture: >>>> (also note that this would apply to sharded DBD scenario as well) >>>> >>>> Master Slave/Replica >>>> M1 ---> R1 >>>> ---> R2 >>>> ---> R3 >>>> ... >>>> ---> R<n> >>>> >>>> * R1, R2 use a persistent storage engine (primarily) to serve as backups to >>>> M1 >>>> * R3...RN use memory based/memcache storage engines and used to handle most >>>> of the "READ ONLY" traffic. >>>> * Data is being updated at the M1. >>>> * Data is being replicated to each of the replicas R1..RN. >>>> * The replicas can be applying the changes at master at different rates and >>>> are thus at different consistency state's >>>> * The application assumes eventual consistency. This implies the following >>>> all request for data made by a thread should only be serviced with data >>>> that >>>> is at a consistency state higher than or equal to of the previous state. >>>> Using timestamp as a crude gauge to measure consistency if thread TH1 >>>> requests data at 10:00 all subsequent data request from TH1 should only see >>>> data that is consistent as of 10:00 or later. >>>> * In reality timestamp is not the correct way to track consistency, which >>>> is >>>> where we rely on SCN within oracle to achieve consistent reads. The >>>> simplest >>>> definition of SCN is a increasing counter which determines the consistency >>>> state of the database. >>>> * C<n> indicates the consistency state.(Needless to say this needs to be >>>> serialized and determined at the end of a transaction) >>>> >>>> Consider the following use case in the above architecture: >>>> >>>> * all request where QUERY_RO can be proxied to all servers but all QUERY >>>> i.e. writes/updates need to be sent to and can be served only by M1. >>>> * new thread TH1 updates (either, INSERT/UPDATE/DATE/DDL) the data at the >>>> master, it received a C1 as part of the confirmation of the update and is >>>> set in the protocol. >>>> * all subsequent QUERY_RO requests via TH1 include C1 as part of the >>>> request >>>> (set in protocol) thus can be served only by servers that have consistency >>>> state >=C1. Each satisfied request resets the consistency state in the >>>> protocol. >>>> * lets assume TH2 performs multiple updates resulting in the M1 to be at >>>> C50. R1 is at C45, R2 is at C38, R3 is at R24, etc >>>> * TH1 request data (QUERY_RO) and is at C24. The request can be routed to >>>> any of the candidate servers which have consistency state >= C24. >>>> * a new TH3 requests data (QUERY_RO) with empty consistent state (or 0) and >>>> thus can be served by the first available server. >>> This only works if there is a single source of data (i.e., as you note >>> below, a >>> single master). >>> >>>> As indicated above adding the concept of SCN and exposing it via the >>>> protocol would make this a really awesome feature. Some other enhancements >>>> that would make the SCN concept even more rock solid include: >>>> * Oracle's SCN is tied to the Oracle instance and is not portable from one >>>> machine to another, except incase of physical standby. This makes it very >>>> difficult to implement partial replication(ability to replicate only >>>> certain >>>> objects/schemas), support for multi-master multi-slave replicated >>>> environment. Not to mention moving a slave from one server to another >>>> impossible unless the new master is a physical standby of the original >>>> server. >>> Exactly. >>> >>>> * support manual "override" of the SCN for slaves, this would allow the >>>> slaves to inherit the SCN from the master per transaction making >>>> slave->master promotion easier. >>> This just adds to the confusion, since "faking" consistency can be a good >>> source >>> of problems. The problem is that you assume a total order, but that is not >>> necessary. >>> >>> Use a vector instead (called "version vector" by some), where you have a >>> partial >>> order, but not a total order on the elements. I have attached a short >>> presentation I did for the replication team at MySQL a few years ago, which >>> describes this approach. >>> >>>> * determine SCN from a central source which uses consensus/paxos, etc >>> That just adds a single point of failure, which of course can be used, but >>> does >>> not scale very well, since you then need to replicate the SCN central >>> source as >>> well, and then you need to have an SCN for every SCN distributed (assuming >>> you >>> use the same scheme for checking consistency of the SCN server), which is >>> kind >>> of recursive. >>> >>>> Does the idea of supporting something like a SCN and exposing it in the >>>> protocol to support use case such as above seem valid? >>> Absolutely. >>> >>> However, it would be nice to be able to handle a branching temporal model as >>> well, not just a linear temporal model. There were some mention that slaves >>> in >>> an "alternate future" frequently is a problem, so right now I am thinking of >>> ways to handle that without introducing an unnecessary overhead. >>> >>> Just my few cents, >>> Mats Kindahl >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~drizzle-discuss >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~drizzle-discuss >>> More help : https://help.launchpad.net/ListHelp > -- Mats Kindahl Senior Software Engineer Database Technology Group Sun Microsystems _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
