Greetings,
It was great to meet everybody at Developer day. I have been catching up
with all the docs on the protocol and discussing them with our CTO and in
the last brainstorm with him we realized that although the proposed protocol
offers great enhancements in terms of exposing shard id / QUERY / QUERY_RO
in the protocol itself, the following idea would make it even better.
To simplify and explain the problem consider the following architecture:
(also note that this would apply to sharded DBD scenario as well)
Master Slave/Replica
M1 ---> R1
---> R2
---> R3
...
---> R<n>
* R1, R2 use a persistent storage engine (primarily) to serve as backups to
M1
* R3...RN use memory based/memcache storage engines and used to handle most
of the "READ ONLY" traffic.
* Data is being updated at the M1.
* Data is being replicated to each of the replicas R1..RN.
* The replicas can be applying the changes at master at different rates and
are thus at different consistency state's
* The application assumes eventual consistency. This implies the following
all request for data made by a thread should only be serviced with data that
is at a consistency state higher than or equal to of the previous state.
Using timestamp as a crude gauge to measure consistency if thread TH1
requests data at 10:00 all subsequent data request from TH1 should only see
data that is consistent as of 10:00 or later.
* In reality timestamp is not the correct way to track consistency, which is
where we rely on SCN within oracle to achieve consistent reads. The simplest
definition of SCN is a increasing counter which determines the consistency
state of the database.
* C<n> indicates the consistency state.(Needless to say this needs to be
serialized and determined at the end of a transaction)
Consider the following use case in the above architecture:
* all request where QUERY_RO can be proxied to all servers but all QUERY
i.e. writes/updates need to be sent to and can be served only by M1.
* new thread TH1 updates (either, INSERT/UPDATE/DATE/DDL) the data at the
master, it received a C1 as part of the confirmation of the update and is
set in the protocol.
* all subsequent QUERY_RO requests via TH1 include C1 as part of the request
(set in protocol) thus can be served only by servers that have consistency
state >=C1. Each satisfied request resets the consistency state in the
protocol.
* lets assume TH2 performs multiple updates resulting in the M1 to be at
C50. R1 is at C45, R2 is at C38, R3 is at R24, etc
* TH1 request data (QUERY_RO) and is at C24. The request can be routed to
any of the candidate servers which have consistency state >= C24.
* a new TH3 requests data (QUERY_RO) with empty consistent state (or 0) and
thus can be served by the first available server.
As indicated above adding the concept of SCN and exposing it via the
protocol would make this a really awesome feature. Some other enhancements
that would make the SCN concept even more rock solid include:
* Oracle's SCN is tied to the Oracle instance and is not portable from one
machine to another, except incase of physical standby. This makes it very
difficult to implement partial replication(ability to replicate only certain
objects/schemas), support for multi-master multi-slave replicated
environment. Not to mention moving a slave from one server to another
impossible unless the new master is a physical standby of the original
server.
* support manual "override" of the SCN for slaves, this would allow the
slaves to inherit the SCN from the master per transaction making
slave->master promotion easier.
* determine SCN from a central source which uses consensus/paxos, etc
Does the idea of supporting something like a SCN and exposing it in the
protocol to support use case such as above seem valid?
-Adi Alurkar
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
