Hello,
w.r.t to my last mail I have following modification in my
proposal .
1. I have looked into *how drizzle replication works* and got the following
information:
For DML we have GPB based 'transaction.proto' which is then converted to
it's class file which solves the low-level problems of serializing and
deserializing, and 'versioning' formatted binary streams of data so that
raw data can change its structure without having to re-implement all new
serialization and deserialization routines. Drizzle has ReplicationService
which converts the row change event into a Transaction Message <Transaction
Context, Statement 1...n>. Now this message is read by reader plugin and
then applier plugin replicate it on the slave (subscriber).
*2. To Replicate from MySQL to Drizzle (Gsoc Idea): *
This process is quiet simple. We read the MySQL row based binlog from the
publisher (master), then we *form* the new Transaction Message using
a slightly modified version of 'transaction.proto' file. This Event of
forming Transaction Message will be CALLED in the * binlog_event of
libdrizzle-redux binlog api *. Once we have the Transaction Message, reader
plugin will push it to publisher plugin on master and then the salve
machine (subscriber) will read and replicate it via Transaction Applier.
The Trickiest part is that we are using all the Replication modules of
Drizzle but to take MySQL binlog and covert them into Drizzle's Transaction
message we are building a NEW module.
*So at last we have reduced the problem* into a module which converts
MySQL binlog into Drizzle's Transaction Message. Possible required changes:
1. As MySQL binlogs are row based, we need to change or introduce new
'Blueprints' for data in proto file (this file will be a modified copy of
current transaction.proto).
2. Also the log can be big so to provide the error checking ans safe
replication we can add CHECKSUM.
3. We can use some kind of Heuristics to remove DDL based inconsistencies
between MySQL and Drizzle.
please verify the proposal.
On Sat, Apr 20, 2013 at 7:42 PM, kuldeep porwal <[email protected]>wrote:
> Hi Sir,
>
> > Start at ways to read the MySQL binlog, which may be part
> of libdrizzle-redux as well as teh binlog api code that's been
> sitting around.
>
> 1. Checked libdrizzle-redux binlog api along with binary logs for MySQL.
> 2. I am now quiet familiar with new binlog api of libdrizzle and also
> tested the code which gets binlog file from MySQL (Row Based form) and
> prints the event associated in it.
> 3. For the GSOC project point of view
> 3.1 Although RBR is now more in use and its safe too, we can infact
> take any replication format SBR or RBR to replicate MySQL database.
> 3.2 I think of two ways for this
> a) As I read about drizzle slave replication, it does
> multi-master replication.
> (on master:) Transaction --> InnoDB ---> (on slave :)
> Replication reader ---> Replication Applier
> Here we can take MySQL binlogs then from that we can form
> logs like InnoDB (mentioned above) and finally the replication process will
> be
> same as drizzle salve replication.
>
> b) we can take binlogs from MySQL and write new Replicater
> Reader and Applier for it. It will be more like MySQL way IO thread and
> MySQL thread.
>
>
> About DDL part: Yes it may cause some problem but that can be solved
> easily by some heuristic and prototyping. I am thinking to work on this
> part later after results :-).
>
> Please suggest: Am I going in right direction or not? Also what should I
> write in my GSOC-Proposal.
>
>
> On Thu, Apr 18, 2013 at 1:26 PM, Stewart Smith
> <[email protected]>wrote:
>
>> kuldeep porwal <[email protected]> writes:
>> > Thanks for the idea. But row-based replication (RBR) was implemented in
>> > MySQL 5.1.5, so you cannot replicate using row-based replication from
>> any
>> > MySQL 5.0 or later master to a slave older than MySQL 5.1.5. That
>> shouldn't
>> > be the problem for us, is it?
>>
>> Requiring 5.1 or later is fine. Not everything would be able to be
>> easily replicated either - after all, DDL could be problematic, as would
>> data that is invalid (0th month/year etc).
>>
>> > I will look into the depth for this and get back to you.
>>
>> Start at ways to read the MySQL binlog, which may be part of
>> libdrizzle-redux as well as teh binlog api code that's been sitting
>> around.
>>
>> --
>> Stewart Smith
>>
>
>
>
> --
> Regards,
> Kuldeep Porwal
> IIIT Hyderabad
> 09550605256
> http://web.iiit.ac.in/~kuldeep.porwal
>
--
Regards,
Kuldeep Porwal
IIIT Hyderabad
09550605256
http://web.iiit.ac.in/~kuldeep.porwal
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
