Hi Robert,
The applying happens with MySQL row based replication events (RBR). However,
we catch them from the master (on the fly), and they never go to binlog file.
In the test, I had several connections to master and a slave with one or
several applier threads configured. Observation was that "RBR stream"
coming from the master, could not flood even a single threaded slave.
Applying of RBR events happens so effectively that event queue does not
build up.
Note also, that the master saves here a little by not writing to binlog file.
I think that I need more CPU/cores to see the potential benefit of parallel
applying.
Cheers, Seppo
Quoting Robert Hodges <[email protected]>:
Hi Seppo,
These results are very interesting. When you apply the change sets
are you submitting SQL or operating at the storage plug-in level?
A lot of the latency in MySQL replication has traditionally been due
to the cost of rerunning statements on the slave. I have not seen
latency comparison for row replication on MySQL 5.1 but should be in
a position to compare them as well as test parallel async
replication by early summer using Tungsten Replicator.
Cheers, Robert
P.s., thanks for the heads up on the patent mine laid by our pals at IBM.
On 5/31/09 11:27 PM PDT, "[email protected]"
<[email protected]> wrote:
Hi,
I have done some experimentations with parallel applying with the Galera
replication engine and to my surprise I could not get much performance gain.
In Galera, the applying happens as follows:
0. Galera is a merge over MySQL 5.1.* and transactions to be applied
contain RBR events
1. a predefined number (1-n) of applier threads will be started at startup
2. applying queue manager assigns tasks (incoming trxs) for the appliers.
For each new task, QM looks for the modified row IDs to see if there
is a conflict with running tasks. Parallelism is allowed only for
non-conflicting tasks.
3. appliers synchronize before commit, so that commit order is preserved
Benchmarking did not show any significant benefit of having parallel
applying.
Reason is that RBR applying is so fast compared to processing
of SQL queries that applying queue did not build up. It turned out that
several applier threads were useful only when transaction length was ~200 SQL
statements.
I have blogged something more about this experimentation here:
http://www.codership.com/content/parallel-applying. But, it should be
enough to
read just the title...
Note that just by looking at the row IDs in the RBR, it is not
guaranteed that
the actual processing of these transactions would not conflict. SE
can require
more "resources" than what is visible in the RBR events (we called this
asymmetric lock granularity issue). However, if appliers do conflict,
it is safe to abort the later one and replay immediately. This will
start hurting
performance if aborting is too frequent.
Despite of the bad results, I have not completely lost my hope for parallel
applying. This was just the first shot, I need to retry with newest code,
different SQL profiles and better HW. We allocate R&D tasks for June sprint
today, I try to fit "parallel applying revisited" in the task list.
Regards,
Seppo Jaakola
--
www.codership.com
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
--
Robert Hodges, CTO, Continuent, Inc.
Email: [email protected]
Mobile: +1-510-501-3728 Skype: hodgesrm
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
