Alex Yurchenko wrote:
On Fri, 24 Jul 2009 10:56:20 +0200, Paul McCullagh
<[email protected]> wrote:
On Jul 23, 2009, at 3:15 PM, Stewart Smith wrote:
On Tue, Jul 21, 2009 at 09:28:54PM -0700, MARK CALLAGHAN wrote:
How is the serial log to be kept in sync with a storage engine given
the Applier interface? MySQL uses two phase commit, but the Applier
interface has one method, ::apply(). In addition to methods for
performing 2PC, keeping a storage engine and the serial log in sync
requires additional methods for crash recovery to support commit or
rollback of transactions in state PREPARED in the storage engine
depending on the outcome recorded in the serial log.
The bit that keeps banging in my head in regards to this is storing it
in the same engine as part of the transaction and so avoiding 2pc.
We discussed this on Drizzle Day, and that was my recommendation.
This would mean, after a transaction has committed, the replication
system asks the engine for a "list of operations" that were performed
by the transaction.
Right, as I mentioned in my original post. If I add a dynamically-sized
array of raw bytes, accessible via std::string, to the Command message's
transaction_context container called engine_passthrough, which could be
accessed as raw bytes via std::string's c_str() method, like so:
drizzled::message::Command &command= get_command_from_somewhere();
const drizzled::message::Command::TransactionContext &tc;
tc= command.transaction_context();
if (tc.has_engine_passthrough())
{
const std::string &engine_stuff= tc.engine_passthrough();
const uint8_t *raw_bytes= (uint8_t *) engine_stuff.c_str();
/* You can now do whatever you want with the raw bytes... */
}
Would this work? It would allow the engine to have a space in the
protocol buffer to put whatever it wanted in there. At the same time,
it would leave it entirely up to the engine to manage what goes in and
comes out of that storage space...
What do you think?
Again, the more specific examples of the kind of thing you'd want to put
in that storage space, the easier it is for me to visualize the changes
needed in the API.
An alternative to this would be using GPB's extension API, and having
each storage engine create its own protobuffer message subclass which
could be stored in the transaction context...but that would mean using
C++, and I know how all you storage guys are stuck on C. ;P
Cheers!
jay
For engines that have this information in their transaction log, it is
a relatively simple task.
This is absolutely the way to go. From the replication perspective picking
log events straight form the storage engine transactional log would save
quite a lot of IO and CPU. In addition those log events can be in the
engine native form, so a blast to apply on a slave.
Then, in 99.9% of cases when there are not cross engine
transactions, we
never need 2pc.
Although I haven't given this intense deep thought as to various
corner cases...
--
Stewart Smith
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
