So if we wrote all operations for a transaction first to ZooKeeper, we
still need something like a Distributed Transaction Manager to
orchestrate the commit process: Send BatchUpdates to each
RegionServer, ask them to commit, then commit or rollback based on
results from all participating RegionServers. Or is there some more
clever way to use ZooKeeper? Maybe encoding a commit protocol into the
Zookeeper nodes...
Looks like google's datastore has a mechanism for keeping groups of
rows (entity groups) together on the same server (datastore node).
Then they allow transactions only on rows in the same group. This way
they don't have to worry about distributed transactions. Rather than
locking, they use optimistic concurrency control. This means they do
the transaction in a sandbox, then check for conflicts from other
transactions before committing.
-clint
On Tue, May 27, 2008 at 2:13 PM, stack <[EMAIL PROTECTED]> wrote:
> Clint Morgan wrote:
>> Zookeeper makes good sense for distributed locking to get isolation.
>> But we still need transaction start, commit, and rollback to get
>> atomicity. I think this properly belongs in hbase.
>>
> Since all clients are going via zookeeper anyways ('isolation'), maybe
> it'd be better to just run the whole transaction management out of
> zookeeper? Clients would open a transaction on zookeeper and put their
> edits there so they were available for rollback and/or commit. If client
> died midway, could ask zookeeper for outstanding transactions and pickup
> whereever it'd left off. Otherwise, on success (or rollback), clean up
> the transaction log.
>
> Alternatively, all clients would have to go via the hbase master so it
> could orchestrate row access. Master would need to hold outstanding
> transactions somewhere either in an in-memory transactions catalog table
> or itself over in zookeeper.
>
> St.Ack
>
