OK - so this is two options for doing the same thing. You use a Leader Election algorithm to make sure that only one node in the cluster is operating on a work unit. Curator has an implementation (it's really just a distributed lock with a slightly different API).
-JZ On 1/5/12 12:04 AM, "Josh Stone" <[email protected]> wrote: >Thanks for the response. Comments below: > >On Wed, Jan 4, 2012 at 10:46 PM, Jordan Zimmerman ><[email protected]>wrote: > >> Hi Josh, >> >> >Second use case: Distributed locking >> This is one of the most common uses of ZooKeeper. There are many >> implementations - one included with the ZK distro. Also, there is >>Curator: >> https://github.com/Netflix/curator >> >> >First use case: Distributing work to a cluster of nodes >> This sounds feasible. If you give more details I and others on this list >> can help more. >> > >Sure. I basically want to handle race conditions where two commands that >operate on the same data are received by my cluster of znodes, >concurrently. One approach is to lock on the data that is effected by the >command (distributed lock). Another approach is make sure that all of the >commands that operate on any set of data are routed to the same node, >where >they can be processed serially using local synchronization. Consistent >hashing is an algorithm that can be used to select a node to handle a >message (where the inputs are the key to hash and the number of nodes in >the cluster). > >There are various implementations for this floating around. I'm just >interesting to know how this is working for anyone else. > >Josh > > >> >> -JZ >> >> ________________________________________ >> From: Josh Stone [[email protected]] >> Sent: Wednesday, January 04, 2012 8:09 PM >> To: [email protected] >> Subject: Use cases for ZooKeeper >> >> I have a few use cases that I'm wondering if ZooKeeper would be suitable >> for and would appreciate some feedback. >> >> First use case: Distributing work to a cluster of nodes using consistent >> hashing to ensure that messages of some type are consistently handled by >> the same node. I haven't been able to find any info about ZooKeeper + >> consistent hashing. Is anyone using it for this? A concern here would be >> how to redistribute work as nodes come and go from the cluster. >> >> Second use case: Distributed locking. I noticed that there's a recipe >>for >> this on the ZooKeeper wiki. Is anyone doing this? Any issues? One >>concern >> would be how to handle orphaned locks if a node that obtained a lock >>goes >> down. >> >> Third use case: Fault tolerance. If we utilized ZooKeeper to distribute >> messages to workers, can it be made to handle a node going down by >> re-distributing the work to another node (perhaps messages that are not >> ack'ed within a timeout are resent)? >> >> Cheers, >> Josh >>
