Parent nodes multi-step transactions

2010-08-23 Thread Gustavo Niemeyer
Greetings, We (a development team at Canonical) are stumbling into a situation here which I'd be curious to understand what is the general practice, since I'm sure this is somewhat of a common issue. It's quite easy to describe it: say there's a parent node A somewhere in the tree. That node

Re: Parent nodes multi-step transactions

2010-08-23 Thread Mahadev Konar
Hi Gustavo, Usually the paradigm I like to suggest is to have something like /A/init Every client watches for the existence of this node and this node is only created after /A has been initialized with the creation of /A/C or other stuff. Would that work for you? Thanks mahadev On 8/23/10

Re: Parent nodes multi-step transactions

2010-08-23 Thread Gustavo Niemeyer
Hi Mahadev,  Usually the paradigm I like to suggest is to have something like /A/init Every client watches for the existence of this node and this node is only created after /A has been initialized with the creation of /A/C or other stuff. Would that work for you? Yeah, this is what I

Non Hadoop scheduling frameworks

2010-08-23 Thread Todd Nine
Hi all, We're using Zookeeper for Leader Election and system monitoring. We're also using it for synchronizing our cluster wide jobs with barriers. We're running into an issue where we now have a single job, but each node can fire the job independently of others with different criteria in the

Re: Non Hadoop scheduling frameworks

2010-08-23 Thread Mahadev Konar
Hi Todd, Just to be clear, are you looking at solving UC1 and UC2 via zookeeper? Or is this a broader question for scheduling on cassandra nodes? For the latter this probably isnt the right mailing list. Thanks mahadev On 8/23/10 4:02 PM, Todd Nine t...@spidertracks.co.nz wrote: Hi all,

Re: Parent nodes multi-step transactions

2010-08-23 Thread Dave Wright
For my $0.02, I really think it would be nice if ZK supported lightweight transactions. By that, I simply mean that a batch of create/update/delete requests could be submitted in a single request, and be processed atomically (if any of the requests would fail, none are applied). I know

Re: Non Hadoop scheduling frameworks

2010-08-23 Thread Ted Dunning
These are pretty easy to solve with ZK. Ephemerality, exclusive create, atomic update and file versions allow you to implement most of the semantics you need. I don't know of any recipes available for this, but they would be worthy additions to ZK. On Mon, Aug 23, 2010 at 11:33 PM, Todd Nine