------------------- Jesse Yates 240-888-2200 @jesse_yates jyates.github.com
On Wed, Apr 18, 2012 at 3:34 PM, Todd Lipcon <[email protected]> wrote: > On Wed, Apr 18, 2012 at 3:29 PM, Jesse Yates <[email protected]> > wrote: > > Its makes dealing with a lot of the zk operations easier to reason about. > > Consider the 'create with parents' or the 'delete recursively' use cases > - > > non-transactional guarantees means we have a lot of extra logic for > > checking under parents on creation, synchronous operations requiring > extra > > checking/notification etc; in short, its a lot of logic for things that > > aren't, on a high-level, all that complicated (and are thought of > > transactionally). > > But how much do we really do these things? Isn't it only when we > bootstrap a new cluster? > Depends on the operation - the replication code has a lot of reliance of moving zk nodes around and the hfile backup stuff that is coming also does dynamic znode allocation. > > > > > > These are the corner cases that are harder to find. For instance, what if > > the parent is there, but the child has been put there - I need to leave a > > watch and then do an update when children changed, and then read in all > the > > children, which could be actually more than I'm looking for, which leads > to > > a bunch of extra IO. If you can be sure that this goes away, we can get > > much simpler code - notify for the node and then read the children. If > the > > children aren't there, they won't show up. this is just one possible > > example, but others can also be constructed. > > You mean in the middle of a recursive op? > Yeah, exactly. Its an annoying use case and one easily messed up. > > > > > It also can enable some really nice, new use cases. For instance, I've > > discussed doing a Paxos-like mechanism that is far more easily enabled by > > having transactional read-modify-write semantics for things like dynamic > > table configuration (not proposing this as a jira (yet), and don't want > to > > get into if its a good idea, but rather just as an example for possible > new > > uses). > > ZK already has version-checking setData() calls which is enough for > optimistic concurrency... I don't get the "paxos-like mechanism" here > when ZK is already giving us consensus and atomic CAS. > Oh yeah, you are right. A little confused there on where the ZK version stuff comes from (since you don't actually set the version, but are just given it by zk) > > -Todd > > > > > On Wed, Apr 18, 2012 at 2:49 PM, Todd Lipcon <[email protected]> wrote: > > > >> ZK 3.4.x still isn't marked "stable", so I'm a little hesitant to move > >> to requiring it just yet. > >> > >> What tangible benefit do we gain from the "transactional" methods? > >> > >> -Todd > >> > >> On Wed, Apr 18, 2012 at 1:48 PM, lars hofhansl <[email protected]> > >> wrote: > >> > +1 > >> > > >> > > >> > > >> > ----- Original Message ----- > >> > From: Jesse Yates <[email protected]> > >> > To: [email protected] > >> > Cc: > >> > Sent: Wednesday, April 18, 2012 1:33 PM > >> > Subject: requiring zookeeper 3.4.x in 0.96 > >> > > >> > Hi all, > >> > > >> > I was working on some patches to make ZK a bit more sane using the > >> > transaction operations recently added (in jira: > >> > https://issues.apache.org/jira/browse/ZOOKEEPER-965 > >> > < > >> > http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29 > >> > > >> > and in the api here: > >> > > >> > http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29 > >> ). > >> > This takes care of a lot of race conditions, etc. that are inherently > >> > present in zk. > >> > > >> > Currently we (afaik) are not requiring ZooKeeper v.3.4.x in production > >> and > >> > instead allow users to run down to at least 3.3.x with the hbase's > 3.4.x > >> > client. Adding in transaction support will break this cross-version > >> > functionality without a way of falling back as ZK doesn't (afaik) > support > >> > client-side look up of the server's version. Further, the fallback > >> > functionality would be very dangerous if we assume transactions but > fall > >> > back to the non-transactional method; if we don't assume transactions, > >> then > >> > the transactional method is just wasted effort. > >> > > >> > I'm proposing that in the singularity (0.96), when we move to > requiring > >> > Hadoop v1.0+ we also make it a requirement that people run ZooKeeper > >> v3.4+ > >> > > >> > Thoughts? > >> > > >> > Thanks! > >> > > >> > ------------------- > >> > Jesse Yates > >> > 240-888-2200 > >> > @jesse_yates > >> > jyates.github.com > >> > > >> > >> > >> > >> -- > >> Todd Lipcon > >> Software Engineer, Cloudera > >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
