>>It's very useful for HBase, YARN, etc. to do hot standby. I've talked to them and they are very excited about it. I would think that as persistent, recursive, auto-retried watches. Just like we don't have ConnLoss any more.Like I said, it's all about motivations..
Can you clarify Hongchao? What is useful for doing hot standby? I guess I don't really understand the feature. It would be nice to know to see what we might be able to add to ZK, after you finish getting netty through :) C On Wed, Apr 8, 2015 at 5:09 PM, Hongchao Deng <[email protected]> wrote: > > Don't we kind of have that implicitly in the xids? Could weexpose that, > if it is useful?No. We couldn't because it's been over-written.It's very > useful for HBase, YARN, etc. to do hot standby. I've talked to them and > they are very excited about it. I would think that as persistent, > recursive, auto-retried watches. Just like we don't have ConnLoss any > more.Like I said, it's all about motivations.. I'm currently trying to > refactor the quorum network to get netty in though.. Hope I can catch up > codebase faster and try to get this point clearer. > - Hongchao Deng > > > Date: Wed, 8 Apr 2015 16:54:11 -0400 > > Subject: Re: design thoughts: node TTLs > > From: [email protected] > > To: [email protected] > > > > To the question of "TTL not tied to session": > > > > As far as I know (and again, if we have any etcd experts in the house I'd > > like to hear otherwise) TTL is an attempt to make auto-cleanup happen > when > > you have a stateless client model, aka, http. That is the point. You can > > disagree that this is useful but it is pretty hard to have a > > stateless-based system with sessions required if you want to create nodes > > that clean themselves up. There is clear evidence that people are having > a > > lot of trouble writing clients for ZK especially in languages like Ruby, > > and both of the major alternatives out there, etcd and Consul, rely on > > http-based APIs (although Consul has some session stuff going on under > the > > covers that I honestly don't understand yet so it may do more magic with > > that). > > > > Spitballing, I think that you'd want to create a special monitor for > TTL'ed > > nodes that tracked their last touch and auto-deleted on timeout, kind of > > the same way we do with sessions only not with the session-specific > > heartbeat, but via an explicit TTL update via an update on that node. > Does > > that make sense? > > > > For the rest: > > > > I don't know enough about http2 to comment on that, maybe that is the > right > > way to go :) > > > > Hongchao: Distributed version management, meaning, the version of the > data > > in a node? Don't we kind of have that implicitly in the xids? Could we > > expose that, if it is useful? > > > > C > > > > On Wed, Apr 8, 2015 at 4:31 PM, Hongchao Deng <[email protected]> > > wrote: > > > > > 1. http?http is going to be out of date soon.. I would suggest http2 > based > > > grpc: > > > http://www.grpc.io/ > > > It also facilitates other client language choices. > > > 2. ttl?If you want to create an ephemeral node, TTL isn't a good > design. > > > The notion of TTL comes from the lease in Chubby. > > > It's all about motivations. IIUC, ZooKeeper was built for:- > configuration > > > management (metadata store)- leader election > > > Other than these two, etcd provides one more: distributed version > > > management. This is related to Kubernetes design. > > > 3. redesign?Any plan to start ZK-4? A summer project would be enough to > > > start :) > > > - Hongchao Deng > > > > > > > Date: Wed, 8 Apr 2015 15:09:25 -0400 > > > > Subject: design thoughts: node TTLs > > > > From: [email protected] > > > > To: [email protected] > > > > > > > > All, > > > > > > > > I've been doing a bit of research on etcd as part of work for an > upcoming > > > > talk, and it has gotten me thinking about what it would take to > create an > > > > http version of ZK for certain operations. For many operations you > could > > > > put an http proxy in front of ZK to translate, even implementing the > > > > "long-poll-style" watch operation to some extent. But it would be > very > > > hard > > > > to do a temporary node via a proxy without a lot of proxy failover > > > > complexity. > > > > > > > > As a bit of background, if you want to do an "ephemeral" node in > etcd, > > > you > > > > basically create a key with a TTL. Unless the key is updated with a > new > > > > TTL, the key will auto-expire when the TTL is reached. Now, I have a > lot > > > of > > > > thoughts about this (seems like you have to implement heartbeats via > http > > > > to truly mimic ephemeral nodes which may not be as simple as all this > > > http > > > > sounds), but I do think that if there is appetite for easy http > access > > > for > > > > consensus systems we should at least take the time to think about > what it > > > > would take for us to provide this. In particular, I think we'd have > to > > > make > > > > it possible to create a node with a TTL that is not tied to a > particular > > > > session. > > > > > > > > Curious to see if anyone has any thoughts on this. It seems like a > bit > > > of a > > > > shame that ZK, which is a good battle-tested system, is frequently > being > > > > passed-over these days because of the complexity of clients, and the > fact > > > > that it is really pretty damn hard to do a client impl in certain > > > languages > > > > (Ruby is the notable one I've heard). > > > > > > > > Best, > > > > C > > > > > > > >
