IMO this is a good idea Camille, we've considered doing this for some of the same reasons you mentioned, poor language support and/or stateless short lived clients that cannot keep a long running TCP connection with the server. The design we've informally considered was similar to what you described. One caveat being, that we see enough value in the 'health ping' to implement it as a generic health/ping service tracking the health of the clients. Essentially, separating the health ping functionality in zookeeper as a separate standalone service supporting HTTP.
> Date: Thu, 9 Apr 2015 10:05:44 -0400 > Subject: Re: design thoughts: node TTLs > From: [email protected] > To: [email protected]; [email protected] > > Yeah agreed. Where can we find someone to do this work? :) Who has a smart > intern this summer?? > On Apr 8, 2015 6:20 PM, "Michi Mutsuzaki" <[email protected]> wrote: > > > Yup that makes sense. > > > > In terms of deployment, it would be nice if there is an option to run > > the http server in the same process as ZooKeeper server :) > > > > On Wed, Apr 8, 2015 at 1:54 PM, Camille Fournier <[email protected]> > > wrote: > > > To the question of "TTL not tied to session": > > > > > > As far as I know (and again, if we have any etcd experts in the house I'd > > > like to hear otherwise) TTL is an attempt to make auto-cleanup happen > > when > > > you have a stateless client model, aka, http. That is the point. You can > > > disagree that this is useful but it is pretty hard to have a > > > stateless-based system with sessions required if you want to create nodes > > > that clean themselves up. There is clear evidence that people are having > > a > > > lot of trouble writing clients for ZK especially in languages like Ruby, > > > and both of the major alternatives out there, etcd and Consul, rely on > > > http-based APIs (although Consul has some session stuff going on under > > the > > > covers that I honestly don't understand yet so it may do more magic with > > > that). > > > > > > Spitballing, I think that you'd want to create a special monitor for > > TTL'ed > > > nodes that tracked their last touch and auto-deleted on timeout, kind of > > > the same way we do with sessions only not with the session-specific > > > heartbeat, but via an explicit TTL update via an update on that node. > > Does > > > that make sense? > > > > > > For the rest: > > > > > > I don't know enough about http2 to comment on that, maybe that is the > > right > > > way to go :) > > > > > > Hongchao: Distributed version management, meaning, the version of the > > data > > > in a node? Don't we kind of have that implicitly in the xids? Could we > > > expose that, if it is useful? > > > > > > C > > > > > > On Wed, Apr 8, 2015 at 4:31 PM, Hongchao Deng <[email protected]> > > > wrote: > > > > > >> 1. http?http is going to be out of date soon.. I would suggest http2 > > based > > >> grpc: > > >> http://www.grpc.io/ > > >> It also facilitates other client language choices. > > >> 2. ttl?If you want to create an ephemeral node, TTL isn't a good design. > > >> The notion of TTL comes from the lease in Chubby. > > >> It's all about motivations. IIUC, ZooKeeper was built for:- > > configuration > > >> management (metadata store)- leader election > > >> Other than these two, etcd provides one more: distributed version > > >> management. This is related to Kubernetes design. > > >> 3. redesign?Any plan to start ZK-4? A summer project would be enough to > > >> start :) > > >> - Hongchao Deng > > >> > > >> > Date: Wed, 8 Apr 2015 15:09:25 -0400 > > >> > Subject: design thoughts: node TTLs > > >> > From: [email protected] > > >> > To: [email protected] > > >> > > > >> > All, > > >> > > > >> > I've been doing a bit of research on etcd as part of work for an > > upcoming > > >> > talk, and it has gotten me thinking about what it would take to > > create an > > >> > http version of ZK for certain operations. For many operations you > > could > > >> > put an http proxy in front of ZK to translate, even implementing the > > >> > "long-poll-style" watch operation to some extent. But it would be very > > >> hard > > >> > to do a temporary node via a proxy without a lot of proxy failover > > >> > complexity. > > >> > > > >> > As a bit of background, if you want to do an "ephemeral" node in etcd, > > >> you > > >> > basically create a key with a TTL. Unless the key is updated with a > > new > > >> > TTL, the key will auto-expire when the TTL is reached. Now, I have a > > lot > > >> of > > >> > thoughts about this (seems like you have to implement heartbeats via > > http > > >> > to truly mimic ephemeral nodes which may not be as simple as all this > > >> http > > >> > sounds), but I do think that if there is appetite for easy http access > > >> for > > >> > consensus systems we should at least take the time to think about > > what it > > >> > would take for us to provide this. In particular, I think we'd have to > > >> make > > >> > it possible to create a node with a TTL that is not tied to a > > particular > > >> > session. > > >> > > > >> > Curious to see if anyone has any thoughts on this. It seems like a bit > > >> of a > > >> > shame that ZK, which is a good battle-tested system, is frequently > > being > > >> > passed-over these days because of the complexity of clients, and the > > fact > > >> > that it is really pretty damn hard to do a client impl in certain > > >> languages > > >> > (Ruby is the notable one I've heard). > > >> > > > >> > Best, > > >> > C > > >> > > >> > >
