I think the issues are very likely the same, whether it's separate or part of the same process. Once you want to worry about HA for the rest/thrift/etc... api you need to have multiple processes. Once you have multiple processes I'm not sure it matters whether the process is in zk or a separate process. The problems are the same.
Patrick On Tue, Apr 14, 2015 at 8:43 AM, Flavio Junqueira < [email protected]> wrote: > That's part of the reason why I find easier to start as an application of > ZK rather than getting stuff entangled with the server. It sounds easier to > have this functionality decoupled from the core, which possibly implies > following the curator route to implement this. But, maybe I'm just being > old school zk. > -Flavio > > > On Monday, April 13, 2015 11:01 PM, Patrick Hunt <[email protected]> > wrote: > > > > That's really the easy part. Coming up with something that will work in > our > context is the harder bit IMO. For example, if you have multiple REST > service endpoints how will that work, one per zk server? Are you going to > tie a particular client session to a particular server, how will sessions > be maintained. Would you want to put a http load balancer in between? > Typically you want to do this otw you need more logic on the client (like > we have today in the java client). Seems like there's a lot of these types > of issues that would need to be worked through. Same (but different) for > thrift support. > > Patrick > > On Mon, Apr 13, 2015 at 2:19 PM, Flavio Junqueira < > [email protected]> wrote: > > > I also like the plugin idea, it's a good one. > > > > -Flavio > > > > > On 13 Apr 2015, at 18:58, Patrick Hunt <[email protected]> wrote: > > > > > > Once something is generally available, embedded and easy to enable, it > > > probably won't take much for someone to extend. Once they see the > > benefits. > > > Although there are already folks interested in what we have today. > > > > > > We could do the same thing re thrift, or any other "plugin" for that > > > matter, (enable dynamic config of embedded endpoint) if there's > interest. > > > > > > Patrick > > > > > > On Mon, Apr 13, 2015 at 10:48 AM, Camille Fournier <[email protected] > > > > > wrote: > > > > > >> Yes ok agreed. I do personally think fully supporting TTL nodes (and > > >> long-poll based watches) for purposes of enabling the same types of > > >> high-level behaviors as etcd/consul would probably be useful, as per > the > > >> original start of this epic thread. But even starting with an embedded > > >> proxy is a good start. > > >> > > >> C > > >> > > >> On Mon, Apr 13, 2015 at 1:42 PM, Patrick Hunt <[email protected]> > wrote: > > >> > > >>> We already have a proxy. My thought was to take advantage of jetty > now > > >>> being embedded in ZK (3.5+) so that a proxy wouldn't be necessary. > > >>> > > >>> Patrick > > >>> > > >>> On Mon, Apr 13, 2015 at 10:39 AM, Camille Fournier < > [email protected] > > > > > >>> wrote: > > >>> > > >>>> I've lost the thread now. Are we talking about inside of the ZK > > servers > > >>>> themselves, or as a proxy in front of them? > > >>>> > > >>>> On Mon, Apr 13, 2015 at 1:35 PM, Jordan Zimmerman < > > >>>> [email protected]> wrote: > > >>>> > > >>>>> I agree that get/set kinds of things is useful - especially for OPs > > >>>>> groups. What if ZK had some sort of pluggable server support? Ops > > folks > > >>>>> could plug in a simple REST server, others could plug in a Thrift > > server, > > >>>>> etc. > > >>>>> > > >>>>> > > >>>>> > > >>>>> On April 13, 2015 at 12:31:51 PM, Patrick Hunt ([email protected]) > > >>>>> wrote: > > >>>>> > > >>>>> I think it depends on the use case. I've had a number of folks > asking > > >>>>> for > > >>>>> the REST support, in those cases I think they are mainly interested > > in > > >>>>> active participation (get/set rather than watches/ephemerals). > > >>>>> > > >>>>> The SPEC page has all the details on what the REST proxy currently > > >>>>> supports. > > >>>>> > > >>>>> Patrick > > >>>>> > > >>>>> On Mon, Apr 13, 2015 at 8:35 AM, Jordan Zimmerman < > > >>>>> [email protected]> wrote: > > >>>>> > > >>>>>> How does the current REST code handle watches and background > > >>>>>> notifications? Are clients required to poll periodically? This is > > >>>>> really > > >>>>>> RPC and REST is not ideal for RPC. So, as I mentioned before, I > > think > > >>>>> the > > >>>>>> ZK team should consider a real RPC system like Thrift instead of > > REST. > > >>>>>> > > >>>>>> -Jordan > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> On April 11, 2015 at 1:33:03 PM, Patrick Hunt ([email protected]) > > >>>>> wrote: > > >>>>>> > > >>>>>> We pack quite a bit of functionality in our client side libraries. > > >>>>> That's > > >>>>>> probably one of the main things you'll noticed if you try to use > > >>>>> REST. But > > >>>>>> then if your use case is primarily configuration, or something > that > > >>>>> doesn't > > >>>>>> require sessions then the current REST support should be more than > > >>>>>> sufficient. What are the other systems doing that we are currently > > not > > >>>>>> doing wrt API support? > > >>>>>> > > >>>>>> I'd think that taking our current rest contrib, updating the > > >>>>> dependencies, > > >>>>>> and deploying as part of the embedded jetty would work well with > > >>>>> minimal > > >>>>>> effort. > > >>>>>> > > >>>>>> Patrick > > >>>>>> > > >>>>>> On Sat, Apr 11, 2015 at 9:22 AM, Camille Fournier < > > [email protected] > > >>>>>> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> I agree that they have to write clients that do work, but there > is > > >>>>>> clearly > > >>>>>>> a desire and willingness out there to do that work in exchange > for > > >>>>> a more > > >>>>>>> "obvious" way of interacting. So supporting it should be > something > > >>>>> that > > >>>>>> we > > >>>>>>> consider if the users of such systems want the option. > > >>>>>>> > > >>>>>>> On Sat, Apr 11, 2015 at 12:07 PM, Jordan Zimmerman < > > >>>>>>> [email protected]> wrote: > > >>>>>>> > > >>>>>>>> REST has issues for end users. They will still have to write > > >>>>> clients > > >>>>>> and > > >>>>>>>> do a lot of extra work. REST support is good in most languages > but > > >>>>>> having > > >>>>>>>> native support is superior. That’s why I choose Thrift for the > > >>>>>> CuratorRPC > > >>>>>>>> Proxy. > > >>>>>>>> > > >>>>>>>> -Jordan > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On April 10, 2015 at 9:32:47 PM, Michi Mutsuzaki ( > > >>>>>> [email protected]) > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>> Yeah it's quite painful to manage another set of processes just > > >>>>> for > > >>>>>>>> proxying requests. I'd definitely use this if I can run it > > >>>>> embedded in > > >>>>>>>> the ZooKeeper process. I'm very excited about the idea of being > > >>>>> able > > >>>>>>>> to use curl to look at what's in ZooKeeper :) > > >>>>>>>> > > >>>>>>>> On Fri, Apr 10, 2015 at 6:03 PM, Patrick Hunt <[email protected] > > > > >>>>>> wrote: > > >>>>>>>>> Here's the spec and readme from contrib/rest: > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > > http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/SPEC.txt?view=markup > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > > http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/README.txt?view=markup > > >>>>>>>>> > > >>>>>>>>> The current implementation is a standalone proxy. It's not > > >>>>> embedded > > >>>>>> in > > >>>>>>> zk > > >>>>>>>>> itself. That might be part of the reason. > > >>>>>>>>> > > >>>>>>>>> Patrick > > >>>>>>>>> > > >>>>>>>>> On Fri, Apr 10, 2015 at 4:43 PM, Camille Fournier < > > >>>>>> [email protected]> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Forgive me for not reading the code, can you share in more > > >>>>> detail > > >>>>>> what > > >>>>>>>> the > > >>>>>>>>>> existing REST proxy provides? I'm curious why people are > > >>>>> jumping to > > >>>>>>> use > > >>>>>>>>>> etcd because of ease of use w/http access if we already have > > >>>>>> something > > >>>>>>>> that > > >>>>>>>>>> works? > > >>>>>>>>>> > > >>>>>>>>>> On Fri, Apr 10, 2015 at 7:28 PM, Patrick Hunt < > > >>>>> [email protected]> > > >>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> There's the REST work in contrib. Both Andrei and I worked > > >>>>> on that > > >>>>>>> - I > > >>>>>>>>>> did > > >>>>>>>>>>> the basic support and Andrei added sessions and heartbeating > > >>>>> among > > >>>>>>>> other > > >>>>>>>>>>> improvements. > > >>>>>>>>>>> > > >>>>>>>>>>> Now that 3.5 has embedded Jetty it should be much simpler to > > >>>>> run > > >>>>>>> REST > > >>>>>>>> as > > >>>>>>>>>>> part of the ZK service itself. When the original proxy work > > >>>>> was > > >>>>>> done > > >>>>>>>>>> Jetty > > >>>>>>>>>>> was not yet part of ZK. > > >>>>>>>>>>> > > >>>>>>>>>>> Patrick > > >>>>>>>>>>> > > >>>>>>>>>>> On Thu, Apr 9, 2015 at 10:20 AM, Jordan Zimmerman < > > >>>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Since Curator is now Apache and I'm no longer at Netflix, I > > >>>>> don't > > >>>>>>>> follow > > >>>>>>>>>>>> Netflix messages very much. Sorry about that. > > >>>>>>>>>>>> > > >>>>>>>>>>>> -Jordan > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> On April 9, 2015 at 12:15:02 PM, Camille Fournier ( > > >>>>>>>> [email protected]) > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> Thanks Jordan! I actually asked on Twitter whether Netflix > > >>>>> had > > >>>>>>>> anything > > >>>>>>>>>>>> but > > >>>>>>>>>>>> didn't get a clear answer. > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Thu, Apr 9, 2015 at 11:22 AM, Jordan Zimmerman < > > >>>>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> FYI > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Curator now has a Thrift-based proxy that has all the ZK > > >>>>> APIs > > >>>>>>>> exposed > > >>>>>>>>>> as > > >>>>>>>>>>>>> well as Curator's added APIs and recipes: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> http://curator.apache.org/curator-x-rpc/index.html > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> -Jordan > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On April 8, 2015 at 2:09:33 PM, Camille Fournier ( > > >>>>>>>> [email protected]) > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> All, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I've been doing a bit of research on etcd as part of work > > >>>>> for > > >>>>>> an > > >>>>>>>>>>>> upcoming > > >>>>>>>>>>>>> talk, and it has gotten me thinking about what it would > > >>>>> take to > > >>>>>>>> create > > >>>>>>>>>>>> an > > >>>>>>>>>>>>> http version of ZK for certain operations. For many > > >>>>> operations > > >>>>>>> you > > >>>>>>>>>> could > > >>>>>>>>>>>>> put an http proxy in front of ZK to translate, even > > >>>>>> implementing > > >>>>>>>> the > > >>>>>>>>>>>>> "long-poll-style" watch operation to some extent. But it > > >>>>> would > > >>>>>> be > > >>>>>>>> very > > >>>>>>>>>>>>> hard > > >>>>>>>>>>>>> to do a temporary node via a proxy without a lot of proxy > > >>>>>>> failover > > >>>>>>>>>>>>> complexity. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> As a bit of background, if you want to do an "ephemeral" > > >>>>> node > > >>>>>> in > > >>>>>>>> etcd, > > >>>>>>>>>>>> you > > >>>>>>>>>>>>> basically create a key with a TTL. Unless the key is > > >>>>> updated > > >>>>>>> with a > > >>>>>>>>>> new > > >>>>>>>>>>>>> TTL, the key will auto-expire when the TTL is reached. > > >>>>> Now, I > > >>>>>>> have > > >>>>>>>> a > > >>>>>>>>>> lot > > >>>>>>>>>>>>> of > > >>>>>>>>>>>>> thoughts about this (seems like you have to implement > > >>>>>> heartbeats > > >>>>>>>> via > > >>>>>>>>>>>> http > > >>>>>>>>>>>>> to truly mimic ephemeral nodes which may not be as simple > > >>>>> as > > >>>>>> all > > >>>>>>>> this > > >>>>>>>>>>>> http > > >>>>>>>>>>>>> sounds), but I do think that if there is appetite for > > >>>>> easy http > > >>>>>>>> access > > >>>>>>>>>>>> for > > >>>>>>>>>>>>> consensus systems we should at least take the time to > > >>>>> think > > >>>>>> about > > >>>>>>>> what > > >>>>>>>>>>>> it > > >>>>>>>>>>>>> would take for us to provide this. In particular, I think > > >>>>> we'd > > >>>>>>>> have to > > >>>>>>>>>>>>> make > > >>>>>>>>>>>>> it possible to create a node with a TTL that is not tied > > >>>>> to a > > >>>>>>>>>> particular > > >>>>>>>>>>>>> session. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Curious to see if anyone has any thoughts on this. It > > >>>>> seems > > >>>>>> like > > >>>>>>> a > > >>>>>>>> bit > > >>>>>>>>>>>> of > > >>>>>>>>>>>>> a > > >>>>>>>>>>>>> shame that ZK, which is a good battle-tested system, is > > >>>>>>> frequently > > >>>>>>>>>> being > > >>>>>>>>>>>>> passed-over these days because of the complexity of > > >>>>> clients, > > >>>>>> and > > >>>>>>>> the > > >>>>>>>>>>>> fact > > >>>>>>>>>>>>> that it is really pretty damn hard to do a client impl in > > >>>>>> certain > > >>>>>>>>>>>>> languages > > >>>>>>>>>>>>> (Ruby is the notable one I've heard). > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Best, > > >>>>>>>>>>>>> C > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>> > > >> > > > > > > > >
