Well, except that separate processes might be a worse experience for the people having to deploy and operate the systems. So, in the interest of operational overhead I personally think if we can get it inside of ZK itself that would be good.
C On Tue, Apr 14, 2015 at 12:04 PM, Patrick Hunt <[email protected]> wrote: > I think the issues are very likely the same, whether it's separate or part > of the same process. Once you want to worry about HA for the > rest/thrift/etc... api you need to have multiple processes. Once you have > multiple processes I'm not sure it matters whether the process is in zk or > a separate process. The problems are the same. > > Patrick > > On Tue, Apr 14, 2015 at 8:43 AM, Flavio Junqueira < > [email protected]> wrote: > >> That's part of the reason why I find easier to start as an application of >> ZK rather than getting stuff entangled with the server. It sounds easier to >> have this functionality decoupled from the core, which possibly implies >> following the curator route to implement this. But, maybe I'm just being >> old school zk. >> -Flavio >> >> >> On Monday, April 13, 2015 11:01 PM, Patrick Hunt <[email protected]> >> wrote: >> >> >> >> That's really the easy part. Coming up with something that will work in >> our >> context is the harder bit IMO. For example, if you have multiple REST >> service endpoints how will that work, one per zk server? Are you going to >> tie a particular client session to a particular server, how will sessions >> be maintained. Would you want to put a http load balancer in between? >> Typically you want to do this otw you need more logic on the client (like >> we have today in the java client). Seems like there's a lot of these types >> of issues that would need to be worked through. Same (but different) for >> thrift support. >> >> Patrick >> >> On Mon, Apr 13, 2015 at 2:19 PM, Flavio Junqueira < >> [email protected]> wrote: >> >> > I also like the plugin idea, it's a good one. >> > >> > -Flavio >> > >> > > On 13 Apr 2015, at 18:58, Patrick Hunt <[email protected]> wrote: >> > > >> > > Once something is generally available, embedded and easy to enable, it >> > > probably won't take much for someone to extend. Once they see the >> > benefits. >> > > Although there are already folks interested in what we have today. >> > > >> > > We could do the same thing re thrift, or any other "plugin" for that >> > > matter, (enable dynamic config of embedded endpoint) if there's >> interest. >> > > >> > > Patrick >> > > >> > > On Mon, Apr 13, 2015 at 10:48 AM, Camille Fournier < >> [email protected]> >> > > wrote: >> > > >> > >> Yes ok agreed. I do personally think fully supporting TTL nodes (and >> > >> long-poll based watches) for purposes of enabling the same types of >> > >> high-level behaviors as etcd/consul would probably be useful, as per >> the >> > >> original start of this epic thread. But even starting with an >> embedded >> > >> proxy is a good start. >> > >> >> > >> C >> > >> >> > >> On Mon, Apr 13, 2015 at 1:42 PM, Patrick Hunt <[email protected]> >> wrote: >> > >> >> > >>> We already have a proxy. My thought was to take advantage of jetty >> now >> > >>> being embedded in ZK (3.5+) so that a proxy wouldn't be necessary. >> > >>> >> > >>> Patrick >> > >>> >> > >>> On Mon, Apr 13, 2015 at 10:39 AM, Camille Fournier < >> [email protected] >> > > >> > >>> wrote: >> > >>> >> > >>>> I've lost the thread now. Are we talking about inside of the ZK >> > servers >> > >>>> themselves, or as a proxy in front of them? >> > >>>> >> > >>>> On Mon, Apr 13, 2015 at 1:35 PM, Jordan Zimmerman < >> > >>>> [email protected]> wrote: >> > >>>> >> > >>>>> I agree that get/set kinds of things is useful - especially for >> OPs >> > >>>>> groups. What if ZK had some sort of pluggable server support? Ops >> > folks >> > >>>>> could plug in a simple REST server, others could plug in a Thrift >> > server, >> > >>>>> etc. >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> On April 13, 2015 at 12:31:51 PM, Patrick Hunt ([email protected]) >> > >>>>> wrote: >> > >>>>> >> > >>>>> I think it depends on the use case. I've had a number of folks >> asking >> > >>>>> for >> > >>>>> the REST support, in those cases I think they are mainly >> interested >> > in >> > >>>>> active participation (get/set rather than watches/ephemerals). >> > >>>>> >> > >>>>> The SPEC page has all the details on what the REST proxy currently >> > >>>>> supports. >> > >>>>> >> > >>>>> Patrick >> > >>>>> >> > >>>>> On Mon, Apr 13, 2015 at 8:35 AM, Jordan Zimmerman < >> > >>>>> [email protected]> wrote: >> > >>>>> >> > >>>>>> How does the current REST code handle watches and background >> > >>>>>> notifications? Are clients required to poll periodically? This is >> > >>>>> really >> > >>>>>> RPC and REST is not ideal for RPC. So, as I mentioned before, I >> > think >> > >>>>> the >> > >>>>>> ZK team should consider a real RPC system like Thrift instead of >> > REST. >> > >>>>>> >> > >>>>>> -Jordan >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> On April 11, 2015 at 1:33:03 PM, Patrick Hunt ([email protected]) >> > >>>>> wrote: >> > >>>>>> >> > >>>>>> We pack quite a bit of functionality in our client side >> libraries. >> > >>>>> That's >> > >>>>>> probably one of the main things you'll noticed if you try to use >> > >>>>> REST. But >> > >>>>>> then if your use case is primarily configuration, or something >> that >> > >>>>> doesn't >> > >>>>>> require sessions then the current REST support should be more >> than >> > >>>>>> sufficient. What are the other systems doing that we are >> currently >> > not >> > >>>>>> doing wrt API support? >> > >>>>>> >> > >>>>>> I'd think that taking our current rest contrib, updating the >> > >>>>> dependencies, >> > >>>>>> and deploying as part of the embedded jetty would work well with >> > >>>>> minimal >> > >>>>>> effort. >> > >>>>>> >> > >>>>>> Patrick >> > >>>>>> >> > >>>>>> On Sat, Apr 11, 2015 at 9:22 AM, Camille Fournier < >> > [email protected] >> > >>>>>> >> > >>>>>> wrote: >> > >>>>>> >> > >>>>>>> I agree that they have to write clients that do work, but there >> is >> > >>>>>> clearly >> > >>>>>>> a desire and willingness out there to do that work in exchange >> for >> > >>>>> a more >> > >>>>>>> "obvious" way of interacting. So supporting it should be >> something >> > >>>>> that >> > >>>>>> we >> > >>>>>>> consider if the users of such systems want the option. >> > >>>>>>> >> > >>>>>>> On Sat, Apr 11, 2015 at 12:07 PM, Jordan Zimmerman < >> > >>>>>>> [email protected]> wrote: >> > >>>>>>> >> > >>>>>>>> REST has issues for end users. They will still have to write >> > >>>>> clients >> > >>>>>> and >> > >>>>>>>> do a lot of extra work. REST support is good in most languages >> but >> > >>>>>> having >> > >>>>>>>> native support is superior. That’s why I choose Thrift for the >> > >>>>>> CuratorRPC >> > >>>>>>>> Proxy. >> > >>>>>>>> >> > >>>>>>>> -Jordan >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> On April 10, 2015 at 9:32:47 PM, Michi Mutsuzaki ( >> > >>>>>> [email protected]) >> > >>>>>>>> wrote: >> > >>>>>>>> >> > >>>>>>>> Yeah it's quite painful to manage another set of processes just >> > >>>>> for >> > >>>>>>>> proxying requests. I'd definitely use this if I can run it >> > >>>>> embedded in >> > >>>>>>>> the ZooKeeper process. I'm very excited about the idea of being >> > >>>>> able >> > >>>>>>>> to use curl to look at what's in ZooKeeper :) >> > >>>>>>>> >> > >>>>>>>> On Fri, Apr 10, 2015 at 6:03 PM, Patrick Hunt < >> [email protected]> >> > >>>>>> wrote: >> > >>>>>>>>> Here's the spec and readme from contrib/rest: >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>>> >> > >>>>> >> > >> http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/SPEC.txt?view=markup >> > >>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>>> >> > >>>>> >> > >> http://svn.apache.org/viewvc/zookeeper/trunk/src/contrib/rest/README.txt?view=markup >> > >>>>>>>>> >> > >>>>>>>>> The current implementation is a standalone proxy. It's not >> > >>>>> embedded >> > >>>>>> in >> > >>>>>>> zk >> > >>>>>>>>> itself. That might be part of the reason. >> > >>>>>>>>> >> > >>>>>>>>> Patrick >> > >>>>>>>>> >> > >>>>>>>>> On Fri, Apr 10, 2015 at 4:43 PM, Camille Fournier < >> > >>>>>> [email protected]> >> > >>>>>>>>> wrote: >> > >>>>>>>>> >> > >>>>>>>>>> Forgive me for not reading the code, can you share in more >> > >>>>> detail >> > >>>>>> what >> > >>>>>>>> the >> > >>>>>>>>>> existing REST proxy provides? I'm curious why people are >> > >>>>> jumping to >> > >>>>>>> use >> > >>>>>>>>>> etcd because of ease of use w/http access if we already have >> > >>>>>> something >> > >>>>>>>> that >> > >>>>>>>>>> works? >> > >>>>>>>>>> >> > >>>>>>>>>> On Fri, Apr 10, 2015 at 7:28 PM, Patrick Hunt < >> > >>>>> [email protected]> >> > >>>>>>> wrote: >> > >>>>>>>>>> >> > >>>>>>>>>>> There's the REST work in contrib. Both Andrei and I worked >> > >>>>> on that >> > >>>>>>> - I >> > >>>>>>>>>> did >> > >>>>>>>>>>> the basic support and Andrei added sessions and heartbeating >> > >>>>> among >> > >>>>>>>> other >> > >>>>>>>>>>> improvements. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Now that 3.5 has embedded Jetty it should be much simpler to >> > >>>>> run >> > >>>>>>> REST >> > >>>>>>>> as >> > >>>>>>>>>>> part of the ZK service itself. When the original proxy work >> > >>>>> was >> > >>>>>> done >> > >>>>>>>>>> Jetty >> > >>>>>>>>>>> was not yet part of ZK. >> > >>>>>>>>>>> >> > >>>>>>>>>>> Patrick >> > >>>>>>>>>>> >> > >>>>>>>>>>> On Thu, Apr 9, 2015 at 10:20 AM, Jordan Zimmerman < >> > >>>>>>>>>>> [email protected]> wrote: >> > >>>>>>>>>>> >> > >>>>>>>>>>>> Since Curator is now Apache and I'm no longer at Netflix, I >> > >>>>> don't >> > >>>>>>>> follow >> > >>>>>>>>>>>> Netflix messages very much. Sorry about that. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> -Jordan >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> On April 9, 2015 at 12:15:02 PM, Camille Fournier ( >> > >>>>>>>> [email protected]) >> > >>>>>>>>>>>> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> Thanks Jordan! I actually asked on Twitter whether Netflix >> > >>>>> had >> > >>>>>>>> anything >> > >>>>>>>>>>>> but >> > >>>>>>>>>>>> didn't get a clear answer. >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> On Thu, Apr 9, 2015 at 11:22 AM, Jordan Zimmerman < >> > >>>>>>>>>>>> [email protected]> wrote: >> > >>>>>>>>>>>> >> > >>>>>>>>>>>>> FYI >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Curator now has a Thrift-based proxy that has all the ZK >> > >>>>> APIs >> > >>>>>>>> exposed >> > >>>>>>>>>> as >> > >>>>>>>>>>>>> well as Curator's added APIs and recipes: >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> http://curator.apache.org/curator-x-rpc/index.html >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> -Jordan >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> On April 8, 2015 at 2:09:33 PM, Camille Fournier ( >> > >>>>>>>> [email protected]) >> > >>>>>>>>>>>>> wrote: >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> All, >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> I've been doing a bit of research on etcd as part of work >> > >>>>> for >> > >>>>>> an >> > >>>>>>>>>>>> upcoming >> > >>>>>>>>>>>>> talk, and it has gotten me thinking about what it would >> > >>>>> take to >> > >>>>>>>> create >> > >>>>>>>>>>>> an >> > >>>>>>>>>>>>> http version of ZK for certain operations. For many >> > >>>>> operations >> > >>>>>>> you >> > >>>>>>>>>> could >> > >>>>>>>>>>>>> put an http proxy in front of ZK to translate, even >> > >>>>>> implementing >> > >>>>>>>> the >> > >>>>>>>>>>>>> "long-poll-style" watch operation to some extent. But it >> > >>>>> would >> > >>>>>> be >> > >>>>>>>> very >> > >>>>>>>>>>>>> hard >> > >>>>>>>>>>>>> to do a temporary node via a proxy without a lot of proxy >> > >>>>>>> failover >> > >>>>>>>>>>>>> complexity. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> As a bit of background, if you want to do an "ephemeral" >> > >>>>> node >> > >>>>>> in >> > >>>>>>>> etcd, >> > >>>>>>>>>>>> you >> > >>>>>>>>>>>>> basically create a key with a TTL. Unless the key is >> > >>>>> updated >> > >>>>>>> with a >> > >>>>>>>>>> new >> > >>>>>>>>>>>>> TTL, the key will auto-expire when the TTL is reached. >> > >>>>> Now, I >> > >>>>>>> have >> > >>>>>>>> a >> > >>>>>>>>>> lot >> > >>>>>>>>>>>>> of >> > >>>>>>>>>>>>> thoughts about this (seems like you have to implement >> > >>>>>> heartbeats >> > >>>>>>>> via >> > >>>>>>>>>>>> http >> > >>>>>>>>>>>>> to truly mimic ephemeral nodes which may not be as simple >> > >>>>> as >> > >>>>>> all >> > >>>>>>>> this >> > >>>>>>>>>>>> http >> > >>>>>>>>>>>>> sounds), but I do think that if there is appetite for >> > >>>>> easy http >> > >>>>>>>> access >> > >>>>>>>>>>>> for >> > >>>>>>>>>>>>> consensus systems we should at least take the time to >> > >>>>> think >> > >>>>>> about >> > >>>>>>>> what >> > >>>>>>>>>>>> it >> > >>>>>>>>>>>>> would take for us to provide this. In particular, I think >> > >>>>> we'd >> > >>>>>>>> have to >> > >>>>>>>>>>>>> make >> > >>>>>>>>>>>>> it possible to create a node with a TTL that is not tied >> > >>>>> to a >> > >>>>>>>>>> particular >> > >>>>>>>>>>>>> session. >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Curious to see if anyone has any thoughts on this. It >> > >>>>> seems >> > >>>>>> like >> > >>>>>>> a >> > >>>>>>>> bit >> > >>>>>>>>>>>> of >> > >>>>>>>>>>>>> a >> > >>>>>>>>>>>>> shame that ZK, which is a good battle-tested system, is >> > >>>>>>> frequently >> > >>>>>>>>>> being >> > >>>>>>>>>>>>> passed-over these days because of the complexity of >> > >>>>> clients, >> > >>>>>> and >> > >>>>>>>> the >> > >>>>>>>>>>>> fact >> > >>>>>>>>>>>>> that it is really pretty damn hard to do a client impl in >> > >>>>>> certain >> > >>>>>>>>>>>>> languages >> > >>>>>>>>>>>>> (Ruby is the notable one I've heard). >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> Best, >> > >>>>>>>>>>>>> C >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>> >> > >>>>>>> >> > >>>>>> >> > >>>>> >> > >>>> >> > >>>> >> > >>> >> > >> >> > >> > >> >> >> >> > >
