Re: Thrift interface
Hi Chris, we were thinking about thrift at one point for marshaling but lack of c binding eliminated it as an option. I don't think it makes sense for us to add dependencies on thrift right now. If there is sufficient user demand we'd definitely consider it. Patrick Chris Smith wrote: I've now got a working stand-alone 'proxy' that acts as a Thrift server and a ZK client. Is this something you guys would want/accept as a contrib module? I appreciate that you already have SWIG and FUSE bindings, but I think the Thrift interface makes more sense in environments where [most] other services are already exposed using Thrift. At present the code is not hugely nice, as it has to convert between Thrift-generated objects and Jute-generated ones; if you did switch to using Thrift RPC for ZK's serialisation layer it'd make this a lot more tidy. Also, it doesn't support watches at the minute, although I can think of several ways this could be done if needed (I don't think we actually need them for what we're going to be using the Thrift interface for). - Chris On Wed, Jul 16, 2008 at 4:47 AM, Benjamin Reed <[EMAIL PROTECTED]> wrote: Hi Chris, We have talked about using Thrift for our serialization layer (I thought we had a Jira opened for it, but I cannot find it). Thrift RPC is a different story. You would lose watch callbacks. I think it would also be hard to support sessions and thus ephemeral nodes. Finally, your client and server throughput would be quite low since clients wouldn't be able to pipeline requests (it appears you can only have one request outstanding at a time) and servers have to dedicate a thread to service a request. ZooKeeper clients are more than simple RPC dispatchers they also do session maintenance and completion processing, so you need something a bit more sophisticated at the client. I assume you want this to get bindings for other languages. We have SWIG and FUSE bindings for this. If you really want thrift, I think there is a combo solution: implement a standalone service, thrift server and zk client, and run it on each ZooKeeper server. ben Chris Smith wrote: Hi, I'm currently thinking about implementing a Thrift interface for ZooKeeper, and wanted to get some opinions on it before I start. I can basically see two possible ways to implement it: * A standalone service which implements both a ZooKeeper client and a Thrift server, and basically acts as a proxy between the two * Integrate the Thrift server with the ZooKeeper server, so that ZK servers themselves handle the Thrift connections I think the former is probably nicer from a development stand-point, but the latter seems better from an end-user stand-point. Has anyone else thought about doing this (or actually done it!), and does anyone have any preferences/comments with regard to the above options? Cheers, Chris Smith
Re: Thrift interface
I've now got a working stand-alone 'proxy' that acts as a Thrift server and a ZK client. Is this something you guys would want/accept as a contrib module? I appreciate that you already have SWIG and FUSE bindings, but I think the Thrift interface makes more sense in environments where [most] other services are already exposed using Thrift. At present the code is not hugely nice, as it has to convert between Thrift-generated objects and Jute-generated ones; if you did switch to using Thrift RPC for ZK's serialisation layer it'd make this a lot more tidy. Also, it doesn't support watches at the minute, although I can think of several ways this could be done if needed (I don't think we actually need them for what we're going to be using the Thrift interface for). - Chris On Wed, Jul 16, 2008 at 4:47 AM, Benjamin Reed <[EMAIL PROTECTED]> wrote: > Hi Chris, > > We have talked about using Thrift for our serialization layer (I thought > we had a Jira opened for it, but I cannot find it). Thrift RPC is a > different story. You would lose watch callbacks. I think it would also > be hard to support sessions and thus ephemeral nodes. Finally, your > client and server throughput would be quite low since clients wouldn't > be able to pipeline requests (it appears you can only have one request > outstanding at a time) and servers have to dedicate a thread to service > a request. > > ZooKeeper clients are more than simple RPC dispatchers they also do > session maintenance and completion processing, so you need something a > bit more sophisticated at the client. > > I assume you want this to get bindings for other languages. We have SWIG > and FUSE bindings for this. > > If you really want thrift, I think there is a combo solution: implement > a standalone service, thrift server and zk client, and run it on each > ZooKeeper server. > > ben > > Chris Smith wrote: >> >> Hi, >> >> I'm currently thinking about implementing a Thrift interface for >> ZooKeeper, and wanted to get some opinions on it before I start. I can >> basically see two possible ways to implement it: >> >> * A standalone service which implements both a ZooKeeper client and a >> Thrift server, and basically acts as a proxy between the two >> * Integrate the Thrift server with the ZooKeeper server, so that ZK >> servers themselves handle the Thrift connections >> >> I think the former is probably nicer from a development stand-point, >> but the latter seems better from an end-user stand-point. Has anyone >> else thought about doing this (or actually done it!), and does anyone >> have any preferences/comments with regard to the above options? >> >> Cheers, >> >> Chris Smith >> > >
Re: Thrift interface
FUSE support is tracked by the following JIRA https://issues.apache.org/jira/browse/ZOOKEEPER-25 a patch is attached to the JIRA containing the feature. It's a big(ger) set of code so I've given it time for review. Should be committed soon and will be part of 3.0. I started working on script support just before we got accepted to Apache, which put that onto the back burner for a bit. We have a design and I implemented a very basic prototype using SWIG that supports perl/python/ruby/php/tcl. I'll be picking scripting support up again once the Apache migration is mostly complete. Patrick Benjamin Reed wrote: Hi Chris, We have talked about using Thrift for our serialization layer (I thought we had a Jira opened for it, but I cannot find it). Thrift RPC is a different story. You would lose watch callbacks. I think it would also be hard to support sessions and thus ephemeral nodes. Finally, your client and server throughput would be quite low since clients wouldn't be able to pipeline requests (it appears you can only have one request outstanding at a time) and servers have to dedicate a thread to service a request. ZooKeeper clients are more than simple RPC dispatchers they also do session maintenance and completion processing, so you need something a bit more sophisticated at the client. I assume you want this to get bindings for other languages. We have SWIG and FUSE bindings for this. If you really want thrift, I think there is a combo solution: implement a standalone service, thrift server and zk client, and run it on each ZooKeeper server. ben Chris Smith wrote: Hi, I'm currently thinking about implementing a Thrift interface for ZooKeeper, and wanted to get some opinions on it before I start. I can basically see two possible ways to implement it: * A standalone service which implements both a ZooKeeper client and a Thrift server, and basically acts as a proxy between the two * Integrate the Thrift server with the ZooKeeper server, so that ZK servers themselves handle the Thrift connections I think the former is probably nicer from a development stand-point, but the latter seems better from an end-user stand-point. Has anyone else thought about doing this (or actually done it!), and does anyone have any preferences/comments with regard to the above options? Cheers, Chris Smith
Re: Thrift interface
Hi Chris, We have talked about using Thrift for our serialization layer (I thought we had a Jira opened for it, but I cannot find it). Thrift RPC is a different story. You would lose watch callbacks. I think it would also be hard to support sessions and thus ephemeral nodes. Finally, your client and server throughput would be quite low since clients wouldn't be able to pipeline requests (it appears you can only have one request outstanding at a time) and servers have to dedicate a thread to service a request. ZooKeeper clients are more than simple RPC dispatchers they also do session maintenance and completion processing, so you need something a bit more sophisticated at the client. I assume you want this to get bindings for other languages. We have SWIG and FUSE bindings for this. If you really want thrift, I think there is a combo solution: implement a standalone service, thrift server and zk client, and run it on each ZooKeeper server. ben Chris Smith wrote: > > Hi, > > I'm currently thinking about implementing a Thrift interface for > ZooKeeper, and wanted to get some opinions on it before I start. I can > basically see two possible ways to implement it: > > * A standalone service which implements both a ZooKeeper client and a > Thrift server, and basically acts as a proxy between the two > * Integrate the Thrift server with the ZooKeeper server, so that ZK > servers themselves handle the Thrift connections > > I think the former is probably nicer from a development stand-point, > but the latter seems better from an end-user stand-point. Has anyone > else thought about doing this (or actually done it!), and does anyone > have any preferences/comments with regard to the above options? > > Cheers, > > Chris Smith >
Thrift interface
Hi, I'm currently thinking about implementing a Thrift interface for ZooKeeper, and wanted to get some opinions on it before I start. I can basically see two possible ways to implement it: * A standalone service which implements both a ZooKeeper client and a Thrift server, and basically acts as a proxy between the two * Integrate the Thrift server with the ZooKeeper server, so that ZK servers themselves handle the Thrift connections I think the former is probably nicer from a development stand-point, but the latter seems better from an end-user stand-point. Has anyone else thought about doing this (or actually done it!), and does anyone have any preferences/comments with regard to the above options? Cheers, Chris Smith