Re: Any interest in a gRPC version of ZooKeeper

[tison]

Interesting, at a first glance I thought it is a client wrapper for
polyglot communicating and
has nothing to do with ZOOKEEPER-102. Will take a look at the JIRA since
you attach
a plan of the whole story.

Best,
tison.


Jordan Zimmerman  于2019年11月19日周二 上午5:23写道：

> We can resurrect https://issues.apache.org/jira/browse/ZOOKEEPER-102
>
> > On Nov 18, 2019, at 4:15 PM, Jonathan Wong 
> wrote:
> >
> > I’d be willing to offer some of my spare time towards this effort.
> >
> > Jonathan Wong
> >
> >> On Nov 18, 2019, at 1:04 PM, Jordan Zimmerman <
> jor...@jordanzimmerman.com> wrote:
> >>
> >> A fresh look at APIs is definitely in order. So, this would be a
> potential ZK 4.x.
> >>
> >>> That said, we added things like rest in the past for similar reasons
> and it
> >>> never really took off... Would be a shame to see the same here.
> >>
> >> ZK has had a lot more activity with recent committers. So, maybe this
> time will be different? If at least one other person will sign up to work
> on it, I'll shepherd it.
> >>
> >> -JZ
> >>
> >>> On Nov 18, 2019, at 12:22 PM, Patrick Hunt  wrote:
> >>>
> >>> There are quite a few benefits to using grpc imo. It's come up a few
> times
> >>> where I've been part of the discussion - ala we make it b/w compat it
> would
> >>> be a good move imo. Then the question becomes what else do we fix at
> the
> >>> same time? e.g. make version fields 64 bit rather than 32? etc...
> there are
> >>> a bunch of zk4 such jiras that could be addressed at the same time (and
> >>> likely in a b/w compat way - ie zk3)
> >>>
> >>> That said, we added things like rest in the past for similar reasons
> and it
> >>> never really took off... Would be a shame to see the same here.
> >>>
> >>> Patrick
> >>>
> >>> On Mon, Nov 18, 2019 at 6:48 AM Jordan Zimmerman <
> jor...@jordanzimmerman.com>
> >>> wrote:
> >>>
> > That looks like great work. In order to address the issues, why not
>  build on top of curator (https://curator.apache.org)?
> 
>  (Note: I'm the main author of Curator). I'd definitely try to make
>  something like Curator for gRPC. I'm not sure exactly what that means
> at
>  this point. But, my main goal is to enable non-JVM clients. We have C,
>  python and few others now but they always lag with changes and are
> hard to
>  maintain.
> 
>  -JZ
> 
> >> On Nov 18, 2019, at 9:45 AM, Jörn Franke 
> wrote:
> >>
> >> That looks like great work. In order to address the issues, why not
>  build on top of curator (https://curator.apache.org)?
> >
> > I could support in case question rise with SASL, but I am not sure
> yet
>  if I find the time to actively develop for this unfortunately
> >
> >> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman <
>  jor...@jordanzimmerman.com>:
> >>
> >> Hi Folks,
> >>
> >> I've written a proof of concept implementation of a
> ServerCnxnFactory
>  that implements gRPC. The goal is to make it possible to easily write
>  ZooKeeper clients in non-JVM languages. Using the proof of concept I
> was
>  able to write a Golang client easily. What's the interest level of
>  something like this? Let's discuss if it's worth pursuing. I'd be
> willing
>  to move this from proof of concept to production but I'll need help
> (1 or 2
>  co-developers).
> >>
> >> If you want to try it, I've pushed the Golang client and some
>  instructions here (let me know if you have any issues - I'm a go
> neophyte).
>  Note: "zookeeper/test.go" is the interesting file:
> >>
> >> https://github.com/Randgalt/zkgrpc <
>  https://github.com/Randgalt/zkgrpc>
> >>
> >> Here's the proof of concept on the ZK server side (the interesting
>  files are RpcServerCnxn.java, RpcServerCnxnFactory.java,
>  RpcZooKeeperServer.java and zookeeper.proto):
> >>
> >>
> 
> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc <
> 
> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc>
> 
> >>
> >> Issues:
> >> Writing a client, even with gRPC, will require some work. Sessions
> have
>  to be maintained, watchers have to be maintained, etc.
> >> Currently, Jute is deeply embedded in ZooKeeper. The proof of
> concept
>  has to emulate Jute byte buffers. Ideally, this will be abstracted so
> that
>  only records could be used so that the gRPC connection doesn't have
> to keep
>  marshalling/unmarshalling byte buffers
> >> I don't know enough about the gRPC client/server implementations to
>  know if it will meet the needs of ZooKeeper. Anyone have experience
> here?
> >> I haven't completely thought through how much work it will take to
>  write useful clients. As I've shown with the proof of concept simple
> ZK
>  CRUD db operations work well. I need to spend time writing a recipe
> such as
>  Leader Election to see how much work is 

Re: Any interest in a gRPC version of ZooKeeper

[Jordan Zimmerman]

We can resurrect https://issues.apache.org/jira/browse/ZOOKEEPER-102

> On Nov 18, 2019, at 4:15 PM, Jonathan Wong  wrote:
> 
> I’d be willing to offer some of my spare time towards this effort. 
> 
> Jonathan Wong
> 
>> On Nov 18, 2019, at 1:04 PM, Jordan Zimmerman  
>> wrote:
>> 
>> A fresh look at APIs is definitely in order. So, this would be a potential 
>> ZK 4.x. 
>> 
>>> That said, we added things like rest in the past for similar reasons and it
>>> never really took off... Would be a shame to see the same here.
>> 
>> ZK has had a lot more activity with recent committers. So, maybe this time 
>> will be different? If at least one other person will sign up to work on it, 
>> I'll shepherd it.
>> 
>> -JZ
>> 
>>> On Nov 18, 2019, at 12:22 PM, Patrick Hunt  wrote:
>>> 
>>> There are quite a few benefits to using grpc imo. It's come up a few times
>>> where I've been part of the discussion - ala we make it b/w compat it would
>>> be a good move imo. Then the question becomes what else do we fix at the
>>> same time? e.g. make version fields 64 bit rather than 32? etc... there are
>>> a bunch of zk4 such jiras that could be addressed at the same time (and
>>> likely in a b/w compat way - ie zk3)
>>> 
>>> That said, we added things like rest in the past for similar reasons and it
>>> never really took off... Would be a shame to see the same here.
>>> 
>>> Patrick
>>> 
>>> On Mon, Nov 18, 2019 at 6:48 AM Jordan Zimmerman 
>>> 
>>> wrote:
>>> 
> That looks like great work. In order to address the issues, why not
 build on top of curator (https://curator.apache.org)?
 
 (Note: I'm the main author of Curator). I'd definitely try to make
 something like Curator for gRPC. I'm not sure exactly what that means at
 this point. But, my main goal is to enable non-JVM clients. We have C,
 python and few others now but they always lag with changes and are hard to
 maintain.
 
 -JZ
 
>> On Nov 18, 2019, at 9:45 AM, Jörn Franke  wrote:
>> 
>> That looks like great work. In order to address the issues, why not
 build on top of curator (https://curator.apache.org)?
> 
> I could support in case question rise with SASL, but I am not sure yet
 if I find the time to actively develop for this unfortunately
> 
>> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman <
 jor...@jordanzimmerman.com>:
>> 
>> Hi Folks,
>> 
>> I've written a proof of concept implementation of a ServerCnxnFactory
 that implements gRPC. The goal is to make it possible to easily write
 ZooKeeper clients in non-JVM languages. Using the proof of concept I was
 able to write a Golang client easily. What's the interest level of
 something like this? Let's discuss if it's worth pursuing. I'd be willing
 to move this from proof of concept to production but I'll need help (1 or 2
 co-developers).
>> 
>> If you want to try it, I've pushed the Golang client and some
 instructions here (let me know if you have any issues - I'm a go neophyte).
 Note: "zookeeper/test.go" is the interesting file:
>> 
>> https://github.com/Randgalt/zkgrpc <
 https://github.com/Randgalt/zkgrpc>
>> 
>> Here's the proof of concept on the ZK server side (the interesting
 files are RpcServerCnxn.java, RpcServerCnxnFactory.java,
 RpcZooKeeperServer.java and zookeeper.proto):
>> 
>> 
 https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc <
 https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc>
 
>> 
>> Issues:
>> Writing a client, even with gRPC, will require some work. Sessions have
 to be maintained, watchers have to be maintained, etc.
>> Currently, Jute is deeply embedded in ZooKeeper. The proof of concept
 has to emulate Jute byte buffers. Ideally, this will be abstracted so that
 only records could be used so that the gRPC connection doesn't have to keep
 marshalling/unmarshalling byte buffers
>> I don't know enough about the gRPC client/server implementations to
 know if it will meet the needs of ZooKeeper. Anyone have experience here?
>> I haven't completely thought through how much work it will take to
 write useful clients. As I've shown with the proof of concept simple ZK
 CRUD db operations work well. I need to spend time writing a recipe such as
 Leader Election to see how much work is required.
>> I'm not sure how things like SASL and reconfig would work with gRPC
>> 
>> -Jordan
 
 
>> 

Re: Any interest in a gRPC version of ZooKeeper

[Jonathan Wong]

I’d be willing to offer some of my spare time towards this effort. 

Jonathan Wong

> On Nov 18, 2019, at 1:04 PM, Jordan Zimmerman  
> wrote:
> 
> A fresh look at APIs is definitely in order. So, this would be a potential 
> ZK 4.x. 
> 
>> That said, we added things like rest in the past for similar reasons and it
>> never really took off... Would be a shame to see the same here.
> 
> ZK has had a lot more activity with recent committers. So, maybe this time 
> will be different? If at least one other person will sign up to work on it, 
> I'll shepherd it.
> 
> -JZ
> 
>> On Nov 18, 2019, at 12:22 PM, Patrick Hunt  wrote:
>> 
>> There are quite a few benefits to using grpc imo. It's come up a few times
>> where I've been part of the discussion - ala we make it b/w compat it would
>> be a good move imo. Then the question becomes what else do we fix at the
>> same time? e.g. make version fields 64 bit rather than 32? etc... there are
>> a bunch of zk4 such jiras that could be addressed at the same time (and
>> likely in a b/w compat way - ie zk3)
>> 
>> That said, we added things like rest in the past for similar reasons and it
>> never really took off... Would be a shame to see the same here.
>> 
>> Patrick
>> 
>> On Mon, Nov 18, 2019 at 6:48 AM Jordan Zimmerman 
>> wrote:
>> 
 That looks like great work. In order to address the issues, why not
>>> build on top of curator (https://curator.apache.org)?
>>> 
>>> (Note: I'm the main author of Curator). I'd definitely try to make
>>> something like Curator for gRPC. I'm not sure exactly what that means at
>>> this point. But, my main goal is to enable non-JVM clients. We have C,
>>> python and few others now but they always lag with changes and are hard to
>>> maintain.
>>> 
>>> -JZ
>>> 
> On Nov 18, 2019, at 9:45 AM, Jörn Franke  wrote:
> 
> That looks like great work. In order to address the issues, why not
>>> build on top of curator (https://curator.apache.org)?
 
 I could support in case question rise with SASL, but I am not sure yet
>>> if I find the time to actively develop for this unfortunately
 
> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman <
>>> jor...@jordanzimmerman.com>:
> 
> Hi Folks,
> 
> I've written a proof of concept implementation of a ServerCnxnFactory
>>> that implements gRPC. The goal is to make it possible to easily write
>>> ZooKeeper clients in non-JVM languages. Using the proof of concept I was
>>> able to write a Golang client easily. What's the interest level of
>>> something like this? Let's discuss if it's worth pursuing. I'd be willing
>>> to move this from proof of concept to production but I'll need help (1 or 2
>>> co-developers).
> 
> If you want to try it, I've pushed the Golang client and some
>>> instructions here (let me know if you have any issues - I'm a go neophyte).
>>> Note: "zookeeper/test.go" is the interesting file:
> 
> https://github.com/Randgalt/zkgrpc <
>>> https://github.com/Randgalt/zkgrpc>
> 
> Here's the proof of concept on the ZK server side (the interesting
>>> files are RpcServerCnxn.java, RpcServerCnxnFactory.java,
>>> RpcZooKeeperServer.java and zookeeper.proto):
> 
> 
>>> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc <
>>> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc>
>>> 
> 
> Issues:
> Writing a client, even with gRPC, will require some work. Sessions have
>>> to be maintained, watchers have to be maintained, etc.
> Currently, Jute is deeply embedded in ZooKeeper. The proof of concept
>>> has to emulate Jute byte buffers. Ideally, this will be abstracted so that
>>> only records could be used so that the gRPC connection doesn't have to keep
>>> marshalling/unmarshalling byte buffers
> I don't know enough about the gRPC client/server implementations to
>>> know if it will meet the needs of ZooKeeper. Anyone have experience here?
> I haven't completely thought through how much work it will take to
>>> write useful clients. As I've shown with the proof of concept simple ZK
>>> CRUD db operations work well. I need to spend time writing a recipe such as
>>> Leader Election to see how much work is required.
> I'm not sure how things like SASL and reconfig would work with gRPC
> 
> -Jordan
>>> 
>>> 
> 

Re: Any interest in a gRPC version of ZooKeeper

[Jordan Zimmerman]

A fresh look at APIs is definitely in order. So, this would be a potential ZK 
4.x. 

> That said, we added things like rest in the past for similar reasons and it
> never really took off... Would be a shame to see the same here.

ZK has had a lot more activity with recent committers. So, maybe this time will 
be different? If at least one other person will sign up to work on it, I'll 
shepherd it.

-JZ

> On Nov 18, 2019, at 12:22 PM, Patrick Hunt  wrote:
> 
> There are quite a few benefits to using grpc imo. It's come up a few times
> where I've been part of the discussion - ala we make it b/w compat it would
> be a good move imo. Then the question becomes what else do we fix at the
> same time? e.g. make version fields 64 bit rather than 32? etc... there are
> a bunch of zk4 such jiras that could be addressed at the same time (and
> likely in a b/w compat way - ie zk3)
> 
> That said, we added things like rest in the past for similar reasons and it
> never really took off... Would be a shame to see the same here.
> 
> Patrick
> 
> On Mon, Nov 18, 2019 at 6:48 AM Jordan Zimmerman 
> wrote:
> 
>>> That looks like great work. In order to address the issues, why not
>> build on top of curator (https://curator.apache.org)?
>> 
>> (Note: I'm the main author of Curator). I'd definitely try to make
>> something like Curator for gRPC. I'm not sure exactly what that means at
>> this point. But, my main goal is to enable non-JVM clients. We have C,
>> python and few others now but they always lag with changes and are hard to
>> maintain.
>> 
>> -JZ
>> 
>>> On Nov 18, 2019, at 9:45 AM, Jörn Franke  wrote:
>>> 
>>> That looks like great work. In order to address the issues, why not
>> build on top of curator (https://curator.apache.org)?
>>> 
>>> I could support in case question rise with SASL, but I am not sure yet
>> if I find the time to actively develop for this unfortunately
>>> 
 Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman <
>> jor...@jordanzimmerman.com>:
 
 Hi Folks,
 
 I've written a proof of concept implementation of a ServerCnxnFactory
>> that implements gRPC. The goal is to make it possible to easily write
>> ZooKeeper clients in non-JVM languages. Using the proof of concept I was
>> able to write a Golang client easily. What's the interest level of
>> something like this? Let's discuss if it's worth pursuing. I'd be willing
>> to move this from proof of concept to production but I'll need help (1 or 2
>> co-developers).
 
 If you want to try it, I've pushed the Golang client and some
>> instructions here (let me know if you have any issues - I'm a go neophyte).
>> Note: "zookeeper/test.go" is the interesting file:
 
  https://github.com/Randgalt/zkgrpc <
>> https://github.com/Randgalt/zkgrpc>
 
 Here's the proof of concept on the ZK server side (the interesting
>> files are RpcServerCnxn.java, RpcServerCnxnFactory.java,
>> RpcZooKeeperServer.java and zookeeper.proto):
 
 
>> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc <
>> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc>
>> 
 
 Issues:
 Writing a client, even with gRPC, will require some work. Sessions have
>> to be maintained, watchers have to be maintained, etc.
 Currently, Jute is deeply embedded in ZooKeeper. The proof of concept
>> has to emulate Jute byte buffers. Ideally, this will be abstracted so that
>> only records could be used so that the gRPC connection doesn't have to keep
>> marshalling/unmarshalling byte buffers
 I don't know enough about the gRPC client/server implementations to
>> know if it will meet the needs of ZooKeeper. Anyone have experience here?
 I haven't completely thought through how much work it will take to
>> write useful clients. As I've shown with the proof of concept simple ZK
>> CRUD db operations work well. I need to spend time writing a recipe such as
>> Leader Election to see how much work is required.
 I'm not sure how things like SASL and reconfig would work with gRPC
 
 -Jordan
>> 
>> 

Re: Any interest in a gRPC version of ZooKeeper

[Patrick Hunt]

There are quite a few benefits to using grpc imo. It's come up a few times
where I've been part of the discussion - ala we make it b/w compat it would
be a good move imo. Then the question becomes what else do we fix at the
same time? e.g. make version fields 64 bit rather than 32? etc... there are
a bunch of zk4 such jiras that could be addressed at the same time (and
likely in a b/w compat way - ie zk3)

That said, we added things like rest in the past for similar reasons and it
never really took off... Would be a shame to see the same here.

Patrick

On Mon, Nov 18, 2019 at 6:48 AM Jordan Zimmerman 
wrote:

> > That looks like great work. In order to address the issues, why not
> build on top of curator (https://curator.apache.org)?
>
> (Note: I'm the main author of Curator). I'd definitely try to make
> something like Curator for gRPC. I'm not sure exactly what that means at
> this point. But, my main goal is to enable non-JVM clients. We have C,
> python and few others now but they always lag with changes and are hard to
> maintain.
>
> -JZ
>
> > On Nov 18, 2019, at 9:45 AM, Jörn Franke  wrote:
> >
> > That looks like great work. In order to address the issues, why not
> build on top of curator (https://curator.apache.org)?
> >
> > I could support in case question rise with SASL, but I am not sure yet
> if I find the time to actively develop for this unfortunately
> >
> >> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman <
> jor...@jordanzimmerman.com>:
> >>
> >> Hi Folks,
> >>
> >> I've written a proof of concept implementation of a ServerCnxnFactory
> that implements gRPC. The goal is to make it possible to easily write
> ZooKeeper clients in non-JVM languages. Using the proof of concept I was
> able to write a Golang client easily. What's the interest level of
> something like this? Let's discuss if it's worth pursuing. I'd be willing
> to move this from proof of concept to production but I'll need help (1 or 2
> co-developers).
> >>
> >> If you want to try it, I've pushed the Golang client and some
> instructions here (let me know if you have any issues - I'm a go neophyte).
> Note: "zookeeper/test.go" is the interesting file:
> >>
> >>   https://github.com/Randgalt/zkgrpc <
> https://github.com/Randgalt/zkgrpc>
> >>
> >> Here's the proof of concept on the ZK server side (the interesting
> files are RpcServerCnxn.java, RpcServerCnxnFactory.java,
> RpcZooKeeperServer.java and zookeeper.proto):
> >>
> >>
> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc <
> https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc>
>
> >>
> >> Issues:
> >> Writing a client, even with gRPC, will require some work. Sessions have
> to be maintained, watchers have to be maintained, etc.
> >> Currently, Jute is deeply embedded in ZooKeeper. The proof of concept
> has to emulate Jute byte buffers. Ideally, this will be abstracted so that
> only records could be used so that the gRPC connection doesn't have to keep
> marshalling/unmarshalling byte buffers
> >> I don't know enough about the gRPC client/server implementations to
> know if it will meet the needs of ZooKeeper. Anyone have experience here?
> >> I haven't completely thought through how much work it will take to
> write useful clients. As I've shown with the proof of concept simple ZK
> CRUD db operations work well. I need to spend time writing a recipe such as
> Leader Election to see how much work is required.
> >> I'm not sure how things like SASL and reconfig would work with gRPC
> >>
> >> -Jordan
>
>

Re: Any interest in a gRPC version of ZooKeeper

[Jordan Zimmerman]

> That looks like great work. In order to address the issues, why not build on 
> top of curator (https://curator.apache.org)?

(Note: I'm the main author of Curator). I'd definitely try to make something 
like Curator for gRPC. I'm not sure exactly what that means at this point. But, 
my main goal is to enable non-JVM clients. We have C, python and few others now 
but they always lag with changes and are hard to maintain.

-JZ

> On Nov 18, 2019, at 9:45 AM, Jörn Franke  wrote:
> 
> That looks like great work. In order to address the issues, why not build on 
> top of curator (https://curator.apache.org)?
> 
> I could support in case question rise with SASL, but I am not sure yet if I 
> find the time to actively develop for this unfortunately 
> 
>> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman :
>> 
>> Hi Folks,
>> 
>> I've written a proof of concept implementation of a ServerCnxnFactory that 
>> implements gRPC. The goal is to make it possible to easily write ZooKeeper 
>> clients in non-JVM languages. Using the proof of concept I was able to write 
>> a Golang client easily. What's the interest level of something like this? 
>> Let's discuss if it's worth pursuing. I'd be willing to move this from proof 
>> of concept to production but I'll need help (1 or 2 co-developers).
>> 
>> If you want to try it, I've pushed the Golang client and some instructions 
>> here (let me know if you have any issues - I'm a go neophyte). Note: 
>> "zookeeper/test.go" is the interesting file:
>> 
>>   https://github.com/Randgalt/zkgrpc 
>> 
>> Here's the proof of concept on the ZK server side (the interesting files are 
>> RpcServerCnxn.java, RpcServerCnxnFactory.java, RpcZooKeeperServer.java and 
>> zookeeper.proto):
>> 
>>   https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc 
>> 
>> 
>> Issues:
>> Writing a client, even with gRPC, will require some work. Sessions have to 
>> be maintained, watchers have to be maintained, etc.
>> Currently, Jute is deeply embedded in ZooKeeper. The proof of concept has to 
>> emulate Jute byte buffers. Ideally, this will be abstracted so that only 
>> records could be used so that the gRPC connection doesn't have to keep 
>> marshalling/unmarshalling byte buffers
>> I don't know enough about the gRPC client/server implementations to know if 
>> it will meet the needs of ZooKeeper. Anyone have experience here?
>> I haven't completely thought through how much work it will take to write 
>> useful clients. As I've shown with the proof of concept simple ZK CRUD db 
>> operations work well. I need to spend time writing a recipe such as Leader 
>> Election to see how much work is required.
>> I'm not sure how things like SASL and reconfig would work with gRPC
>> 
>> -Jordan

Re: Any interest in a gRPC version of ZooKeeper

[Jörn Franke]

That looks like great work. In order to address the issues, why not build on 
top of curator (https://curator.apache.org)?

I could support in case question rise with SASL, but I am not sure yet if I 
find the time to actively develop for this unfortunately 

> Am 18.11.2019 um 15:25 schrieb Jordan Zimmerman :
> 
> Hi Folks,
> 
> I've written a proof of concept implementation of a ServerCnxnFactory that 
> implements gRPC. The goal is to make it possible to easily write ZooKeeper 
> clients in non-JVM languages. Using the proof of concept I was able to write 
> a Golang client easily. What's the interest level of something like this? 
> Let's discuss if it's worth pursuing. I'd be willing to move this from proof 
> of concept to production but I'll need help (1 or 2 co-developers).
> 
> If you want to try it, I've pushed the Golang client and some instructions 
> here (let me know if you have any issues - I'm a go neophyte). Note: 
> "zookeeper/test.go" is the interesting file:
> 
>https://github.com/Randgalt/zkgrpc 
> 
> Here's the proof of concept on the ZK server side (the interesting files are 
> RpcServerCnxn.java, RpcServerCnxnFactory.java, RpcZooKeeperServer.java and 
> zookeeper.proto):
> 
>https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc 
> 
> 
> Issues:
> Writing a client, even with gRPC, will require some work. Sessions have to be 
> maintained, watchers have to be maintained, etc.
> Currently, Jute is deeply embedded in ZooKeeper. The proof of concept has to 
> emulate Jute byte buffers. Ideally, this will be abstracted so that only 
> records could be used so that the gRPC connection doesn't have to keep 
> marshalling/unmarshalling byte buffers
> I don't know enough about the gRPC client/server implementations to know if 
> it will meet the needs of ZooKeeper. Anyone have experience here?
> I haven't completely thought through how much work it will take to write 
> useful clients. As I've shown with the proof of concept simple ZK CRUD db 
> operations work well. I need to spend time writing a recipe such as Leader 
> Election to see how much work is required.
> I'm not sure how things like SASL and reconfig would work with gRPC
> 
> -Jordan

Any interest in a gRPC version of ZooKeeper

[Jordan Zimmerman]

Hi Folks,

I've written a proof of concept implementation of a ServerCnxnFactory that 
implements gRPC. The goal is to make it possible to easily write ZooKeeper 
clients in non-JVM languages. Using the proof of concept I was able to write a 
Golang client easily. What's the interest level of something like this? Let's 
discuss if it's worth pursuing. I'd be willing to move this from proof of 
concept to production but I'll need help (1 or 2 co-developers).

If you want to try it, I've pushed the Golang client and some instructions here 
(let me know if you have any issues - I'm a go neophyte). Note: 
"zookeeper/test.go" is the interesting file:

https://github.com/Randgalt/zkgrpc 

Here's the proof of concept on the ZK server side (the interesting files are 
RpcServerCnxn.java, RpcServerCnxnFactory.java, RpcZooKeeperServer.java and 
zookeeper.proto):

https://github.com/apache/zookeeper/compare/master...Randgalt:wip-grpc 
 

Issues:
Writing a client, even with gRPC, will require some work. Sessions have to be 
maintained, watchers have to be maintained, etc.
Currently, Jute is deeply embedded in ZooKeeper. The proof of concept has to 
emulate Jute byte buffers. Ideally, this will be abstracted so that only 
records could be used so that the gRPC connection doesn't have to keep 
marshalling/unmarshalling byte buffers
I don't know enough about the gRPC client/server implementations to know if it 
will meet the needs of ZooKeeper. Anyone have experience here?
I haven't completely thought through how much work it will take to write useful 
clients. As I've shown with the proof of concept simple ZK CRUD db operations 
work well. I need to spend time writing a recipe such as Leader Election to see 
how much work is required.
I'm not sure how things like SASL and reconfig would work with gRPC

-Jordan