Re: Ruby ZooKeeper bindings
Hey, Given that Hadoop is moving over to Avro RPC, and Java, C, and Python bindings for Avro should be usable by the end of January with the 1.3 release (fingers crossed), would it make sense to spend a bit of energy on Ruby bindings for Avro as well? If you're interested, I'd be happy to put you in touch with some folks from Twitter working on the Ruby Avro bindings. Either way, excited to see the work! Later, Jeff On Tue, Dec 22, 2009 at 10:21 AM, wrote: > Sounds good. I'll open a Jira ticket when the patches are ready for > review and discussion. I agree that maintaining the code in multiple > repositories is something to avoid if at all possible. > > Thanks, > Andrew > > > > On Mon, Dec 21, 2009 at 04:31:33PM -0800, Mahadev Konar wrote: > > Hi andrew, > > If the changes to zookeeper source tree are minor and do not change the > > functionality of the system/api's/data structures, I think its better to > > file it as a patch to the main zookeeper source tree. Its would be > > unadvisable to maintain 2 source tree's for the c code or even a patch in > > the ruby bindings that applies to the c source tree at runtime > compilation. > > I would suggest that you can open a jira and propose the changes that you > > intend to make on the zookeeper source tree and the reasoning behind it. > We > > can discuss any alternatives if we need to on the jira. > > > > Thanks > > mahadev > > -- > reynh...@quesera.com >
Re: Ruby ZooKeeper bindings
Sounds good. I'll open a Jira ticket when the patches are ready for review and discussion. I agree that maintaining the code in multiple repositories is something to avoid if at all possible. Thanks, Andrew On Mon, Dec 21, 2009 at 04:31:33PM -0800, Mahadev Konar wrote: > Hi andrew, > If the changes to zookeeper source tree are minor and do not change the > functionality of the system/api's/data structures, I think its better to > file it as a patch to the main zookeeper source tree. Its would be > unadvisable to maintain 2 source tree's for the c code or even a patch in > the ruby bindings that applies to the c source tree at runtime compilation. > I would suggest that you can open a jira and propose the changes that you > intend to make on the zookeeper source tree and the reasoning behind it. We > can discuss any alternatives if we need to on the jira. > > Thanks > mahadev -- reynh...@quesera.com
Re: Ruby ZooKeeper bindings
Hi andrew, If the changes to zookeeper source tree are minor and do not change the functionality of the system/api's/data structures, I think its better to file it as a patch to the main zookeeper source tree. Its would be unadvisable to maintain 2 source tree's for the c code or even a patch in the ruby bindings that applies to the c source tree at runtime compilation. I would suggest that you can open a jira and propose the changes that you intend to make on the zookeeper source tree and the reasoning behind it. We can discuss any alternatives if we need to on the jira. Thanks mahadev On 12/21/09 10:57 AM, "reynh...@quesera.com" wrote: > Hi Mahadev, > > Open sourcing the Ruby bindings is definitely the plan. > > The current code requires patches to zookeeper.c and zk_hashtable.c > to work around some sticky parts of MRI Ruby's threading model. > > I spent some time trying to put all of the Ruby-specific stuff into > a Ruby C extension and keep the ZK code clean, but it looked like it > would involve maintaining a callback dispatch and return table and > other complexities that weren't reasonable for my application. I > ended up just hacking the threading workarounds into the ZK API code, > which is obviously suboptimal. > > I will add a configure option and #ifdefs to protect the standard > build from ruby.h dependencies, e.g.: > > --with-ruby19-lib=RUBY_LIB_PATH > > ...but it will still be a bit messy. If the interest level makes > it worthwhile to take on the complexity of bringing the Ruby stuff > into the official ZK tree, that's great. If not, we can include > the diff (and/or a patched ZK C API tree??) in the ruby-zookeeper > gem distribution (it will be AL2.0 too). > > .. > > Spurious bug report aside, the last few days have been productive. > All API functions are now implemented except ACL get/set, and only > a few remaining bugs: > > - threading issue in async create of an existing path on MRI > - threading issue in async get of a non-existent path on MRI > - intermittent (but frequent) context mangling on JRuby > > I also want to bake a few of the ZK recipes into the Ruby libs, > but that should be straightforward (thanks to the great docs) :) > > The code is in a private github repo presently, but if anyone > wants to look at it in its pre-beta state, let me know and we'll > open it up. > > Thanks, > Andrew > > > > On Sun, Dec 20, 2009 at 08:45:13PM -0800, Mahadev Konar wrote: >> Hi Andrew, >> >> Great to see ruby bindings for zookeeper. Would you be contributing it back >> to the zookeeper source tree? I am sure lots of folks would be interested in >> it. Please post to the list if you have any questions regarding the api or >> questions regarding how to contribute code back to the source tree. >> >> >> Thanks >> mahadev
Ruby ZooKeeper bindings
Hi Mahadev, Open sourcing the Ruby bindings is definitely the plan. The current code requires patches to zookeeper.c and zk_hashtable.c to work around some sticky parts of MRI Ruby's threading model. I spent some time trying to put all of the Ruby-specific stuff into a Ruby C extension and keep the ZK code clean, but it looked like it would involve maintaining a callback dispatch and return table and other complexities that weren't reasonable for my application. I ended up just hacking the threading workarounds into the ZK API code, which is obviously suboptimal. I will add a configure option and #ifdefs to protect the standard build from ruby.h dependencies, e.g.: --with-ruby19-lib=RUBY_LIB_PATH ...but it will still be a bit messy. If the interest level makes it worthwhile to take on the complexity of bringing the Ruby stuff into the official ZK tree, that's great. If not, we can include the diff (and/or a patched ZK C API tree??) in the ruby-zookeeper gem distribution (it will be AL2.0 too). .. Spurious bug report aside, the last few days have been productive. All API functions are now implemented except ACL get/set, and only a few remaining bugs: - threading issue in async create of an existing path on MRI - threading issue in async get of a non-existent path on MRI - intermittent (but frequent) context mangling on JRuby I also want to bake a few of the ZK recipes into the Ruby libs, but that should be straightforward (thanks to the great docs) :) The code is in a private github repo presently, but if anyone wants to look at it in its pre-beta state, let me know and we'll open it up. Thanks, Andrew On Sun, Dec 20, 2009 at 08:45:13PM -0800, Mahadev Konar wrote: > Hi Andrew, > > Great to see ruby bindings for zookeeper. Would you be contributing it back > to the zookeeper source tree? I am sure lots of folks would be interested in > it. Please post to the list if you have any questions regarding the api or > questions regarding how to contribute code back to the source tree. > > > Thanks > mahadev -- reynh...@quesera.com
