Andrew Reynhout commented on ZOOKEEPER-661:

I have a WIP patch for the ZK C API, and an alpha level Ruby library that works 
fairly well.  

The following is a diff of the ZK C API patch (excerpted for clarity), for 
example and discussion.  The full patch is basically as below, repeated 
multiple times for multiple completion types, plus some autoconf tweaks to 
enable the ruby-specific code.  I can attach the full patch, but this is much 
less noisy and seems much more useful at this stage.

{code:title=zookeeper.c process_completions() diff}
             case COMPLETION_STAT:
                 LOG_DEBUG(("Calling COMPLETION_STAT for xid=%x 
                 if (rc) {
+                    if (getenv("RUBY_MRI_VERSION") != NULL)
+                        ruby_stat_completion_wrapper(cptr->c.stat_result, rc, 
0, cptr->data);
+                    else
                     cptr->c.stat_result(rc, 0, cptr->data);
                 } else {
                     struct SetDataResponse res;
                     deserialize_SetDataResponse(ia, "reply", &res);
+                    if (getenv("RUBY_MRI_VERSION") != NULL)
+                        ruby_stat_completion_wrapper(cptr->c.stat_result, rc, 
&res.stat, cptr->data);
+                    else
                     cptr->c.stat_result(rc, &res.stat, cptr->data);

And this is the new code to support the Ruby bindings.

{code:title=zookeeper.c ruby support functions}

static struct Stat *stat_dup(struct Stat *stat)
  struct Stat *stat_copy = malloc(sizeof(struct Stat));

  memcpy(stat_copy,stat,sizeof(struct Stat));
  return stat_copy;

typedef struct stat_completion_data {
  stat_completion_t dc;
  int rc;
  struct Stat *stat;
  void *ctx;
  } stat_completion_data;

static void ruby_stat_completion_wrapper_2(struct stat_completion_data *cb)
  cb->dc(cb->rc, cb->stat, cb->ctx);
static void ruby_stat_completion_wrapper(stat_completion_t dc, int rc, struct 
Stat *stat, void *ctx)
  struct stat_completion_data *cb;

  cb = (stat_completion_data *)malloc(sizeof(stat_completion_data));
  cb->dc = dc;
  cb->rc = rc;
  cb->stat = stat ? stat_dup(stat) : 0;
  cb->ctx = ctx ? strdup(ctx) : 0;

  rb_thread_create((void *)ruby_stat_completion_wrapper_2, cb);

*Known Issues*

 - I'm not treating the context pointer properly (should be struct buffer), and 
the strdup() is egregious -- but surprisingly this _works_ under MRI (standard) 
Ruby, and the correct approach does not...!  However:
 - In JRuby, the context data is mangled.  This might be my bug (I can't get 
good data out of ctx->buff and ctx->len...  inspection of the pointer address 
doesn't look like the expected struct layout either..), or possibly a quirk of 
the data structure expectations in JRuby FFI.  WIP, but lowered priority -- a) 
I don't think passing contexts into completions serve much purpose in Ruby, and 
b) I need to do some more reading to figure out next steps.
 - Memory allocations are not freed properly in a few cases -- I think I need 
to allocate the space with a C-Ruby call so that Ruby can GC it when 
appropriate, because the C thread could be long gone by then.  This is small 
but since the Ruby code might be long-running, becomes important.
 - Checking an environment variable to decide whether to wrap the callback in a 
Ruby thread is ugly.  However, JRuby doesn't require thread wrapping due to 
superior JVM threading (but might to resolve the context struct problem, TBD) 
so I can't just test a Ruby global that would be set by both, and I don't want 
to wrap all callbacks just because libruby was included at compile time.  New 
versions of JRuby set a unique global that I could use in conjunction with the 
standard Ruby globals to test "ruby-but-not-jruby", but that would cause 
failures on older JRuby versions.

Does this code (or the vastly improved version of it that will result from the 
ensuing discussions!) look like something that could/should be part of the 
standard ZooKeeper API/contrib distribution?  I'm currently bundling the ZK C 
API diff with the ruby-zookeeper Ruby library, but that isn't ideal.  The 
ruby-zookeeper gem will be released under AL2.0, which matches ZooKeeper 

Thank you for any comments or suggestions.

> Add Ruby bindings
> -----------------
>                 Key: ZOOKEEPER-661
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-661
>             Project: Zookeeper
>          Issue Type: New Feature
>          Components: contrib-bindings
>         Environment: MRI Ruby 1.9
> JRuby 1.4
>            Reporter: Andrew Reynhout
>            Priority: Minor
> Add Ruby bindings to the ZooKeeper distribution.
> Ruby presents special threading difficulties for asynchronous ZK calls (aget, 
> watchers, etc).  It looks like the simplest workaround is to patch the ZK C 
> API.
> Proposed approach will be described in comment.
> Please use this ticket for discussion and suggestions.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to