Thank you for such a detailed response! Please see mine below.
On Mon, Nov 5, 2012 at 6:52 PM, Marvin Humphrey <[email protected]> wrote:
>
> On Thu, Nov 1, 2012 at 11:08 PM, Logan Bell <[email protected]> wrote:
> > To summarize this is where we're at in regard to binding ruby to Lucy:
> >
> > 1. We have a rakefile clownfish/runtime/ruby that builds builds lemon,
> > clownfish, and CFC.c in the ext folder. This CFC.c contains bindings for
> > Ruby for the following clownfish objects:
> > Clownfish::CFC::Model::Hierarchy
> > and Clownfish::CFC::Binding::Core.
> >
> > 2. We also have a rakefile in clownfish/compiler/ruby that invokes the
> > aforementioned runtime rake file, builds charmonizer and the ruby
> > charmonizer.rb file, and invokes the Clownfish::CFC::Model::Hierarchy
> > and Clownfish::CFC::Binding::Core ruby objects to generate the autogen
> > folder. This rakefile also builds, utilizing Rake::ExtensionTask, the
> > Bind
> > code in the ext folder which contain routines to convert Clownfish
> > charbufs/varrays to ruby objects.
>
> Nice work. :)
>
> FWIW, recent changes to CFCVersion.h have broken the Ruby build -- we
> started
> pound-including "charmony.h" (which isn't there) rather than "stdint.h".
> Thus, the very next step will be to integrate Charmonizer.
>
> That will involve compiling clownfish/compiler/common/charmonizer.c and
> running the resulting executable to produce charmony.h (see the "charmony"
> task in ruby/Rakefile), and then adding some include dirs to feed
> charmony.h
> to CFCVersion.h.
Got it, I will look into this.
>
> > 3. There is also a Rakefile in root/ruby that I believe is compiling
> > Lucy
> > itself. After taking a cursory glance at it appears to do be compiling
> > charmonizer and clownfish too, so I think this build script may need to
> > be
> > revisited and/or refactored at a later point.
> >
> > With my limited scope and understanding, I believe going forward the
> > next
> > logical step would be to start creating a Clownfish::CFC::Binding::Ruby
> > that implements a write_ext_typemap method which in turn would call out
> > to
> > CFCRubyTypeMap_write_ext_typemap.
>
> The purpose of CFCPerlTypeMap_write_ext_typemap() function is to create
> the
> file named "typemap" needed by Perl's XS compiler, xsubpp. I've had
> limited
> experience with the Ruby C API, but to the best of my knowledge there's no
> file which is analogous to "typemap".
>
> The xsubpp compiler uses "typemap" when it compiles "Foo.xs" to "Foo.c"
>
> For instance, consider the following XS function[1]:
>
> lucy_Obj*
> fetch(self, tick)
> lucy_VArray *self;
> uint32_t tick;
> CODE:
> RETVAL = Lucy_VA_Fetch(self, tick);
> OUTPUT: RETVAL
>
> It needs to convert a `lucy_VArray*` from Perl to C on input, and then
> convert
> a `lucy_Obj*` return value into something Perl can deal with on output .
> To
> determine how it does that, it uses the following entries from the typemap
> file:
>
> LUCY_VARRAY_
> $var = (lucy_VArray*)XSBind_sv_to_cfish_obj($arg, LUCY_VARRAY,
> NULL);
>
> ...
>
> LUCY_OBJ_
> $arg = (SV*)Cfish_Obj_To_Host((cfish_Obj*)$var);
>
> Here's the C code which xsubpp spits out; if you look close, you can see
> where
> the typemap entries were used:
>
> XS(XS_Clownfish__VArray_fetch); /* prototype to pass
> -Wmissing-prototypes */
> XS(XS_Clownfish__VArray_fetch)
> {
> dXSARGS;
> if (items != 2) {
> croak_xs_usage(cv, "self, tick");
> }
> {
> lucy_VArray * self =
> (lucy_VArray*)XSBind_sv_to_cfish_obj(ST(0), LUCY_VARRAY, NULL);
> uint32_t tick = (uint32_t)SvUV(ST(1));
> lucy_Obj * RETVAL;
> #line 3813 "lib/Clownfish.xs"
> RETVAL = Lucy_VA_Fetch(self, tick);
> #line 4325 "lib/Clownfish.c"
> ST(0) = (SV*)Cfish_Obj_To_Host((cfish_Obj*)RETVAL);
>
> sv_2mortal(ST(0));
> }
> XSRETURN(1);
> }
>
> To the best of my knowledge, Ruby doesn't have anything like "typemap" --
> because Ruby extensions are written directly in C, and there's no
> intermediate
> stage like a .xs file to be compiled by something like xsubpp.
>
> Therefore, I don't think we will need write_ext_typemap(). HOWEVER...
>
> If you look at CFCPerlTypeMap.h, there are three functions:
>
> /** Return an expression which converts from a Perl scalar to a
> variable
> * of the specified type.
> *
> * @param type A Clownfish::CFC::Model::Type, which will be used to
> select
> * the mapping code.
> * @param xs_var The C name of the Perl scalar from which we are
> * extracting a value.
> */
> char*
> CFCPerlTypeMap_from_perl(struct CFCType *type, const char *xs_var);
>
> /** Return an expression converts from a variable of type `type` to a
> Perl
> * scalar.
> *
> * @param type A Clownfish::CFC::Model::Type, which will be used to
> select
> * the mapping code.
> * @param cf_var The name of the variable from which we are extracting
> a
> * value.
> */
> char*
> CFCPerlTypeMap_to_perl(struct CFCType *type, const char *cf_var);
>
> /** Auto-generate a "typemap" file that adheres to the conventions
> * documented in "perlxs".
> *
> * We generate this file on the fly rather than maintain a static copy
> * because we want an entry for each Clownfish type so that we can
> * differentiate between them when checking arguments. Keeping the
> * entries up-to-date manually as classes come and go would be a pain.
> *
> * @param hierarchy A Clownfish::CFC::Model::Hierarchy.
> */
> void
> CFCPerlTypeMap_write_xs_typemap(struct CFCHierarchy *hierarchy);
>
> We need to adapt the first two routines for Ruby (but not the third).
>
> Marvin Humphrey
>
> [1] This XS code ought to work fine, but it's not what we actually use; I
> took
> some liberties and cleaned it up a bit for the purposes of
> illustration.
Fascinating. I believe I got off course while looking through
Clownfish::CFC::Perl::Build and noticed Clownfish::CFC::Binding::Perl
and the first method it was firing off was write_xs_typemap. Further
with a combination of spelunking I saw the functions you mentioned in
CFCPerlTypeMap.h, which map to the perl equivalent: to_perl/from_perl.
Since they all happened to be associated in one header file I assumed
they all needed to be implemented. Well that assumption was wrong :),
but I feel a bit more enlightened now.
So, with that, correct me if I'm wrong, but it sounds like the
immediate next steps (outside of the charmonizer issue) is the
following:
1. Create a CFCRubyType.h/c with the associated functions
CFCRubyTypeMap_from_ruby/CFCRubyTypeMap_to_ruby
2. Map these in the CFC.c file in compiler/ruby/ext/Clownfish
After these steps, again referring back to
Clownfish::CFC::Perl::Build, it calls out to the binding object the
following methods: write_boot and write_bindings. I'm imagining, this
would be where the real work will begin by creating CFCRuby.c to
implement these calls. So if I'm correct the first step is to start
fashioning write_boot, which I believe is the boot strapping code for
clownfish? Or if not, could you elaborate a bit on what it does?
Thanks again for your detailed and helpful response. Look forward to
more conversations as we try to put this together.
Thanks,
Logan
