Taking on one aspect at a time...first up, methods from superclasses.
** Stubbing methods **
When extending an existing class, only non-final public or protected
methods or constructors can be overridden in the subclass.
With Ruby, we may not have all those overrides in place at the time
the first object is constructed; even more complicated, the object may
implement those in a singleton class after construction. So it seems
like for any *potential* override, we need to create a stub in the
generated class.
The stub would work like this:
* Check if there's a Ruby method implemented for the given name; if
so, dispatch to it
* If the superclass method is abstract, dispatch to method_missing
* Dispatch directly to the super method otherwise
The up side is that most of the usual metaprogramming will work.
method_missing-based impls will work only if the superclass does not
already implement the method, which is perhaps what you'd expect to
happen. The down side is that if there are a lot of public and
protected methods in the superclasses, they'll all get stubs, which
could mean a lot of extra bytecode generated. I'm not sure of a way
around this.
** Super dispatching **
Dispatching to the superclass method is a tricky problem. Normally,
the "super" keyword in Java (which produces an "invokespecial"
bytecode) can only be called form within the class where you're trying
to super. In our case, since the executing Ruby code may not even live
in a Java class (still being interpreted), we need another way to
dispatch to super.
I believe we can do this by implementing bridge methods that can do
the super invocation for us. For example, if there's a superclass
method "foo" we want to invoke, we'd generate something like this in
the child class:
public final bridge returnValue MyClass$foo$super(arg list) {
return super.foo(arg list);
}
We would then need to detect that a "super" keyword was actually
within a method on a Java subclass, so instead of using the usual
dispatch sequence we would go straight to this bridge method.
An alternative would be to have a bogus superclass that knows how to
do all these super invocations for us, and then the "super" keyword
would basically work the same as it does now, dispatching through that
intermediate class.
More to come.
- Charlie
On Wed, Dec 16, 2009 at 1:53 AM, Charles Oliver Nutter
<[email protected]> wrote:
> Taking notes on this thread on how some decisions might need to be made.
>
> Interface logic is all pretty easy and working already. If you are
> extending an existing Ruby class or no class (Object) and include
> interfaces, the logic goes like this:
>
> 1. A new class is generated, with the superclass being whatever the
> Java-side class is for the Ruby-side superclass. For example, "class
> Foo; include Runnable; end" would have as its Java superclass
> org.jruby.RubyObject. "class Foo < String; include Runnable; end"
> would have as its Java superclass org.jruby.RubyString. This allows
> extending existing Ruby core types and adding an interface
> impl...something that was not possible before.
> 2. All interfaces included so far are tallied up and their methods
> aggregated by name. All methods get implemented (to appease Java type
> requirements for interface impls) so that they dispatch directly to
> the single named Ruby method. For example, if you have five overloads
> for "valueOf" they'll all dispatch to a single Ruby method named
> "valueOf" or "value_of" (or else to method_missing).
> 3. The resulting "real" class is set up as the backing store for the
> Ruby class, so that all instantiations now actually construct
> instances of that generated Java class. This allows both the Ruby
> world and the Java world to see exactly the same object.
>
> Class extension is more complicated...
>
> When a Ruby class extends a Java class, we ideally want it to still be
> an IRubyObject so it can pass through JRuby calls without a wrapper.
> But this requires implementing all the IRubyObject methods.
>
> I have partially solved this on my branch by having a new
> BasicObjectStub with simple versions of all the IRubyObject methods,
> and a BasicObjectStubGenerator that can create all the method bodies.
> The new class extension logic uses this to "fill in the blanks" so
> that the "real" class can actually extend the one you specified, but
> also implement IRubyObject and pass through JRuby successfully. This
> does increase the size of the class a good amount since the
> IRubyObject interface is pretty large, but those are the breaks. Until
> a miracle happens and we can eliminate IRubyObject in the call path,
> this is the most straightforward way.
>
> The trickier bits of extension are still unimplemented, however:
>
> ** Constructor sequences **
>
> We would like our "initialize" to actually serve the role of the
> Java-land constructor, but that's tricky; the typical sequence for
> construction is to first allocate the object and then call initialize
> against it with initialization arguments. In Java, you never see the
> allocate phase, and the entire construction process passes through the
> constructor method. We need a way to unify the two construction paths
> in a reasonable way.
>
> We also need our constructor to store off the original Ruby instance
> and RubyClass the object should be associated with, while still
> allowing no-arg construction for reflective construction from Java.
> This may force the issue of moving Ruby to being classloader-global,
> so that any classes generated under a given JRuby instance will know
> automatically which Ruby instance they came from.
>
> ** Super and "special" calls **
>
> Subclasses need to be able to call protected members of the superclass
> as well as invoke the superclass's constructor.
>
> For protected methods, we currently just set them "accessible" so they
> can be invoked from anywhere. It's not an ideal solution, however,
> since it doesn't work in secured environments and can be costly. We
> *could* generate bridge methods in the subclass for all protected
> methods from the superclass, but that could be too many when you
> consider deep hierarchies with lots of protected methods. We'll need
> to look into what Groovy does and see if we can do that, though I
> suspect they solve it by using setAccessible.
>
> The "special" calls I mean are superclass constructor calls, which are
> only callable from within the child class and potentially only from
> within the child class's constructor. I'm not sure there's any way to
> get around this, and I know that Groovy has had to deal with the same
> challenge since the actual logic for their dispatch may not happen in
> the actual constructor body. We'll need to do some experimentation
> here.
>
> ** Abstract methods **
>
> Like for interfaces, we will need to automatically implement all
> abstract methods from the parent class to do dynamic calls. If we
> don't, it won't be possible to extend abstract superclasses using
> metaprogramming or method_missing. We may even want to consider doing
> this for *all* overridable methods from the superclass, so that any
> one of them can be dynamically replaced such that both the Java and
> Ruby worlds see it. We'd be the first and only JVM dynamic language to
> take things to this level; even Groovy ends up being mostly stuck with
> the original method definitions when called from Java.
>
> ** Better IRubyObject stubs **
>
> The current stubs are pretty weak. They don't allow Ruby instance
> variables at all on concrete subclasses. They don't do anything with
> freezing. They punt on a number of things, and we'll probably want to
> improve them. In addition, there's going to be cases in current JRuby
> code where we assume IRubyObject instances always extend
> RubyBasicObject or RubyObject and cast unconditionally. We'll need to
> search those out and replace them with something more agnostic.
>
> ...
>
> That's about all I have at the moment.
>
> On Wed, Dec 16, 2009 at 12:28 AM, Charles Oliver Nutter
> <[email protected]> wrote:
>> I'm working on a new branch of code ("newji" on the kenai repository)
>> that has started to add the following features:
>>
>> * Ruby classes implementing a Java interface will actually be instance
>> of a *real* Java class created on first instantiation. No more
>> two-headed object and chicken/egg initialization hassles.
>> * Ruby classes extending Java classes will *actually* extend those
>> classes and be *real* instances of the resulting class. No more
>> two-headed...
>>
>> This is part of a first effort to unify all the various
>> class-generation schemes we have, from the standard JI (generating
>> little stub classes and wrapping them) to ruby2java (generate real
>> classes ahead-of-time that dynamically invoke via Ruby classes) to
>> become_java! (generate real classes at runtime). The ultimate goal of
>> this is to finally make Ruby's type hierarchy fit correctly into
>> Java's type hierarchy, to bridge the remaining integration gap with
>> languages like Scala and Groovy.
>>
>> This first step is already working very well for interface
>> implementation. Check out this gist:
>>
>> http://gist.github.com/257634
>>
>> This shows a Foo class that implements Runnable. When instantiated,
>> the instance of Foo is actually *physically* an instance of the
>> generated Java class. There's no wrapping layers at all, and the
>> object can be reflectively manipulated.
>>
>> The extension logic is shown there as well, but its not as
>> full-featured as the existing (extremely complex) logic. However
>> filling in the blanks will not be particularly difficult, and that's
>> what I'll be working on the remainder of this week. Next steps will be
>> to work out how to get "initialize" and Java construction to cooperate
>> nicely and adding back the features that the current class-extension
>> logic does (like implementing abstract methods for you).
>>
>> Once the basic logic for those two halves of Java integration code
>> generation are working as well as before, all the become_java! logic
>> will be added. This will essentially merge become_java into normal
>> Java integration, allowing you to add annotations, specify signatures,
>> and so on.
>>
>> The final, and most complex step, will be to get this all to work
>> ahead of time as well. This will require much more work and probably
>> will not go into this upcoming release. Basically we need a way to
>> determine statically) or through a minimal load process, like
>> ruby2java does) what classes to generate, what classes they extend and
>> interfaces the implement, and all their methods and annotations. It's
>> not so much a code-generation problem as a Ruby DSL/parsing/compiler
>> problem. But it's solvable, like all things.
>>
>> I'm starting to get very excited about this work. A change this major
>> would start to warrant a JRuby 2.0, with one other 2.0 piece being the
>> new interpreter/compiler. I'm starting to think 2010 is the year
>> JRuby's finally going to meet all the loftiest goals we've set for
>> it...
>>
>> - Charlie
>>
>
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
