More thoughts on stubbing... It appears that the current extension logic does not eagerly stub out all the methods that could potentially be overridden, which is perhaps not a surprise. Perhaps we only generate the override stubs for methods present at first construction? That would prevent method_missing from working to implement abstract methods from a superclass, but perhaps that's acceptable. Perhaps we treat abstract methods from superclasses as methods that do exist (for purposes of method_missing) but which raise errors?
An example from current logic: class Foo < java.util.ArrayList end No extra methods are generated into the subclass. class Foo < java.util.ArrayList def size; 0; end end Only a stub for the size method is generated. It's also up for debate whether interface impl might want to work this way; if you haven't implemented it by the time you construct the class, should we assume you're never going to implement it? Since current logic *does* allow you to add implementations much later, we couldn't get away with this change. But I'm not sure. - Charlie On Sat, Dec 19, 2009 at 5:22 PM, Charles Oliver Nutter <[email protected]> wrote: > Taking on one aspect at a time...first up, methods from superclasses. > > ** Stubbing methods ** > > When extending an existing class, only non-final public or protected > methods or constructors can be overridden in the subclass. > > With Ruby, we may not have all those overrides in place at the time > the first object is constructed; even more complicated, the object may > implement those in a singleton class after construction. So it seems > like for any *potential* override, we need to create a stub in the > generated class. > > The stub would work like this: > > * Check if there's a Ruby method implemented for the given name; if > so, dispatch to it > * If the superclass method is abstract, dispatch to method_missing > * Dispatch directly to the super method otherwise > > The up side is that most of the usual metaprogramming will work. > method_missing-based impls will work only if the superclass does not > already implement the method, which is perhaps what you'd expect to > happen. The down side is that if there are a lot of public and > protected methods in the superclasses, they'll all get stubs, which > could mean a lot of extra bytecode generated. I'm not sure of a way > around this. > > ** Super dispatching ** > > Dispatching to the superclass method is a tricky problem. Normally, > the "super" keyword in Java (which produces an "invokespecial" > bytecode) can only be called form within the class where you're trying > to super. In our case, since the executing Ruby code may not even live > in a Java class (still being interpreted), we need another way to > dispatch to super. > > I believe we can do this by implementing bridge methods that can do > the super invocation for us. For example, if there's a superclass > method "foo" we want to invoke, we'd generate something like this in > the child class: > > public final bridge returnValue MyClass$foo$super(arg list) { > return super.foo(arg list); > } > > We would then need to detect that a "super" keyword was actually > within a method on a Java subclass, so instead of using the usual > dispatch sequence we would go straight to this bridge method. > > An alternative would be to have a bogus superclass that knows how to > do all these super invocations for us, and then the "super" keyword > would basically work the same as it does now, dispatching through that > intermediate class. > > More to come. > > - Charlie > > On Wed, Dec 16, 2009 at 1:53 AM, Charles Oliver Nutter > <[email protected]> wrote: >> Taking notes on this thread on how some decisions might need to be made. >> >> Interface logic is all pretty easy and working already. If you are >> extending an existing Ruby class or no class (Object) and include >> interfaces, the logic goes like this: >> >> 1. A new class is generated, with the superclass being whatever the >> Java-side class is for the Ruby-side superclass. For example, "class >> Foo; include Runnable; end" would have as its Java superclass >> org.jruby.RubyObject. "class Foo < String; include Runnable; end" >> would have as its Java superclass org.jruby.RubyString. This allows >> extending existing Ruby core types and adding an interface >> impl...something that was not possible before. >> 2. All interfaces included so far are tallied up and their methods >> aggregated by name. All methods get implemented (to appease Java type >> requirements for interface impls) so that they dispatch directly to >> the single named Ruby method. For example, if you have five overloads >> for "valueOf" they'll all dispatch to a single Ruby method named >> "valueOf" or "value_of" (or else to method_missing). >> 3. The resulting "real" class is set up as the backing store for the >> Ruby class, so that all instantiations now actually construct >> instances of that generated Java class. This allows both the Ruby >> world and the Java world to see exactly the same object. >> >> Class extension is more complicated... >> >> When a Ruby class extends a Java class, we ideally want it to still be >> an IRubyObject so it can pass through JRuby calls without a wrapper. >> But this requires implementing all the IRubyObject methods. >> >> I have partially solved this on my branch by having a new >> BasicObjectStub with simple versions of all the IRubyObject methods, >> and a BasicObjectStubGenerator that can create all the method bodies. >> The new class extension logic uses this to "fill in the blanks" so >> that the "real" class can actually extend the one you specified, but >> also implement IRubyObject and pass through JRuby successfully. This >> does increase the size of the class a good amount since the >> IRubyObject interface is pretty large, but those are the breaks. Until >> a miracle happens and we can eliminate IRubyObject in the call path, >> this is the most straightforward way. >> >> The trickier bits of extension are still unimplemented, however: >> >> ** Constructor sequences ** >> >> We would like our "initialize" to actually serve the role of the >> Java-land constructor, but that's tricky; the typical sequence for >> construction is to first allocate the object and then call initialize >> against it with initialization arguments. In Java, you never see the >> allocate phase, and the entire construction process passes through the >> constructor method. We need a way to unify the two construction paths >> in a reasonable way. >> >> We also need our constructor to store off the original Ruby instance >> and RubyClass the object should be associated with, while still >> allowing no-arg construction for reflective construction from Java. >> This may force the issue of moving Ruby to being classloader-global, >> so that any classes generated under a given JRuby instance will know >> automatically which Ruby instance they came from. >> >> ** Super and "special" calls ** >> >> Subclasses need to be able to call protected members of the superclass >> as well as invoke the superclass's constructor. >> >> For protected methods, we currently just set them "accessible" so they >> can be invoked from anywhere. It's not an ideal solution, however, >> since it doesn't work in secured environments and can be costly. We >> *could* generate bridge methods in the subclass for all protected >> methods from the superclass, but that could be too many when you >> consider deep hierarchies with lots of protected methods. We'll need >> to look into what Groovy does and see if we can do that, though I >> suspect they solve it by using setAccessible. >> >> The "special" calls I mean are superclass constructor calls, which are >> only callable from within the child class and potentially only from >> within the child class's constructor. I'm not sure there's any way to >> get around this, and I know that Groovy has had to deal with the same >> challenge since the actual logic for their dispatch may not happen in >> the actual constructor body. We'll need to do some experimentation >> here. >> >> ** Abstract methods ** >> >> Like for interfaces, we will need to automatically implement all >> abstract methods from the parent class to do dynamic calls. If we >> don't, it won't be possible to extend abstract superclasses using >> metaprogramming or method_missing. We may even want to consider doing >> this for *all* overridable methods from the superclass, so that any >> one of them can be dynamically replaced such that both the Java and >> Ruby worlds see it. We'd be the first and only JVM dynamic language to >> take things to this level; even Groovy ends up being mostly stuck with >> the original method definitions when called from Java. >> >> ** Better IRubyObject stubs ** >> >> The current stubs are pretty weak. They don't allow Ruby instance >> variables at all on concrete subclasses. They don't do anything with >> freezing. They punt on a number of things, and we'll probably want to >> improve them. In addition, there's going to be cases in current JRuby >> code where we assume IRubyObject instances always extend >> RubyBasicObject or RubyObject and cast unconditionally. We'll need to >> search those out and replace them with something more agnostic. >> >> ... >> >> That's about all I have at the moment. >> >> On Wed, Dec 16, 2009 at 12:28 AM, Charles Oliver Nutter >> <[email protected]> wrote: >>> I'm working on a new branch of code ("newji" on the kenai repository) >>> that has started to add the following features: >>> >>> * Ruby classes implementing a Java interface will actually be instance >>> of a *real* Java class created on first instantiation. No more >>> two-headed object and chicken/egg initialization hassles. >>> * Ruby classes extending Java classes will *actually* extend those >>> classes and be *real* instances of the resulting class. No more >>> two-headed... >>> >>> This is part of a first effort to unify all the various >>> class-generation schemes we have, from the standard JI (generating >>> little stub classes and wrapping them) to ruby2java (generate real >>> classes ahead-of-time that dynamically invoke via Ruby classes) to >>> become_java! (generate real classes at runtime). The ultimate goal of >>> this is to finally make Ruby's type hierarchy fit correctly into >>> Java's type hierarchy, to bridge the remaining integration gap with >>> languages like Scala and Groovy. >>> >>> This first step is already working very well for interface >>> implementation. Check out this gist: >>> >>> http://gist.github.com/257634 >>> >>> This shows a Foo class that implements Runnable. When instantiated, >>> the instance of Foo is actually *physically* an instance of the >>> generated Java class. There's no wrapping layers at all, and the >>> object can be reflectively manipulated. >>> >>> The extension logic is shown there as well, but its not as >>> full-featured as the existing (extremely complex) logic. However >>> filling in the blanks will not be particularly difficult, and that's >>> what I'll be working on the remainder of this week. Next steps will be >>> to work out how to get "initialize" and Java construction to cooperate >>> nicely and adding back the features that the current class-extension >>> logic does (like implementing abstract methods for you). >>> >>> Once the basic logic for those two halves of Java integration code >>> generation are working as well as before, all the become_java! logic >>> will be added. This will essentially merge become_java into normal >>> Java integration, allowing you to add annotations, specify signatures, >>> and so on. >>> >>> The final, and most complex step, will be to get this all to work >>> ahead of time as well. This will require much more work and probably >>> will not go into this upcoming release. Basically we need a way to >>> determine statically) or through a minimal load process, like >>> ruby2java does) what classes to generate, what classes they extend and >>> interfaces the implement, and all their methods and annotations. It's >>> not so much a code-generation problem as a Ruby DSL/parsing/compiler >>> problem. But it's solvable, like all things. >>> >>> I'm starting to get very excited about this work. A change this major >>> would start to warrant a JRuby 2.0, with one other 2.0 piece being the >>> new interpreter/compiler. I'm starting to think 2010 is the year >>> JRuby's finally going to meet all the loftiest goals we've set for >>> it... >>> >>> - Charlie >>> >> > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
