Re: Another reflection bug; compatibility results
Stuart Ballard wrote: "Edouard G. Parmelan" wrote: Could you add value of serialVersionUID in Japize ? Hey, great idea :) I'll try to get this done by tomorrow (my hacking gets done on the train to and from work). Well, it took a little longer than I thought due to Real World work going crazy (gr) but the versions of Japize and japicompat up at http://stuart.wuffies.net/japi/ now handle this case correctly. Checking svuids needs to be turned on explicitly by passing -v to japicompat (otherwise you'd get a lot of spam if an implementation hadn't concerned itself with this issue). The results for jdk11-vs-kaffe are at http://stuart.wuffies.net/japi/v-results-jdk11-kaffe.txt . 108 classes fail this check, bringing Kaffe up to 248 total errors. Enjoy :) Stuart. PS At some point soon I'll install JDK1.2 and produce the results for kaffe-vs-jdk12. When I do I'll post the URL of the results to the list. If anyone beats me to it, feel free to send me jdk12.japi or jdk13.japi :)
Re: Another reflection bug; compatibility results
Stuart Ballard wrote:
To my knowledge, even though static final constants are part of the
public API, their values are not given by Sun and so they have to be
determined experimentally.
Exactly why I want to report them as part of Japize... to make it
possible to automate testing that Kaffe (and GNU Classpath) has gotten
all the values right.
Could you add value of serialVersionUID in Japize ?
The following code will retreive/compute this value:
ObjectStreamClass stream = ObjectStreamClass.lookup(clz);
if (stream != null) {
// this class is Serializable, register serialVersionUID
emitSerialVersionUID(stream.getSerialVersionUID());
}
Thanks.
--
Edouard G. Parmelan
http://egp.free.fr
Re: Another reflection bug; compatibility results
"Edouard G. Parmelan" wrote:
Could you add value of serialVersionUID in Japize ?
The following code will retreive/compute this value:
ObjectStreamClass stream = ObjectStreamClass.lookup(clz);
if (stream != null) {
// this class is Serializable, register serialVersionUID
emitSerialVersionUID(stream.getSerialVersionUID());
}
Hey, great idea :) I'll try to get this done by tomorrow (my hacking
gets done on the train to and from work).
Stuart.
Re: Another reflection bug; compatibility results
Godmar Back wrote: Have you tried calling 1.2's Class.forName(,false,) and then using getField().getValue() on a final static field? Does doing this invoke clinit? I need to be able to run on 1.1... In 1.1 w/ reflection, you're obviously out of luck since Class.forName will call clinit This is a fairly specific situation that is probably extremely rare. My program attempts to print out a description of the public API of a class in a machine-readable format that can then be compared to other such descriptions. Compile-time constants are part of the public API, other fields' values (even public static final ones) are not. Reflection doesn't let me distinguish between the two cases. Whether or not something is a compile-time constant or not is of no relevance IMO. For instance, a class may define a public static final int K = 5; A compiler is free to inline the 5 where it sees K, but not required to do so. The compiler may as well produce a getfield instruction. To my knowledge, even though static final constants are part of the public API, their values are not given by Sun and so they have to be determined experimentally. - Godmar
Re: Another reflection bug; compatibility results
Okay, try it again now. We now also search superinterfaces if you invoke Class.getMethod() on an interface class. However, since there's no notion of superinterfaces at the bytecode level, there's also no ordering - this means that I believe that the result of such a getMethod call is not and in fact cannot be well-defined. We may or may not match what the JDK returns. I hadn't realized that before. Also, since there no ordering of superinterfaces, there's also no notion of "overriding" - hence both Kaffe and JDK report methods in all superinterfaces. However, getMethod will stop looking at the first match. This leads (in both JDK and Kaffe) to what you call "Non-equality" in your test. However, I believe that none of this should have an adverse impact on the interface compatibility checking you're doing. For instance, you should be able to easily figure out what's what by checking whether a class inherits a method or not - one way to find that out is to check whether getClass().getMethod().getClass() != getClass(). - Godmar
Re: Another reflection bug; compatibility results
Godmar Back wrote: Okay, try it again now. We now also search superinterfaces if you invoke Class.getMethod() on an interface class. Excellent. Kaffe now gives exactly 0 "M" (the NoSuchMethodException) and only about 7 or 8 "!" (the non-equality) on the whole of its class libraries. As you point out, it's probably impossible to avoid all "!"s. Thanks for explaining why I was getting these in the JDK :) Also, since there no ordering of superinterfaces, there's also no notion of "overriding" - hence both Kaffe and JDK report methods in all superinterfaces. However, getMethod will stop looking at the first match. This leads (in both JDK and Kaffe) to what you call "Non-equality" in your test. Hmm, okay. That doesn't seem to be a problem for what I'm doing. Since I only want each distinct method reported once, and I don't care which class it's defined on (I don't even report that information), I'll continue to use the inequality test as a way to ensure I don't see any methods twice. It is safe to assume that two calls to getMethod() with the same arguments *in the same run of Kaffe* (and without the possibility of any classes being gc'd) will always return the same method, right? However, I believe that none of this should have an adverse impact on the interface compatibility checking you're doing. For instance, you should be able to easily figure out what's what by checking whether a class inherits a method or not - one way to find that out is to check whether getClass().getMethod().getClass() != getClass(). Actually, I don't even care whether it's inherited or not - I just didn't want the same method showing up twice. I only bothered to put the check in at all as a workaround for the fact that Kaffe was reporting overridden methods all the way up, and I wanted to make sure I only got the "definitive" one. I'm glad I did or I'd never have realized that the same method could *legitimately* show up twice if it's inherited from two superinterfaces. Thanks for your time :) Stuart.
Re: Another reflection bug; compatibility results
Godmar Back wrote: In 1.1 w/ reflection, you're obviously out of luck since Class.forName will call clinit Blah. I guess I should explore options that look at the bytecode, rather than reflection, then... (the most important reason I need to run on 1.1 is to get reliable docs *for* 1.1) Whether or not something is a compile-time constant or not is of no relevance IMO. For instance, a class may define a public static final int K = 5; A compiler is free to inline the 5 where it sees K, but not required to do so. The compiler may as well produce a getfield instruction. Unfortunately it does make a difference. The only reason primitive constants are part of the public API is because they *can* be inlined (code compiled against the JDK could break on Kaffe if the compiler had inlined the value of something that kaffe defined differently; it doesn't matter that the compiler is free to *not* do this - just the possibility that it can is enough). If the value is not a primitive constant, it couldn't be inlined, so a getfield call would *have* to be issued... and the right value would be "got" on both VMs even if they were different. To my knowledge, even though static final constants are part of the public API, their values are not given by Sun and so they have to be determined experimentally. Exactly why I want to report them as part of Japize... to make it possible to automate testing that Kaffe (and GNU Classpath) has gotten all the values right. Stuart.
Re: Another reflection bug; compatibility results
methods twice. It is safe to assume that two calls to getMethod() with the same arguments *in the same run of Kaffe* (and without the possibility of any classes being gc'd) will always return the same method, right? Right. - Godmar
Re: Another reflection bug; compatibility results
Stuart Ballard wrote: Well, I've made some changes to the class since I "announced" it here a while back and put up a homepage at http://stuart.wuffies.net/japi/ . The reason I mention this is (1) I'd like feedback, and (2) I have the results of running japicompat between jdk11 and kaffe, and there are 425 reported errors, although most of them are duplicated several times because they are errors in java.awt.Component or java.awt.event.AWTEvent and they get reported for every subclass (I'd like to eliminate these duplicates but I can't think of an easy way to do it). I think you can use c.getDeclaredXXX() in place of c.getXXX() to get ride of inherited fields/methods. Raw idea: Did you think it's possible to use a ClassLoader to load jdk1.1 classes and an other to load Kaffe (or GNU Classpath) classes ? If so, you could have a one pass comparison framework. An other solution could be to use a classes manipulations packages as at.dms.classfile or gnu.bytecode. Both use GPL Licences. -- Edouard G. Parmelan http://egp.free.fr
Re: Another reflection bug; compatibility results
"Edouard G. Parmelan" wrote: Stuart Ballard wrote: Well, I've made some changes to the class since I "announced" it here a while back and put up a homepage at http://stuart.wuffies.net/japi/ . The reason I mention this is (1) I'd like feedback, and (2) I have the results of running japicompat between jdk11 and kaffe, and there are 425 reported errors, although most of them are duplicated several times because they are errors in java.awt.Component or java.awt.event.AWTEvent and they get reported for every subclass (I'd like to eliminate these duplicates but I can't think of an easy way to do it). I think you can use c.getDeclaredXXX() in place of c.getXXX() to get ride of inherited fields/methods. Right, but it's legal for an implementation to move a method up to a superclass, so I need to include superclasses' methods in the output listing or you could get bogus errors. The removal of duplicates would have to happen in japicompat.pl. I know how to do this but it's not trivial; basically I need to sort the input so that every class is processed before its subclasses, and then test each error to see whether the same error occurred on the superclass. It's on my todo list (and even on the webpage as of a few minutes ago...) I do want to get what I have commented better before I start adding features like that though; it would take a fairly large rewrite. Raw idea: Did you think it's possible to use a ClassLoader to load jdk1.1 classes and an other to load Kaffe (or GNU Classpath) classes ? Yes, but URLClassLoader is 1.2 only and I'm using 1.1 primarily. I could write a ClassLoader from scratch, but that sounds hard! Also, I'm not sure that I want to know what would happen if I tried to load java.lang.Object from a classloader, and there are issues with native code; I can only imagine what would happen if I tried to load Kaffe's java.lang.Object onto the JDK with its native code intact... If so, you could have a one pass comparison framework. Originally it bothered me that I couldn't provide that, but now I'm beginning to think that this way is better. After all, I can distribute a ~1Mb jdk11.japi file and save all my users downloading the full ~8Mb of JDK1.1 (the gains for 1.2 and 1.3 are presumably bigger still); it also makes it possible to run testing on a machine containing only free software, so long as you have a copy of the japi file generated by JDK1.1. An other solution could be to use a classes manipulations packages as at.dms.classfile or gnu.bytecode. Both use GPL Licences. That's another approach, but Japize works pretty well at this point - most of the work I want to do is on japicompat. The last 3 or 4 major changes I've added to Japize have been Zip support, command line changes, filesystem support, and so on... the actual API outputting has been pretty stable for a whole 3 days of heavy hacking on the rest of it. I'd need a pretty convincing reason to drop Japize and use something else. If you're interested in pursuing one of the other approaches, by all means go for it; if it works within the same framework as Japize and japicompat then I'd certainly be happy to integrate the code. I do have a slight twinge of regret that as things stand now I can't ever produce a jdk10.japi... Thanks for the input :) Stuart.
Re: Another reflection bug; compatibility results
Stuart Ballard wrote: Raw idea: Did you think it's possible to use a ClassLoader to load jdk1.1 classes and an other to load Kaffe (or GNU Classpath) classes ? Yes, but URLClassLoader is 1.2 only and I'm using 1.1 primarily. I could write a ClassLoader from scratch, but that sounds hard! Also, I'm not sure that I want to know what would happen if I tried to load java.lang.Object from a classloader, and there are issues with native code; I can only imagine what would happen if I tried to load Kaffe's java.lang.Object onto the JDK with its native code intact... For now, Japize does not extrate final values (or I missing something) so with method Class.forName(String name, boolean initialize, ClassLoader loader) [yes it's 1.2] clinit is never called, so no native code :-) If so, you could have a one pass comparison framework. Originally it bothered me that I couldn't provide that, but now I'm beginning to think that this way is better. After all, I can distribute a ~1Mb jdk11.japi file and save all my users downloading the full ~8Mb of JDK1.1 (the gains for 1.2 and 1.3 are presumably bigger still); it also makes it possible to run testing on a machine containing only free software, so long as you have a copy of the japi file generated by JDK1.1. You'r right, ~1Mb is better. I do have a slight twinge of regret that as things stand now I can't ever produce a jdk10.japi... I will try to write a small ClassLoader to use with Kaffe to generate jdk10.japi. -- Edouard G. Parmelan http://egp.free.fr
Re: Another reflection bug; compatibility results
"Edouard G. Parmelan" wrote: For now, Japize does not extrate final values (or I missing something) so with method Class.forName(String name, boolean initialize, ClassLoader loader) [yes it's 1.2] clinit is never called, so no native code :-) Actually, Japize does extract all field values that are public, static, final, and of a primitive type or String. The intent is to catch primitive constants, since the values have to be the same for compatibility. However, there are two possible errors here; one is that being public, static and final is not actually sufficient to be a primitive constant; the field also has to be initialized to a compile-time constant value. I'm hoping that it's rare for classes to have public static final primitive-typed variables that are not compile time constants. The other possible error is that null is a compile-time constant, even for non-String objects - Japize would not catch this. This also seems unlikely - why would you declare a public variable if it's going to be a compile-time constant null? java.lang.reflect.Field needs an isPrimitiveConstant() method... However, I think that asking for the value of a field, even a public static final one, is enough to cause clinit to get called... right? :( I do have a slight twinge of regret that as things stand now I can't ever produce a jdk10.japi... I will try to write a small ClassLoader to use with Kaffe to generate jdk10.japi. That would be awesome :) I was thinking about perl hacks on the output of javap... ;) Stuart.
Re: Another reflection bug; compatibility results
Stuart Ballard wrote: However, I think that asking for the value of a field, even a public static final one, is enough to cause clinit to get called... right? :( It depends if it is compile time constant. If yes, then clinit will not be called. From JLS 12.4.1 [when initialization occurs] [...] A non-constant field declared in T (rather than inherited from a superclass or superinterface) is used or assigned. A constant field is one that is (explicitly or implicitly) both final and static, and that is initialized with the value of a compile-time constant expression (§15.27). Java specifies that a reference to a constant field must be resolved at compile time to a copy of the compile-time constant value, so uses of such a field are never active uses. See §13.4.8 for a further discussion. [...] It seems that it is duty of compiler to not emit such calls. I wonder what will happen if getstatic will be called anyway (by creating method by hand). From JLS I understand it is illegal code and should be verified away ?? Quite implementation dependent I'm afraid. Artur
Re: Another reflection bug; compatibility results
Artur Biesiadowski wrote: Stuart Ballard wrote: However, I think that asking for the value of a field, even a public static final one, is enough to cause clinit to get called... right? :( It depends if it is compile time constant. If yes, then clinit will not be called. From JLS 12.4.1 snipped It seems that it is duty of compiler to not emit such calls. I wonder what will happen if getstatic will be called anyway (by creating method by hand). From JLS I understand it is illegal code and should be verified away ?? Quite implementation dependent I'm afraid. Unfortunately, I'm not generating the code by hand, but using Reflection; reflection hadn't even been written when the JLS was created and the reflection javadocs are pretty lame. As far as I can tell the JLS has never been adequately updated to cover it. The reflection specification from the 1.1 docs (at http://java.sun.com/products/jdk/1.1/docs/guide/reflection/) is just a copy of the javadocs in a different format, with some other basic information about security models and things but nothing about the obscure situations that come up, like this one. What I really want is some sort of reflective call that will *tell* me if a field is a compile-time constant (well, technically a primitive constant, which additionally requires that it be static and final) and allow me to get its value *if it is* without calling clinit. There's clearly no such method in the standard reflection classes :( It would be easy to add one to Kaffe, but I want to be writing portable Java code... I may have to resort to one of these bytecode-interpretation libraries eventually :( Stuart.
Re: Another reflection bug; compatibility results
I worked around this by testing whether
mth.equals(cls.getMethod(mth.getName(), mth.getParameterTypes())). This
seems to expose another bug because on numerous occasions I get a
NoSuchMethodException from this check, which shouldn't ever be
possible... the imaginary conversation between my code and the class
goes something like this:
me: Hi, what methods do you support?
class: I support a method called foo with parameter types {bar, baz}.
me: Okay, give me the method called foo with parameter types {bar, baz}.
class: I have no such method!
I'll see if I can produce a simplified test case for both these bugs.
For the second bug, that would be nice.
I checked in a fix for the first problem (reporting overridden methods twice.)
Update libraries/clib/native/Class.c
- Godmar
Re: Another reflection bug; compatibility results
Have you tried calling 1.2's Class.forName(,false,) and then using getField().getValue() on a final static field? Does doing this invoke clinit? If not, we should be able to fix Kaffe accordingly; I bet 10:1 it will call clinit. But, actually, I don't really understand what that should be good for anyway. The only reason that accessing a final/static field is not an active use is so the compiler can inline these constants. I may have to resort to one of these bytecode-interpretation libraries eventually :( I very much recommend it and I recommend JavaClass in particular. Extremely easy to use - all types are named exactly as in the VM spec. You can pretty much code without looking at the API. - Godmar
