RE: Classpath build process and VM-specific issues
Etienne Gagnon wrote: Jeroen Frijters wrote: Indeed. The goal is to find the optimal solution that would be spec compliant, portable and efficient. and later: I'm not the one nitpicking about pure ISO C portability (I don't use JNI, so I couldn't care less), ... and later: and is of thus ranks lower than my proposal on 2 counts: 1- Efficiency: For a JNI based implementation, maybe, but I'd argue that anyone using JNI doesn't care about performance anyway. You contradict yourself. First you say that the optimal is spec compliant, portable, and efficient. Then you say that you couldn't care less about the spec compliant JNI interface, that portability across JVMs/compilers on a single platform is of no interest, and that efficiency of JNI is not an objective of your proposal. Let me explain. I don't see Classpath as a monolithic entitity. I see Classpath as two parts, Java and native. I use the Java part, I don't use the native part. I care about Classpath as a whole. I think the native part is valuable (because it increases the value of Classpath as a whole, it also increases the value of the Java part). In the particular case we are discussing (memory mapped NIO), I think that JNI is a lousy interface. I think it will prove to be far too slow, therefor I want to use the proper abstractions that allow VMs to optimize these operations. IMHO, RawData is better abstraction than a byte[] for this particular purpose. I just wrote a little benchmark comparing byte[] and RawData for reading bytes (i.e. dereferencing the pointer) and another one that tests alloc/free. It turns out that RawData is actually marginally faster (in both cases). This is on x86 32bit on Sun JDK 1.4.1 using standard JNI. I'm not going to give the code, because that'll just trigger another round of language lawyering, but I'm convinced that C language issues do not have any significant impact on the performance of either solution. For reference, my (partially) optimized implementation of dereferencing the RawData pointer is almost an order of magnitude faster than the JNI implementation running on the Sun JVM (and two orders of magnitude faster than the JNI implementation running on my VM). Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Archie Cobbs wrote: Note: by this logic byte[] is the most portable/generic way to hold VM private data. It places no portability restrictions, only (possibly) performance ones. However, I have yet to hear a convincing argument that proves byte[] is slower than RawData (or whatever) on ALL platforms. IMHO, that is a flawed argument. RawData allows better performance (for VMs that do extra work), byte[] actively prevents that. So the trade-off is *marginally* better portability (byte[]) or the option of better performance (RawData). Not to mention the fact that RawData is a distinct type and opaque from the Java side. The contents byte[] can be manipulated in Java, or a completely wrong byte array could be passed. So I agree with Andrew and Steve, that it is also a better option from a software engineering pov. E.g., take JC as an example. byte[] and RawData containing long both require one unwrap to get the pointer. RawData containing long wastes 4 bytes on 32-bit platforms, but byte[]-length also costs 4 bytes, so size is a wash. But the beauty of RawData is that you can use 32 bit pointers on 32 bit platforms! Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Steven Augart wrote: If we were to do this in the GNU Classpath Java code, then the only solution I see is to use a preprocessor, and expand gpointer to an int or long as appropriate, based upon the standard pointer representation in that platform's usual ABI. That wouldn't work for me. My (single) binary runs on both 32 and 64 bit platforms. That's why I like using an object reference. BTW, I don't actually store a native pointer in an object reference. I replace RawData references with a native pointer type that the CLI support (System.IntPtr). This is a primitive type, but it can also be boxed when you assign it to an object reference. Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Stuart Ballard wrote: 2) Unusual VMs: Things where JNI-centric assumptions don't hold true. For example, IKVM and Jaos(?) don't use JNI at all within Classpath, and their natural pointer type is just a normal object reference. gcj with CNI also falls into this category. I'd like to qualify this. IKVM does have native pointers. This discussion is about RawData use in NIO, in which case we are talking about native memory (presumably in most/all VM types). For general VM state, I wouldn't like to see RawData used, because on IKVM, Jaos and Jikes RVM, you'd typically want the VM state to be in a Java heap object as well. Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
If the RawData type were to be used, would you be able to share a Classpath installation with other Classpath-based virtual machines? --Steve Augart Jeroen Frijters wrote: Steven Augart wrote: If we were to do this in the GNU Classpath Java code, then the only solution I see is to use a preprocessor, and expand gpointer to an int or long as appropriate, based upon the standard pointer representation in that platform's usual ABI. That wouldn't work for me. My (single) binary runs on both 32 and 64 bit platforms. That's why I like using an object reference. BTW, I don't actually store a native pointer in an object reference. I replace RawData references with a native pointer type that the CLI support (System.IntPtr). This is a primitive type, but it can also be boxed when you assign it to an object reference. -- Steven Augart Jikes RVM, a free, open source, Virtual Machine: http://oss.software.ibm.com/jikesrvm ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Steven Augart wrote: If the RawData type were to be used, would you be able to share a Classpath installation with other Classpath-based virtual machines? For some purposes yes. For performance (and some bootstrapping) reasons I compile Classpath code ahead of time to a CLI assembly, so at runtime there wouldn't be any sharing, but the compilation process could use the shared Classpath installation. Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: I am starting to have difficulty understanding Classpath's goals and motivations. When I proposed to add to Classpath some mechanism to allow it to be customized to each VM, I was told that it would be a heresy to encode any VM-specific thing into Classpath, as the vision was: a single Classpath installation would be shared by many VMs on a single system, e.g. common class libraries, common JNI libraries, etc. Now, what you are proposing is that JNI libraries would be compiled specifically for each VM. What have I missed? That I do not speak for Classpath, and that different people have different goals and opinions. Personally, I don't see much point in common class libraries, common JNI libraries. It might be neat, but: (a) it complicates getting good performance, and (b) coordinating Classpath and VM releses would be too difficult. Yes, it's more elegant to install a single copy of Classpath, but GCJ is not going to use the system installation of Classpath, and I wouldn't expect others to do so either. -- --Per Bothner [EMAIL PROTECTED] http://per.bothner.com/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Grzegorz B. Prokopski wrote:
On (07/04/04 13:47), Jeroen Frijters wrote:
Andrew Haley wrote:
This seems to be identical to my proposal.
I no longer understand what we're arguing about...
I know. I thought we were still trying to convince Etienne :-)
I thought the goal was finding the optimal solution that would be spec
compliant, portable and efficient.
Indeed. The goal is to find the optimal solution that would be spec
compliant, portable and efficient. Since I obviously believe that my
proposal is better than the byte[] proposal, I would like to convice
Etienne (and you) of this. I fail to see how you can take issue with
that.
And I still haven't seen good example from Andrew about how native
part of his opaque class would look like and what burden would it
have.
I don't feel like writing real JNI code, but here is the general idea:
jobject WrapPointer(JNIEnv* env, void* p)
{
#if PLATFORM_HAS_32_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData32);
jmethodID mid = env-GetMethodID(pclass, init, (I)V);
return env-NewObject(pclass, mid, p);
#else if PLATFORM_HAS_64_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData64);
jmethodID mid = env-GetMethodID(pclass, init, (J)V);
return env-NewObject(pclass, mid, p);
#else if PLATFORM_HAS_128_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData128);
jmethodID mid = env-GetMethodID(pclass, init, (JJ)V);
jlong hi = HI_FROM_PTR(p);
jlong lo = LO_FROM_PTR(p);
return env-NewObject(pclass, mid, hi, lo);
#else
#error unsupported
#endif
}
void* UnwrapPointer(JNIEnv* env, jobject pobj)
{
#if PLATFORM_HAS_32_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData32);
jfieldID fid = env-GetFieldID(pclass, ptr, I);
return (void*)env-GetIntField(pobj, fid);
#else if PLATFORM_HAS_64_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData64);
jfieldID fid = env-GetFieldID(pclass, ptr, J);
return (void*)env-GetLongField(pobj, fid);
#else if PLATFORM_HAS_128_BIT_NATIVE_POINTER
jclass pclass = env-FindClass(RawData128);
jfieldID fidHi = env-GetFieldID(pclass, ptrHi, J);
jfieldID fidLo = env-GetFieldID(pclass, ptrLo, J);
jlong hi = env-GetLongField(pobj, fidLo);
jlong lo = env-GetLongField(pobj, fidLo);
return PTR_FROM_HI_LO(hi, lo);
#else
#error unsupported
#endif
}
I a real implementation you'd obviously cache the jclass and
jmethodID/jfieldID. I don't think this is less efficient or less
portable than using a byte[]. It only requires a (tiny) bit of extra
code to support different pointer sizes.
About the other solution - a comment. Why again use
something magical?
A magica class in this case called RawData.
There is nothing magical about it, but it does allow JITs (on VMs that
don't use JNI to interact with the RawData pointers) to optimize access.
Regards,
Jeroen
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: Andrew Haley wrote: Okay, but ANS specifically does not allow you to do this subtraction. Also, there is no guarantee that every pointer is representable as a ptrdiff_t. (6.5.6 Para 9, if you're interested) The point is: if your platform is one that does *not* have 8-bit chars (!!!), http://www.36bit.org/ for platforms where CHAR_BITS is 9. And for some non-related fun, http://www-1.ibm.com/servers/enable/site/porting/iseries/overview/faq_misc.html for a platform with 128 bit pointers. cheers, dalibor topic ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Etienne Gagnon wrote:
Jeroen Frijters wrote:
Indeed. The goal is to find the optimal solution that would be spec
compliant, portable and efficient. Since I obviously believe that my
proposal is better than the byte[] proposal, I would like to convice
Etienne (and you) of this. I fail to see how you can take issue with
that.
Fair.
I don't feel like writing real JNI code, but here is the
general idea:
So, you don't have a real proposal... We need to compare *real* JNI
code to real JNI code, otherwise, we'll be arguing about
misconceptions.
I still think that my proposal is better than the one you
sketched below.
jobject WrapPointer(JNIEnv* env, void* p)
{
#if PLATFORM_HAS_32_BIT_NATIVE_POINTER
For one thing, my proposal does not need any specific #ifdef
per pointer size.
Absolutely, that is a big advantage of the byte[] approach, but IMHO it
optimizes for the wrong scenario. I (and I suspect most others) don't
care about pointer sizes other than 32/64 bit.
jclass pclass = env-FindClass(RawData32);
jmethodID mid = env-GetMethodID(pclass, init, (I)V);
That's supposed to be faster than my proposal
This is only done once. But you are missing my point about efficiency.
The JNI approach isn't about performance, it is going to be slow, no
matter what. I just want the option the make my implementation fast. The
byte[] approach doesn't give me this option. It forces me into
suboptimal implementation.
return env-NewObject(pclass, mid, p);
** Bzzzt *** Please try again...
Now that my proposal has been criticised to death on the smallest
nitpicks of pure ISO C portability, let me comment on the portability
]of your code.
I'm not the one nitpicking about pure ISO C portability (I don't use
JNI, so I couldn't care less), but we could use the exact same trick you
used in your JBYTE_IS_NOT_SIGNED_CHAR versions of wrap/unwrap. So this
is a non-issue.
So, no, your proposal is NOT portable,
It is portable (can be made portable by someone who cares, i.e. not me).
and is of thus ranks lower than my proposal on 2 counts:
1- Efficiency:
For a JNI based implementation, maybe, but I'd argue that anyone using
JNI doesn't care about performance anyway.
Regards,
Jeroen
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Jeroen Frijters wrote:
Indeed. The goal is to find the optimal solution that would be spec
compliant, portable and efficient. Since I obviously believe that my
proposal is better than the byte[] proposal, I would like to convice
Etienne (and you) of this. I fail to see how you can take issue with
that.
Fair.
I don't feel like writing real JNI code, but here is the general idea:
So, you don't have a real proposal... We need to compare *real* JNI
code to real JNI code, otherwise, we'll be arguing about misconceptions.
I still think that my proposal is better than the one you sketched below.
jobject WrapPointer(JNIEnv* env, void* p)
{
#if PLATFORM_HAS_32_BIT_NATIVE_POINTER
For one thing, my proposal does not need any specific #ifdef per pointer
size.
jclass pclass = env-FindClass(RawData32);
jmethodID mid = env-GetMethodID(pclass, init, (I)V);
That's supposed to be faster than my proposal
return env-NewObject(pclass, mid, p);
** Bzzzt *** Please try again...
Now that my proposal has been criticised to death on the smallest
nitpicks of pure ISO C portability, let me comment on the portability
]of your code.
The ISO C standard says:
[#6] Any pointer type may be converted to an integer type;
the result is implementation-defined. If the result cannot
be represented in the integer type, the behavior is
undefined. The result need not be in the range of values of |
any integer type.
So, the conversion of p to jlong or jint yields a *compiler-specific*
value; ISO C does not guarantees anything about the actual value resulting
from the cast. In other words, a compiler would be allowed to change
the bits of the pointer value when converting.
Now, *my* proposal, when jbyte == signed char, does not convert
the pointer to an integer, which is something non-portable across
*compilers* on a same platform. [Remember: the *main* objective of
JNI is portability across VMs (and compilers) on the same platform,
because you shouldn't have to recompile the library to work with
another VM].
So, no, your proposal is NOT portable, and is of thus ranks lower than
my proposal on 2 counts:
1- Efficiency:
- my approach needs no call to FindClass GetMethodID
- wrapping the pointer causes no execution of Java byte code,
but in your case, the constructor init must be executed.
2- Portability:
- Your code uses an implementation-defined (i.e. compiler-specific)
conversion; and cannot guarantee that the pointer will be actually
restored on unwrapping.
FYI: Annex K portability Issues (informative) of the ISO C specification
specifically points out the following as being implementation defined:
K.3.7 Arrays and pointers
[#1]
-- The result of converting a pointer to an integer or
vice versa (6.3.2.3).
Your proposal fails portability on this account.
I a real implementation you'd obviously cache the jclass and
jmethodID/jfieldID. I don't think this is less efficient or less
portable than using a byte[].
Caching jclass/jmethodID/jfieldID will not solve all the problems;
you still call the constructor init. Not only will this cause
execution of Java code (which is not required in my proposal), but
on non-optimizing JVMs, it will cause execution of 3 constructors(!!):
1- RawData32.init
2- RawData.init
3- Object.init
It only requires a (tiny) bit of extra
code to support different pointer sizes.
Mine requires none.
Also, it is highly probable that we'll get to 128-bit platforms one
day: just imagine if you could use a secure hash, like SHA-1 as
hashcode to store data in a virtual memory implementation with 128-bit
addresses! :-) I *am* serious.
There is nothing magical about it, but it does allow JITs (on VMs that
don't use JNI to interact with the RawData pointers) to optimize access.
Your proposal is slower for JNI-compliant VMs, and it is not portable.
You see: this is why I wanted you to show me JNI code, as I have thought
about the problem for quite a while before settling with the idea of
byte arrays.
Etienne
--
Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/
SableVM: http://www.sablevm.org/
SableCC: http://www.sablecc.org/
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Jeroen Frijters writes: return env-NewObject(pclass, mid, p); ** Bzzzt *** Please try again... Now that my proposal has been criticised to death on the smallest nitpicks of pure ISO C portability, let me comment on the portability of your code. That wasn't my point in nitpicking. What I was attempting was showing that to prefer one approach to another soley on the grounds of pure ISO compatibility is probably a mistake. Every approach so far has lacked correctness in some way. Strong typing for legibility and reliability is IMO for more important. Here's a real example from SWT: public static final synchronized native int g_signal_handlers_unblock_matched(long /*PTR*/ instance, int mask, int signal_id, int detail, long /*PTR*/ closure, long /*PTR*/ func, long /*PTR*/ data); Here it is with passing the pointers as byte arrays: public static final synchronized native int g_signal_handlers_unblock_matched(byte[] instance, int mask, int signal_id, int detail, byte[] closure, byte[] func, byte[] data); And here it is with named opaque types: public static final synchronized native int g_signal_handlers_unblock_matched(gpointer instance, int mask, int signal_id, int detail, GClosurePointer closure, gpointer func, gpointer data); Just for reference, here is the corresponding C declaration: guint g_signal_handlers_unblock_matched (gpointer instance, GSignalMatchType mask, guint signal_id, GQuark detail, GClosure *closure, gpointer func, gpointer data); Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Now that my proposal has been criticised to death on the smallest nitpicks of pure ISO C portability, let me comment on the portability ]of your code. The ISO C standard says: [#6] Any pointer type may be converted to an integer type; the result is implementation-defined. If the result cannot be represented in the integer type, the behavior is undefined. The result need not be in the range of values of | any integer type. So, the conversion of p to jlong or jint yields a *compiler-specific* value; ISO C does not guarantees anything about the actual value resulting from the cast. In other words, a compiler would be allowed to change the bits of the pointer value when converting. Exactly: the only reliable way I can see to do this is to use memcpy() from the big pointer to a byte array. But, compared to the overhead of a JNI call, that is no big deal. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Jeroen Frijters wrote: Indeed. The goal is to find the optimal solution that would be spec compliant, portable and efficient. and later: I'm not the one nitpicking about pure ISO C portability (I don't use JNI, so I couldn't care less), ... and later: and is of thus ranks lower than my proposal on 2 counts: 1- Efficiency: For a JNI based implementation, maybe, but I'd argue that anyone using JNI doesn't care about performance anyway. You contradict yourself. First you say that the optimal is spec compliant, portable, and efficient. Then you say that you couldn't care less about the spec compliant JNI interface, that portability across JVMs/compilers on a single platform is of no interest, and that efficiency of JNI is not an objective of your proposal. OK. So, it is clear that I am wasting my time, here. I now clearly understand that the main motivation is for Classpath to use less portable approaches when they make CNI faster, as the performance of CNI and the other non-spec compliant interfaces is the main objective. Fine. I'll devote my time elsewhere. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Jeroen Frijters wrote: Indeed. The goal is to find the optimal solution that would be spec compliant, portable and efficient. and later: I'm not the one nitpicking about pure ISO C portability (I don't use JNI, so I couldn't care less), ... and later: and is of thus ranks lower than my proposal on 2 counts: 1- Efficiency: For a JNI based implementation, maybe, but I'd argue that anyone using JNI doesn't care about performance anyway. You contradict yourself. First you say that the optimal is spec compliant, portable, and efficient. Then you say that you couldn't care less about the spec compliant JNI interface, that portability across JVMs/compilers on a single platform is of no interest, and that efficiency of JNI is not an objective of your proposal. OK. So, it is clear that I am wasting my time, here. I now clearly understand that the main motivation is for Classpath to use less portable approaches when they make CNI faster, as the performance of CNI and the other non-spec compliant interfaces is the main objective. Come on now. We need to find a clean way to do this, and although it may seem like I'm being overly pushy, my main concern is to do what's right for everyone who uses JNI in Classpath, long term. Perhaps we need to define what we're really aiming at. I would have thought: 1. Correctness (well-defined on the platforms we care about.) 2. Portability (to the platforms we care about.) 3. Maintainability/Readability of code. 4. Efficiency. Does anyone really disagree with these priorities? Okay, perhaps we need to agree what platforms we care about. Fine. I'll devote my time elsewhere. That's a shame. I wouldn't like to think I was responsible in some way for pushing you (and SableVM) away form the Classpath list. It seems to me that we've been focussing on small details without agreeing on our goals. The goals of IKVM and SableVM are different, but I feel sure that a compromise is possible. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: the main motivation is for Classpath to use less portable approaches when they make CNI faster, as the performance of CNI and the other non-spec compliant interfaces is the main objective. Jikes RVM uses JNI and is unlikely to ever implement CNI. There are other classpath-using VMs that implement JNI. Obviously, most of those of us who work on a specific VM are thinking about what proposals will mean for our particular VM. For the Jikes RVM crowd, Dave Grove, Mike Hind, and I will object loudly to anything that really messes up JNI, and I'm sure the other JNI users will speak up for their VMs. -- Steven Augart Jikes RVM, a free, open source, Virtual Machine: http://oss.software.ibm.com/jikesrvm ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Perhaps we need to define what we're really aiming at. I would have thought: 1. Correctness (well-defined on the platforms we care about.) 2. Portability (to the platforms we care about.) 3. Maintainability/Readability of code. 4. Efficiency. Does anyone really disagree with these priorities? Not me. Okay, perhaps we need to agree what platforms we care about. Here are my votes... - We don't care about platforms that don't have 8 bit bytes or where typeof(jchar) != signed char. - We don't care about platforms where 'jlong' is insufficient to hold VM private data. E.g., on a platform with 128 bit pointers, then the VM must use a hash table or something to derive the actual pointer (or whatever it needs). I.e., we deem 64 bits enough to uniquely describe all unique VM-private data structures. - We don't use ``because it runs faster on my VM'' as evidence of being more efficient; It must run faster on ALL VM's for it to mean anything. - When there are non-obvious tradeoffs in efficiency (i.e., it's faster on some VMs, slower on others, or otherwise muddy) then we choose the option which is the most portable and/or generic. - Requiring the VM to specially recognize and not trace a RawData object pointer type decreases portability (having a RawData type that merely is a normal object holding a long does not). Note: by this logic byte[] is the most portable/generic way to hold VM private data. It places no portability restrictions, only (possibly) performance ones. However, I have yet to hear a convincing argument that proves byte[] is slower than RawData (or whatever) on ALL platforms. E.g., take JC as an example. byte[] and RawData containing long both require one unwrap to get the pointer. RawData containing long wastes 4 bytes on 32-bit platforms, but byte[]-length also costs 4 bytes, so size is a wash. byte[] is portable to 128 bit platforms while RawData containing long is not (sure, probably won't happen in my lifetime, but... ). Finally, RawData as opaque pointer doesn't require any unwraping but does hit GC cycle time severely as every object has to be check for being in this special RawData class. So for JC byte[] is best. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Strong typing for legibility and reliability is IMO for more important. I strongly agree with this, on software engineering grounds. And here it is with named opaque types: public static final synchronized native int g_signal_handlers_unblock_matched(gpointer instance, int mask, int signal_id, int detail, GClosurePointer closure, gpointer func, gpointer data); These are great examples from SWT. If we were to do this in the GNU Classpath Java code, then the only solution I see is to use a preprocessor, and expand gpointer to an int or long as appropriate, based upon the standard pointer representation in that platform's usual ABI. I have no idea what we will do when 128-bit machines become available. Probably by that time Sun will have added a type called long long or long128, and we'll expand gpointer to that on the 128-bit machines. Like Andrew, I am uncomfortable with using byte[] to represent a raw pointer. I also see Etienne's point of view that a class to encapsulate the data is pretty heavy-weight, and that only a few VMs would implement special handling for the RawPointer type. Worse, a VM that did handle it specially would not binary-compatible with one that treated it as just another class. Ick; there are no good solutions. I hate to introduce the notion of a preprocessor. Oh well, I guess I just did. -- Steven Augart Jikes RVM, a free, open source, Virtual Machine: http://oss.software.ibm.com/jikesrvm ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Archie Cobbs writes: Note: by this logic byte[] is the most portable/generic way to hold VM private data. It places no portability restrictions, only (possibly) performance ones. However, I have yet to hear a convincing argument that proves byte[] is slower than RawData (or whatever) on ALL platforms. E.g., take JC as an example. byte[] and RawData containing long both require one unwrap to get the pointer. RawData containing long wastes 4 bytes on 32-bit platforms, but byte[]-length also costs 4 bytes, so size is a wash. byte[] is portable to 128 bit platforms while RawData containing long is not (sure, probably won't happen in my lifetime, but... ). Am I the only one who cares about type checking? :-) Finally, RawData as opaque pointer doesn't require any unwraping but does hit GC cycle time severely as every object has to be check for being in this special RawData class. So for JC byte[] is best. We get away with this in GCJ because a pointer outside the heap is ignored. I would have thought that many other GCs do this too. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Archie Cobbs wrote: Okay, perhaps we need to agree what platforms we care about. Here are my votes... - We don't care about platforms that don't have 8 bit bytes or where typeof(jchar) != signed char. - We don't care about platforms where 'jlong' is insufficient to hold VM private data. E.g., on a platform with 128 bit pointers, then the VM must use a hash table or something to derive the actual pointer (or whatever it needs). I.e., we deem 64 bits enough to uniquely describe all unique VM-private data structures. I'd like to see some qualification of don't care. Seems to me that there are multiple levels of caring: 1) Normal VMs on Normal platforms. Use JNI, don't have any weirdnesses about the sizes or types of JNI data types. 2) Unusual VMs: Things where JNI-centric assumptions don't hold true. For example, IKVM and Jaos(?) don't use JNI at all within Classpath, and their natural pointer type is just a normal object reference. gcj with CNI also falls into this category. 3) Unusual platforms: Any real and existing platforms to which a Classpath port is a real possibility, but violate one of the two criteria above, or other normally reasonable assumptions about the behavior of the CPU, OS or C compiler. 4) Non-free VMs: generally JDK derivatives that we have no control over and are unlikely to change to adopt our practices, or even optimize for them. 5) Nonexistent VMs or nonexistent platforms: VMs or platforms that are theoretically possible or theoretically permitted by the standards, but which never actually occur in any real platform now or in the forseeable future, or will never be able to run even a minimal JVM (Z80 or 6502 chips probably fall into the latter category?) Seems to me that whatever is adopted needs to take into account the existence of types 2 and 3, but it's okay to optimize for performance on type 1. I'm not sure that we should invest *any* time worrying about types 4 or 5 (although I can conceive of situations where portability to type 4 is desirable, it's certainly not the general case *within Classpath*) The byte array solution has negative implications for, at least, type 2: it's generally impossible to round-trip between an object reference and a byte[] without a hashtable and serious inefficiency. The opaque RawData class may have negative performance implications for type 1 (does everyone agree on this? hard to tell), which makes it undesirable, but it is at least usable on all types (including the ones we don't care about). Nobody yet in this thread has talked about the possibility of just using Object references, unless I've missed it. The advantage would be that you can store whatever type you like - byte[] or an opaque RawData class or a native pointer or anything - the disadvantage would be that the native code couldn't be shared when different things were stored there. This solution would help type 2 systems (which generally don't share the JNI native code anyway) while still allowing the JNI default code to use byte[] (or for the flamewar to continue over what it should use ;) ) I think that it's important to draw a distinction between these various types of VM, because they have different needs, different performance characteristics, and in some cases (like the byte[] solution on type 2) can't use one particular approach *at all* - or at least, not without severe, orders-of-magnitude-worse performance problems. Stuart. -- Stuart Ballard, Senior Web Developer NetReach, Inc. (215) 283-2300, ext. 126 http://www.netreach.com/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Finally, RawData as opaque pointer doesn't require any unwraping but does hit GC cycle time severely as every object has to be check for being in this special RawData class. So for JC byte[] is best. We get away with this in GCJ because a pointer outside the heap is ignored. I would have thought that many other GCs do this too. They probably do, but JC doesn't right now.. it assumes objects live either in the heap or in per-loader memory, and in the latter case, it assumes the object must be a Class instance. Relaxing these assumptions wouldn't cost that much, but the point is of course that there would be a non-zero impact so the change wouldn't be free. In the future JC may support stack-allocated objects, in which case it would then know to ignore any reference pointing to a weird place, so this will probably change. As I've stated before, there are lots of different tradeoffs with all the VM's out there and IMHO Classpath should not try to assume too much about what's efficient, how things should be done, etc. but rather specific a fairly generic way of doing things and let the VM implementors do their VM-specific tweaks, as they are surely going to have to do anyway. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Andrew Haley wrote: Yes, but it's not a question of whether the type jbyte is the same size as a character type, but whether it is treated in the same way as a character by the compiler. Hi Andrew. Please look carefully at my proposal (sent minutes ago), and tell me if it still contains non-portable bits. I provide in it a piece of C code that will generate warnings if jbyte is not treated as signed char by the compiler, if I am right. [Look at the test() function]. OK, done that. Cheers, Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Andrew Haley wrote:
jbyte must have a single platform-specific definition, as all
JVMs on that platform should be able to execute the same JNI
library code (no recompilation required).
I didn't know that. Is that requirement documented anywhere? I can't
see how you'd do it on a machine with 32-bit and 64-bit modes, for
example.
Platform = Machine + OS. I don't have any reference, but I believe that
Etienne is right in saying that the same library should be usuable with
all JVMs on a specific platform.
Clearly the highest performance comes from using a naked long.
No, the highest performance comes from using a platform specific
pointer. I think that for a JIT it is easier to replace a typed object
reference with a native pointer then a long. That's why I'm very much in
favor of using RawData. It can even be abstract, that way the JNI layer
can allocate sub types that contain the appropriate number of bits to
contain a native pointer.
For example:
public abstract class RawData {}
public final class RawData32 extends RawData
{
private int pointer;
}
public final class RawData64 extends RawData
{
private long pointer;
}
All the Java code would ever see are RawData references, but the JNI
layer knows that all RawData references are actually 32 or 64 bit (or
whatever that platform requires).
Regards,
Jeroen
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Jeroen Frijters writes:
Andrew Haley wrote:
jbyte must have a single platform-specific definition, as all
JVMs on that platform should be able to execute the same JNI
library code (no recompilation required).
I didn't know that. Is that requirement documented anywhere? I can't
see how you'd do it on a machine with 32-bit and 64-bit modes, for
example.
Platform = Machine + OS. I don't have any reference, but I believe that
Etienne is right in saying that the same library should be usuable with
all JVMs on a specific platform.
But it's not necessarily possible. Clearly it's desirable, no
argument there.
Clearly the highest performance comes from using a naked long.
No, the highest performance comes from using a platform specific
pointer.
I think that for a JIT it is easier to replace a typed object
reference with a native pointer then a long. That's why I'm very
much in favor of using RawData. It can even be abstract, that way
the JNI layer can allocate sub types that contain the appropriate
number of bits to contain a native pointer.
For example:
public abstract class RawData {}
public final class RawData32 extends RawData
{
private int pointer;
}
public final class RawData64 extends RawData
{
private long pointer;
}
All the Java code would ever see are RawData references, but the JNI
layer knows that all RawData references are actually 32 or 64 bit (or
whatever that platform requires).
This seems to be identical to my proposal.
I no longer understand what we're arguing about...
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes:
Andrew Haley wrote:
Maybe, but that's not the only thing. It's possible to define jbyte
so that it is an 8 bit signed value but not a character type, and JNI
does not forbid this. I suspect that all the platforms we use define
jbyte to be a character type, but I can see no overpowering reason to
introduce a dependency on that.
jbyte must have a single platform-specific definition, as all
JVMs on that platform should be able to execute the same JNI
library code (no recompilation required).
I didn't know that. Is that requirement documented anywhere? I can't
see how you'd do it on a machine with 32-bit and 64-bit modes, for
example. Do we know that every JVM will use the same calling
convention for CNI?
I would argue that if a char type has 8 bits on a platform, there
is a strong case for it to be defined as typedef signed char
jbyte, and I would gues VM implementors would be veery unhappy if
Sum (or any other) decided to define jbyte otherwise on that
platform.
BUT I agree, it could be false on some system. So, assuming
the worst-case scenario, I have attached an updated version of my
byte array proposal that is, as far as I can tell, robust across
all possible platforms.
It contains 2 utility inline functions: wrap and unwrap. I
provide 2 versions of each, one version for systems where jbyte ==
signed char, and one version for systems where jbyte != signed
char. A test function, ideal for use in an autoconf macro, is
provided that issues a warning when jbyte != signed char.
So, Andrew, does this version pass the portability test?
I would really like to see the native counterpart of your opaque types and
compare the theoretical performance of it relative to the byte array
proposal.
Well, the proposals for this and opaque types aren't really
equivalent, in that I make no performance claims for using an opaque
wrapper for a native pointer. I do claim a maintenace and
understandability advantage, however.
Clearly the highest performance comes from using a naked long. But
both long and byte array have the disadvantage in that they're
untyped. The lack of typedef in Java is lamentable, but in its
absence we just have to do the best we can.
/* test function (unused). This function will cause a compiler
* warning when jbyte != signed char */
Maybe, maybe not. Is this guaranteed by the standard to cause a
warning?
/* Assuming that jbyte != signed char */
#define PTRDIFF_BIT (sizeof(ptrdiff_t) * CHAR_BIT)
#define BARRAY_SIZE ((PTRDIFF_BIT + 7) / 8)
/*
wrap: wraps a pointer into a jbyteArray in order to store in into a
Java object instance.
*/
inline static jbyteArray
wrap (JNIEnv *env, void *ptr)
{
jbyteArray nativePtr;
jbyte bytes[BARRAY_SIZE];
/* make a robust integer out of pointer value */
ptrdiff_t value = (char *) ptr - (char *) 0;
Don't do this subtraction, it isn't portable. Use intptr_t instead.
size_t i;
for (i = 0; i BARRAY_SIZE; i++)
{
bytes[i] = (value 0x0ff);
I don't think the 0x0ff is needed, but that probably doesn't matter.
And a little stylistic nit: it isn't a good idea to cat return values
from malloc().
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Andrew Haley wrote: This seems to be identical to my proposal. I no longer understand what we're arguing about... I know. I thought we were still trying to convince Etienne :-) Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
So, I have to admit that I haven't been following this thread in all of its glory :-), but the JNI spec is quite clear on the sizes of the various primitives. See the table in section 12.1.1 of the spec. A jbyte is signed 8 bit quantity. A jchar is an unsigned 16 bit quantity, etc. There are not defined in terms of things like C's char and int. It doesn't matter what the native platform thinks an int is, a jint is a signed 32 bit quantity. --dave___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
On Wed, Apr 07, 2004 at 11:19:47AM +0100, Andrew Haley wrote: jbyte must have a single platform-specific definition, as all JVMs on that platform should be able to execute the same JNI library code (no recompilation required). I didn't know that. Is that requirement documented anywhere? I can't see how you'd do it on a machine with 32-bit and 64-bit modes, for example. Do we know that every JVM will use the same calling convention for CNI? CNI is a gcj-specific thing, so gcj can do whatever they like... As for JNI, yes it is a requirement. JNI was *specifically* developed to get rid of the multiple VM-specific native interfaces that were developed in early Java implementations (Sun's, Netscape's, Microsoft's). In the JNI specification (e.g. chapters 11 to 13 of ISBN 0-201-32577-2 The Java Native Interface, Programmer's Guide and Specification), you find the following text: 11.1 Design Goals The most important goal of the JNI design is ensuring that it offers binary compatibility among different Java virtual machine implementations on a given host environment. The same native library binary will run on different virtual machine implementations on a given host environment without the need for recompilation. /* test function (unused). This function will cause a compiler * warning when jbyte != signed char */ Maybe, maybe not. Is this guaranteed by the standard to cause a warning? Do we really care? I think that it is sufficient to clearly document the requirement to define or not JBYTE_IS_NOT_SIGNED_CHAR appropriately. We both probably know very few platforms (I know of none, but I don't currently work on embedded processors...) that do not have 8-bit bytes (= chars). This function is simply an appropriate test to add to a configure script in a GNU environment, for the programmer's convenience *only*. /* make a robust integer out of pointer value */ ptrdiff_t value = (char *) ptr - (char *) 0; Don't do this subtraction, it isn't portable. Use intptr_t instead. ANSI/ISO 9899-1990 (the old standard, I guess) does not have intptr_t. And a little stylistic nit: it isn't a good idea to cat return values from malloc(). cat? What do you mean? [I know the example JNI code should have checked that malloc() had actually succeeded. And the code has a memory leak, as the malloc()ed memory isn't freed, but this code was simply meant as a test for wrapping/unwrapping native pointer... Writing robust and portable C code is painful. ;-)] Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Jeroen Frijters wrote:
That's why I'm very much in favor of using RawData. ...
public abstract class RawData {}
public final class RawData32 extends RawData
{
private int pointer;
}
public final class RawData64 extends RawData
{
private long pointer;
}
All the Java code would ever see are RawData references, but the JNI
layer knows that all RawData references are actually 32 or 64 bit (or
whatever that platform requires).
But this is a completely different model than the CNI RawData,
since you're adding an extra layer of indirection.
In CNI, a RawData is not an object that *contains* a pointer;
the pointer is the RawData reference itself.
I.e.
inline void* RawData_to_pointer (RawData *rd)
{
return (void*) (rd);
}
*not*
inline void* RawData_to_pointer (RawData *rd)
{
return (void*) (rd-pointer);
}
Adding the extra indirection seems like a bad idea to me.
I'd rather use jlong. But I don't see any real reason not to
use RawData. I can see on a few JVMs it may cause problems,
but they can easily sed the source to convert RawData to long.
Just think of RawData has a macro.
--
--Per Bothner
[EMAIL PROTECTED] http://per.bothner.com/
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
On Wed, Apr 07, 2004 at 11:42:08AM +0100, Andrew Haley wrote: Platform = Machine + OS. I don't have any reference, but I believe that Etienne is right in saying that the same library should be usuable with all JVMs on a specific platform. But it's not necessarily possible. Clearly it's desirable, no argument there. The JNI specification says otherwise. All the Java code would ever see are RawData references, but the JNI layer knows that all RawData references are actually 32 or 64 bit (or whatever that platform requires). This seems to be identical to my proposal. I no longer understand what we're arguing about... Why don't you show me the native couterpart of your proposal? I've been asking for it for some time, now, so that we can comment on it. If it's better than my proposal, fine, but I need to actually see it to make my mind about it. I've been showing code specifically so that we discuss about *concrete* things. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Per Bothner wrote:
Jeroen Frijters wrote:
That's why I'm very much in favor of using RawData. ...
public abstract class RawData {}
public final class RawData32 extends RawData
{
private int pointer;
}
public final class RawData64 extends RawData
{
private long pointer;
}
All the Java code would ever see are RawData references, but the JNI
layer knows that all RawData references are actually 32 or
64 bit (or
whatever that platform requires).
But this is a completely different model than the CNI RawData,
since you're adding an extra layer of indirection.
No. In CNI you would continue to use RawData as the pointer. I'm doing
the same in my VM. The subclassing is only for JNI.
Regards,
Jeroen
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: On Wed, Apr 07, 2004 at 11:42:08AM +0100, Andrew Haley wrote: Platform = Machine + OS. I don't have any reference, but I believe that Etienne is right in saying that the same library should be usuable with all JVMs on a specific platform. But it's not necessarily possible. Clearly it's desirable, no argument there. The JNI specification says otherwise. No, it doesn't. It says that it's a goal. All the Java code would ever see are RawData references, but the JNI layer knows that all RawData references are actually 32 or 64 bit (or whatever that platform requires). This seems to be identical to my proposal. I no longer understand what we're arguing about... Why don't you show me the native couterpart of your proposal? I've been asking for it for some time, now, so that we can comment on it. If it's better than my proposal, fine, but I need to actually see it to make my mind about it. I've been showing code specifically so that we discuss about *concrete* things. Okay. Since there's been so much confusion I agree that's a good idea. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Per Bothner writes:
Jeroen Frijters wrote:
That's why I'm very much in favor of using RawData. ...
public abstract class RawData {}
public final class RawData32 extends RawData
{
private int pointer;
}
public final class RawData64 extends RawData
{
private long pointer;
}
All the Java code would ever see are RawData references, but the JNI
layer knows that all RawData references are actually 32 or 64 bit (or
whatever that platform requires).
But this is a completely different model than the CNI RawData,
since you're adding an extra layer of indirection.
In CNI, a RawData is not an object that *contains* a pointer;
the pointer is the RawData reference itself.
I.e.
inline void* RawData_to_pointer (RawData *rd)
{
return (void*) (rd);
}
*not*
inline void* RawData_to_pointer (RawData *rd)
{
return (void*) (rd-pointer);
}
Adding the extra indirection seems like a bad idea to me.
I'd rather use jlong.
I take your point. However, if Eclipse used named pointer types
rather than just int or long I would probably saved about a
month's work. I'd like us not to repeat that mistake.
Okay, we could agree that documentation along the lines of foo is a
native pointer would help. But I have a keen bias for self
documenting code where possible.
But I don't see any real reason not to use RawData. I can see on a
few JVMs it may cause problems, but they can easily sed the source
to convert RawData to long. Just think of RawData has a macro.
Or perhaps redefine RawData from its current raw form to the form
with an extra layer of indirection -- that wouldn't require the Java
client code to change.
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Per Bothner wrote: I can see on a few JVMs it may cause problems, but they can easily sed the source to convert RawData to long. Just think of RawData has a macro. I am starting to have difficulty understanding Classpath's goals and motivations. When I proposed to add to Classpath some mechanism to allow it to be customized to each VM, I was told that it would be a heresy to encode any VM-specific thing into Classpath, as the vision was: a single Classpath installation would be shared by many VMs on a single system, e.g. common class libraries, common JNI libraries, etc. Now, what you are proposing is that JNI libraries would be compiled specifically for each VM. What have I missed? Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Okay, but ANS specifically does not allow you to do this subtraction. Also, there is no guarantee that every pointer is representable as a ptrdiff_t. (6.5.6 Para 9, if you're interested) The point is: if your platform is one that does *not* have 8-bit chars (!!!), then you can simply rewrite the appropriate code to put some bit representation of your pointer into a jbyte array (using whatever trick you need), and the only place where you would have to do it is the the wrap/unwarp functions. No modification on the Java side. MORE IMPORTANTLY: As long as your platform has 8-bit chars/bytes (99,9%), [and that no crasy person decided that on such platform jbyte would be different from signed char], then my proposed code will just work out-of-the-box, and be quite efficient (for JNI code). Comment: Now that you raised 6.5.6 para 9: how do you intend to cast such pointer to a jlong, in your proposal? [ironic] OK. Let's stop splitting hairs in half. I have showed you a valuable and practical proposal. I'd like to see your full proposal, JNI part included. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Hi Etienne, while I think that your proposal would work with our VM it would also be quite ineficient. Our arrays can be fragemented so we nee to do some magic when accessing them in native code. Do you have any system that requires a pointer size larger than 64bits ? My feeling is that once we hit systems of that size Java is in trouble for having only 63bits for file size and 31 bits array indces. ingo Etienne Gagnon wrote: Andrew Haley wrote: Maybe, but that's not the only thing. It's possible to define jbyte so that it is an 8 bit signed value but not a character type, and JNI does not forbid this. I suspect that all the platforms we use define jbyte to be a character type, but I can see no overpowering reason to introduce a dependency on that. jbyte must have a single platform-specific definition, as all JVMs on that platform should be able to execute the same JNI library code (no recompilation required). I would argue that if a char type has 8 bits on a platform, there is a strong case for it to be defined as typedef signed char jbyte, and I would gues VM implementors would be veery unhappy if Sum (or any other) decided to define jbyte otherwise on that platform. BUT I agree, it could be false on some system. So, assuming the worst-case scenario, I have attached an updated version of my byte array proposal that is, as far as I can tell, robust across all possible platforms. It contains 2 utility inline functions: wrap and unwrap. I provide 2 versions of each, one version for systems where jbyte == signed char, and one version for systems where jbyte != signed char. A test function, ideal for use in an autoconf macro, is provided that issues a warning when jbyte != signed char. So, Andrew, does this version pass the portability test? I would really like to see the native counterpart of your opaque types and compare the theoretical performance of it relative to the byte array proposal. Etienne -- Ingo Prötel [EMAIL PROTECTED] aicas GmbHhttp://www.aicas.com Haid-und-Neu-Str. 18phone +49 721 663 968-32 76131 Karlsruhe fax +49 721 663 968-93 Germany ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Etienne Gagnon wrote: There is a single JNI source code source base, used by everyone. Please don't say that. Not everyone uses the C part of Classpath. Besides, if you want NIO (we are still talking about NIO, right?) to perform well, you're going to need some VM magic. Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Andrew Haley wrote: malloc() returns a char*, not a jbyte*. So, can you tell me the difference between a jbyte and a signed char? Sigh. No it isn't, and this code will break with gcc. OK, maybe I am tired and I don't see it. GCC -Wall does not complain about the attached example. When you have aliasing errors, the result is usually incorrect code, not compile time warnings. ... Also, when a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object. We agree on this. Now: what's different between signed char and jbyte? Can they actually be distinct on some platform? Sure. That's the question, really: whether on all platforms a jbyte is necessarily a character type. It isn't descibed as one on the JNI spec, so it may or may not be. On existing platforms it probably is, but on those platforms a pointer will always fit into a long. It all depends on how jbyte is defined: it can either be a signed character type or not. I'm suggesting that we should not depend on that. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: I should have compiled with -pedantic, of course... I've included a few fixes in the attachment. malloc() returns a char*, not a jbyte*. [#1] byte addressable unit of data storage large enough to hold any member of the basic character set of the execution environment [#2] NOTE 1 It is possible to express the address of each individual byte of an object uniquely. [#3] NOTE 2 A byte is composed of a contiguous sequence of bits, the number of which is implementation-defined. The least significant bit is called the low-order bit; the most significant bit is called the high-order bit. 3.5 [#1] character bit representation that fits in a byte * Do we actually have to deal with platforms that have non 8-bit chars? Maybe, but that's not the only thing. It's possible to define jbyte so that it is an 8 bit signed value but not a character type, and JNI does not forbid this. I suspect that all the platforms we use define jbyte to be a character type, but I can see no overpowering reason to introduce a dependency on that. I guess quite a few other things/algorithms in the class library would break if it is so... Perhaps it's time to fix them, then. It's fine to be pedantic, but up to a point... That depends on what you mean by pedantry. Both issues are in a sense pedantic: the 128-bit pointer type and the non-character jbyte type. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Etienne Gagnon wrote: FYI: The JNI specification guarantees that jbyte is an 8-bit signed value. Hmmm... Thinking about all this mess of non-specified C byte length... Can JNI actually be implemented on a 16-bit per byte system? Anybody has a reasonable answer? To consider: 5.2.4.2.1 Sizes of integer types limits.h [#1] The values given below shall be replaced by constant expressions suitable for use in #if preprocessing directives... ... -- number of bits for smallest object that is not a bit- field (byte) CHAR_BIT 8 ... So, what how would one define jbyte on a platform where CHAR_BIT is defined as 16? JNI depends on having such a type. You'd have to do something non-ANS. I suspect that compilers on machines with non 8-bit bytes would define a byte type as an ANS extension. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
David Holmes writes: Well if we're being pedantic and imprecise to boot ... malloc() returns a char*, not a jbyte*. malloc() returns void* not char*. It hasn't returned char* since pre ANSI C. True, yes. Hence the pointer returned by malloc can be legally converted to any real pointer type with clearly defined semantics. Taking the address of a void* and pretending that it is the address of a jbyte* is not legal. However, it is legal to convert between an int and a pointer type and a pointer type and an int. If the address represented by the int is incorrectly aligned for the pointer type then the result is undefined. Otherwise the result is implementation specific. Hence you can write legal but (by definition) non-portable code to do what you want. No, because of aliasing rules. You can't convert a pointer between two types that are not character types and then dereference it. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Do we actually have to deal with platforms that have non 8-bit chars? I guess quite a few other things/algorithms in the class library would break if it is so... It's fine to be pedantic, but up to a point... FYI: The JNI specification guarantees that jbyte is an 8-bit signed value. Yes, but it's not a question of whether the type jbyte is the same size as a character type, but whether it is treated in the same way as a character by the compiler. I asked the question on the gcc mailing list. (BTW: Alias set zero is the set of types that may alias every other type. In ISO C, character types are in alias set zero.) --- From: Richard Henderson [EMAIL PROTECTED] To: Andrew Haley [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Byte types and aliasing Date: Tue, 6 Apr 2004 11:20:09 -0700 On Tue, Apr 06, 2004 at 01:27:55PM +0100, Andrew Haley wrote: Is a type of mode(byte) guaranteed to be in alias set zero? No. A language can do whatever they want. --- By this reading, I take it that a JNI implementor could define jbyte not to be a character type. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote:
Maybe, but that's not the only thing. It's possible to define jbyte
so that it is an 8 bit signed value but not a character type, and JNI
does not forbid this. I suspect that all the platforms we use define
jbyte to be a character type, but I can see no overpowering reason to
introduce a dependency on that.
jbyte must have a single platform-specific definition, as all JVMs on that
platform should be able to execute the same JNI library code (no recompilation
required). I would argue that if a char type has 8 bits on a platform, there
is a strong case for it to be defined as typedef signed char jbyte, and I would
gues VM implementors would be veery unhappy if Sum (or any other) decided to
define jbyte otherwise on that platform.
BUT I agree, it could be false on some system. So, assuming the worst-case
scenario, I have attached an updated version of my byte array proposal that
is, as far as I can tell, robust across all possible platforms.
It contains 2 utility inline functions: wrap and unwrap. I provide 2 versions
of each, one version for systems where jbyte == signed char, and one version
for systems where jbyte != signed char. A test function, ideal for use in
an autoconf macro, is provided that issues a warning when jbyte != signed char.
So, Andrew, does this version pass the portability test?
I would really like to see the native counterpart of your opaque types and
compare the theoretical performance of it relative to the byte array
proposal.
Etienne
--
Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/
SableVM: http://www.sablevm.org/
SableCC: http://www.sablecc.org/
/* Put in the public domain by Etienne M. Gagnon [EMAIL PROTECTED] */
/* DO NOT EDIT THIS FILE - it is machine generated */
#include jni.h
/* Header for class JNITest */
#ifndef _Included_JNITest
#define _Included_JNITest
#ifdef __cplusplus
extern C
{
#endif
/*
* Class: JNITest
* Method:getPtr
* Signature: ()[B
*/
JNIEXPORT jbyteArray JNICALL Java_JNITest_getPtr (JNIEnv *, jclass);
/*
* Class: JNITest
* Method:testPtr
* Signature: ([B)V
*/
JNIEXPORT void JNICALL Java_JNITest_testPtr (JNIEnv *, jclass, jbyteArray);
#ifdef __cplusplus
}
#endif
#endif
/* Put in the public domain by Etienne M. Gagnon [EMAIL PROTECTED] */
public class JNITest
{
static
{
System.loadLibrary(test);
}
public static void main(String[] args)
{
byte[] nativePtr = getPtr();
testPtr(nativePtr);
}
private static native byte[] getPtr();
private static native void testPtr(byte[] nativPtr);
}
/* Put in the public domain by Etienne M. Gagnon [EMAIL PROTECTED] */
#include jni.h
#include stdio.h
#include stdlib.h
#include wrap.c
/*
* Class: JNITest
* Method:getPtr
* Signature: ()[B
*/
JNIEXPORT jbyteArray JNICALL
Java_JNITest_getPtr (JNIEnv *env, jclass class)
{
int *int_ptr;
int_ptr = (int *) malloc (sizeof (int));
*int_ptr = 0x47;
return wrap (env, int_ptr);
}
/*
* Class: JNITest
* Method:testPtr
* Signature: ([B)V
*/
JNIEXPORT void JNICALL
Java_JNITest_testPtr (JNIEnv *env, jclass class, jbyteArray nativePtr)
{
int *int_ptr = unwrap (env, nativePtr);
printf (value = %x\n, *int_ptr);
}
#include jni.h
#include stdio.h
#include stddef.h
#include limits.h
/* On platforms where jbyte != signed char, you must
* define JBYTE_IS_NOT_SIGNED_CHAR.
*
* See the test function below that causes compiler
* warnings when jbyte != signed char to remind you
* of defining JBYTE_IS_NOT_SIGNED_CHAR in such case.
*
* On most platforms, this shouldn't matter.
*/
/* Uncomment the following if necessary. */
/* #define JBYTE_IS_NOT_SIGNED_CHAR 1 */
#ifndef JBYTE_IS_NOT_SIGNED_CHAR
/* Assuming that jbyte == signed char */
/* test function (unused). This function will cause a compiler
* warning when jbyte != signed char */
static void
test ()
{
jbyte *j = NULL;
/*
* IMPORTANT
* =
*
* If you get a warning for the assignment below, you *MUST* define
* JBYTE_IS_NOT_UNSIGNED_CHAR.
*/
signed char *k = j;
/* just to get rid of compile warnings */
j = k;
test ();
}
/*
wrap: wraps a pointer into a jbyteArray in order to store in into a
Java object instance.
*/
inline static jbyteArray
wrap (JNIEnv *env, void *ptr)
{
jbyteArray nativePtr;
nativePtr = (*env)-NewByteArray (env, (jint) sizeof (void *));
if (nativePtr == NULL) /* Exception raised */
return NULL;
(*env)-SetByteArrayRegion (env, nativePtr, 0, (jint) sizeof (void *),
(jbyte *) ptr);
return nativePtr;
}
/*
unwrap: unwraps a pointer stored in a jbyteArray.
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Yes, but it's not a question of whether the type jbyte is the same size as a character type, but whether it is treated in the same way as a character by the compiler. Hi Andrew. Please look carefully at my proposal (sent minutes ago), and tell me if it still contains non-portable bits. I provide in it a piece of C code that will generate warnings if jbyte is not treated as signed char by the compiler, if I am right. [Look at the test() function]. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Quoting Etienne Gagnon ([EMAIL PROTECTED]): I would really like to see the native counterpart of your opaque types and compare the theoretical performance of it relative to the byte array proposal. Sorry if I lost track of the discussion, but when the proposal of an abstract class for virtual machine data came up, I thought that one of the reasons for this boxing was the portability to virtual machines of any kind. A byte array approach would have performance and design implications not for JNI-based implementations, but for more exotic VMs which do not have any native pointers at all (IKVM?). Also, is this still about the Class.vmData slot? I note that the proposed java.nio patch already uses a RawData class. d. -- Common Lisp is the Borg of programming languages -- Bill Clementson pgp0.pgp Description: PGP signature ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: Can JNI actually be implemented on a 16-bit per byte system? Anybody has a reasonable answer? Such a system, to implement JNI, will have to have a compiler-defined extension for a signed 8-bit byte type, a bit like the GNU long long extension. ANSI C allows a compiler to add additional types, as long as it implements the standard ones. It will also need an unsigned 8 bit quantity (jboolean), signed and unsigned 16 bit ones (jshort, jchar), signed 32 and 64 bit quantities (jint, jlong), and (the stinger) 32 bit and 64 bit floating point values (jfloat, jdouble). Now, I'm not sure how many people who use JNI are going to actually depend upon a jfloat being exactly 32 bits long (and not one bit more!) or a jdouble being exactly 64 bits long, but according to table 3-1 in the JNI spec they are indeed entitled to make that assumption. So, any C compiler that you want to write JNI code with had better have a way to specify floating point types with those lengths. -- Steven Augart Jikes RVM, a free, open source, Virtual Machine: http://oss.software.ibm.com/jikesrvm ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Michael Koch wrote: The RawData pointer will get the real address to a native memory region. It can be directly used and has always the pointer size of the platform. There will never be problems on platforms with more then 64 bit. Its really simply. Its just not done in classpath yet because I developed NIO on gcj so far. This hopefully changes now. (JamVM, be my friend ...). For what it's worth, I really like this approach (using RawData in NIO) and I'm looking forward on getting NIO working on IKVM. Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 15:10 schrieb Andrew Haley: Michael Koch writes: To be honest I think we should not have RawData in Classpath. The trouble with RawData (as it is used in gcj) is that it breaks the Java type system. Its bizarre semantics mean that you have something that looks like an object reference but you can't use it as one. You can't convert it to an instance of Object, for example. A garbage collector can't inspect it. A VM that doesn't expect this behaviour might crash. If we use RawData like that in Classpath, a VM may need to be modified. It might not even be possible. For that reason, I believe that we should not use RawData in Classpath, although some VM implementors might choose to. I think we need a better abstraction. Perhaps a Pointer class that may be subclassed to swtPeerPointer, nativeIntPointer, and so on. That will get over the enormous and painful maintenance problem with SWT, where you have no idea what type a pointer is, because everything is implicily void*. This is truly Evil. If some VM implementers want to use a RawData reference as a raw pointer, fine. But IMO our reference JNI shouldn't. It is used in VMClass, but it seems no VM implementor actually likes how that is setup now. And it is used in some of the new nio implementation classes. But I have a hard time seeing how our reference JNI implementation will make use of it. All places where it is used have unimplemented native jni parts. How are you planning to use this? Maybe we could add state (a Object or byte[] field to the class) that holds the real data (== address). Better. And some access method to get add the address it represents (that then might be optimized by a particular VM). But that would mean that it isn't the same type as the one from libgcj (or Ptr from kaffe) since those classes are not actually accessed through normal java code or through JNI, but handled specially through CNI or Kaffe native/vm code. As Andrew says we could maybe just use long if we want to store an address. Andrew also worked hard on getting SWT (Eclipse) 64 bit clean. What I have seen from the (huge) patch this was mainly done by turning addresses as stored by JNI from int to long. BTW, was this work ever accepted into Eclipse? The RawData pointer will get the real address to a native memory region. It can be directly used and has always the pointer size of the platform. Uh, I don't understand what you mean, sorry. Why is a Java reference the same size as a native pointer? Perhaps I misunderstood you. AFAIK a java refernce is a pointer and this pointer can point everywhere. Right ? Tell me when I'm wrong. Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Michael Koch writes: Uh, I don't understand what you mean, sorry. Why is a Java reference the same size as a native pointer? Perhaps I misunderstood you. AFAIK a java refernce is a pointer and this pointer can point everywhere. Right ? No, not at all. A java reference is simply a handle by which an object may be found. It might well be a hash code, or a table index. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Michael Koch writes: To be honest I think we should not have RawData in Classpath. The trouble with RawData (as it is used in gcj) is that it breaks the Java type system. Its bizarre semantics mean that you have something that looks like an object reference but you can't use it as one. You can't convert it to an instance of Object, for example. A garbage collector can't inspect it. A VM that doesn't expect this behaviour might crash. If we use RawData like that in Classpath, a VM may need to be modified. It might not even be possible. For that reason, I believe that we should not use RawData in Classpath, although some VM implementors might choose to. I think we need a better abstraction. Perhaps a Pointer class that may be subclassed to swtPeerPointer, nativeIntPointer, and so on. That will get over the enormous and painful maintenance problem with SWT, where you have no idea what type a pointer is, because everything is implicily void*. This is truly Evil. If some VM implementers want to use a RawData reference as a raw pointer, fine. But IMO our reference JNI shouldn't. It is used in VMClass, but it seems no VM implementor actually likes how that is setup now. And it is used in some of the new nio implementation classes. But I have a hard time seeing how our reference JNI implementation will make use of it. All places where it is used have unimplemented native jni parts. How are you planning to use this? Maybe we could add state (a Object or byte[] field to the class) that holds the real data (== address). Better. And some access method to get add the address it represents (that then might be optimized by a particular VM). But that would mean that it isn't the same type as the one from libgcj (or Ptr from kaffe) since those classes are not actually accessed through normal java code or through JNI, but handled specially through CNI or Kaffe native/vm code. As Andrew says we could maybe just use long if we want to store an address. Andrew also worked hard on getting SWT (Eclipse) 64 bit clean. What I have seen from the (huge) patch this was mainly done by turning addresses as stored by JNI from int to long. BTW, was this work ever accepted into Eclipse? The RawData pointer will get the real address to a native memory region. It can be directly used and has always the pointer size of the platform. Uh, I don't understand what you mean, sorry. Why is a Java reference the same size as a native pointer? Perhaps I misunderstood you. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley writes: Michael Koch writes: Uh, I don't understand what you mean, sorry. Why is a Java reference the same size as a native pointer? Perhaps I misunderstood you. AFAIK a java refernce is a pointer and this pointer can point everywhere. Right ? No, not at all. A java reference is simply a handle by which an object may be found. It might well be a hash code, or a table index. Or -- and this is by no means unlikely -- you might have a 32-bit addressable heap on a 64-bit system. This would be a pretty good implementation strategy on some systems. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 16:16 schrieb Andrew Haley: Michael Koch writes: Uh, I don't understand what you mean, sorry. Why is a Java reference the same size as a native pointer? Perhaps I misunderstood you. AFAIK a java refernce is a pointer and this pointer can point everywhere. Right ? No, not at all. A java reference is simply a handle by which an object may be found. It might well be a hash code, or a table index. Oh, didnt knew this. Perhaps I'm too much oriented towards CNI ... Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Andrew Haley wrote: Michael Koch writes: To be honest I think we should not have RawData in Classpath. The trouble with RawData (as it is used in gcj) is that it breaks the Java type system. Its bizarre semantics mean that you have something that looks like an object reference but you can't use it as one. You can't convert it to an instance of Object, for example. Not necessarily. On my (admittedly weird) VM, RawData is a direct pointer, but it can also be treated as an object, this is not particularly efficient way of using it (it gets boxed), but it does work. Etienne Gagnon wrote: IMO, the cleanest approach is really the use of a byte array. But it is inefficient and hard to optimize. I don't see why RawData doesn't give you the same capabilities as a byte array (as an extreme example, you can replace Classpath's RawData with your own version that wraps a byte array). Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
I Agree with Andrew, regarding the reference JNI library code; this code is *NOT* the VM*.java interface, so it should be written to be compatible with any JNI compliant JVM. As far as I know, disguising a native pointer into a Java reference is not portable across JVMs. IMO, the cleanest approach is really the use of a byte array. It allows for *any* native pointer length, without making this length explicit in any Java source code (assuming these arrays are allocated in the native JNI parts). The Garbage collector can move this array as it wishes, yet the value will be preserved. I oppose the use of long-term maintenance trouble things such as int or long. As for performance, the additional indirection cost, to get the byte array content value (e.g. the native pointer) is offset by the cost of the JNI call itself. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes:
IMO, the cleanest approach is really the use of a byte array.
Well, I tell you what: we could wrap the {byte[],long} in an opaque
type, call it a native pointer, and be done with it.
- Everyone who can use a long as a pointer (and, in practice, this is
everyone) will do so.
- Anyone who needs to use a byte[] as a pointer will do so.
- We can still have some type safety and readability by making
subclasses of native pointer.
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Jeroen Frijters writes: Andrew Haley wrote: Michael Koch writes: To be honest I think we should not have RawData in Classpath. The trouble with RawData (as it is used in gcj) is that it breaks the Java type system. Its bizarre semantics mean that you have something that looks like an object reference but you can't use it as one. You can't convert it to an instance of Object, for example. Not necessarily. On my (admittedly weird) VM, RawData is a direct pointer, but it can also be treated as an object, this is not particularly efficient way of using it (it gets boxed), but it does work. Yeah, but that's a special case. In general, on most VMs, you can't convert a RawData to an Object. Etienne Gagnon wrote: IMO, the cleanest approach is really the use of a byte array. But it is inefficient and hard to optimize. Right. The choice of a byte array is the worst of all possible worlds. Well okay, it is at least portable, but it has no other virtues. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 18:07 schrieb Andrew Haley: Jeroen Frijters writes: Andrew Haley wrote: Michael Koch writes: To be honest I think we should not have RawData in Classpath. The trouble with RawData (as it is used in gcj) is that it breaks the Java type system. Its bizarre semantics mean that you have something that looks like an object reference but you can't use it as one. You can't convert it to an instance of Object, for example. Not necessarily. On my (admittedly weird) VM, RawData is a direct pointer, but it can also be treated as an object, this is not particularly efficient way of using it (it gets boxed), but it does work. Yeah, but that's a special case. In general, on most VMs, you can't convert a RawData to an Object. Etienne Gagnon wrote: IMO, the cleanest approach is really the use of a byte array. But it is inefficient and hard to optimize. Right. The choice of a byte array is the worst of all possible worlds. Well okay, it is at least portable, but it has no other virtues. I'm not really experienced with JNI and look what you all come up with. To me the byte[] solutions seems to be the most complicated to use because its easy possible to cast it to void*. Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew, Andrew Haley wrote: - Everyone who can use a long as a pointer (and, in practice, this is everyone) will do so. It's *NOT* the VM interface! There is a single JNI source code source base, used by everyone. So, there is no SableVM does it this way, gcj that way, and Kaffe the other. It has to work also with Sun VM, for example. AGAIN: I AM NOT TALKING ABOUT THE VM INTERFACE. I hope the message is clear! Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Michael Koch writes: I think Andrew's solution might be viable. His solution is already used in SWT. SWT runs with SUN JRE so it seems to be portable enough for us. Well, not exactly: I'm suggesting that we wrap all those longs in an opaque type. But otherwise, yes. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 19:30 schrieb Andrew Haley: Michael Koch writes: I think Andrew's solution might be viable. His solution is already used in SWT. SWT runs with SUN JRE so it seems to be portable enough for us. Well, not exactly: I'm suggesting that we wrap all those longs in an opaque type. But otherwise, yes. Can you give an example on how to do this ? Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 18:50 schrieb Etienne Gagnon: Andrew, Andrew Haley wrote: - Everyone who can use a long as a pointer (and, in practice, this is everyone) will do so. It's *NOT* the VM interface! There is a single JNI source code source base, used by everyone. So, there is no SableVM does it this way, gcj that way, and Kaffe the other. It has to work also with Sun VM, for example. To be clear, gcj uses CNI. In CNI references are pointers and the RawData reference is used as pointer to something (e.g. an address space). AGAIN: I AM NOT TALKING ABOUT THE VM INTERFACE. I hope the message is clear! We should come up with a portable way to do this in JNI. I think everybody is aware of the fact that this is not the VM interface. We just think that byte[] is not really straigt forward and some people try to come up with something different. I think Andrew's solution might be viable. His solution is already used in SWT. SWT runs with SUN JRE so it seems to be portable enough for us. Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Well, not exactly: I'm suggesting that we wrap all those longs in an opaque type. But otherwise, yes. So, how do you do opaque types, in Java? And how do you guarantee portability to 128bit systems? Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Andrew, Andrew Haley wrote: - Everyone who can use a long as a pointer (and, in practice, this is everyone) will do so. It's *NOT* the VM interface! There is a single JNI source code source base, used by everyone. It's easy to parameterize. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes: Andrew Haley wrote: Well, not exactly: I'm suggesting that we wrap all those longs in an opaque type. But otherwise, yes. So, how do you do opaque types, in Java? You write the code using a class that wraps your native pointer: a class with a single member. And how do you guarantee portability to 128bit systems? You tell people if you have a pointer that won't fit in a jlong, you'll need to change three lines of code. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 5. April 2004 20:06 schrieb Andrew Haley:
Etienne Gagnon writes:
Andrew Haley wrote:
Well, not exactly: I'm suggesting that we wrap all those longs
in an opaque type. But otherwise, yes.
So, how do you do opaque types, in Java?
You write the code using a class that wraps your native pointer: a
class with a single member.
And how do you guarantee portability to 128bit systems?
You tell people if you have a pointer that won't fit in a jlong,
you'll need to change three lines of code.
So we will have code like this:
package gnu.java.lang:
public class NativePointer
{
public long address;
}
Michael ?
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote:
Sure. Instead of putting a native pointer in a long or in a byte[],
you just declare a class with a single field that contains the
pointer. Everyone who needs a pointer makes a suitably named
subclass, so they'll know what they're pointing to.
How is that more efficient than a byte array?
Here's a concrete example:
class SomeRandomLibraryClass
{
static
{
System.loadLibrary(SomeRandomJNILibrary);
}
private byte[] nativePointer;
private native byte[] initNativeData(...);
private native void someNativeMethod(byte[] nativePointer, ...);
public SomeRandomLibraryClass(...)
{
...
nativePointer = initNativeData(...);
}
public void someMethod(...)
{
someNativeMethod(nativePointer, ...) /** THE REAL CALL **/
}
}
In the C code, we'll have:
JNIEXPORT void JNICALL
Java_somepackage_someNativeMethod
(JNIEnv *env, jobject this, jbyteArray nativePointer, ...)
{
void *ptr;
(*env)-GetByteArrayRegion(env, nativePointer, 0, sizeof(void *), (jbyte *) ptr);
...
/* do whatever with ptr */
...
}
So, how is your opaque type more efficient (and portable)?
Etienne
--
Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/
SableVM: http://www.sablevm.org/
SableCC: http://www.sablecc.org/
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Michael Koch writes:
Am Montag, 5. April 2004 20:06 schrieb Andrew Haley:
Etienne Gagnon writes:
Andrew Haley wrote:
Well, not exactly: I'm suggesting that we wrap all those longs
in an opaque type. But otherwise, yes.
So, how do you do opaque types, in Java?
You write the code using a class that wraps your native pointer: a
class with a single member.
And how do you guarantee portability to 128bit systems?
You tell people if you have a pointer that won't fit in a jlong,
you'll need to change three lines of code.
So we will have code like this:
package gnu.java.lang:
public class NativePointer
{
public long address;
}
Yes. But don't use it directly: use
class gtkPeerPointer extends NativePointer
{
}
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Michael Koch writes: Am Montag, 5. April 2004 19:30 schrieb Andrew Haley: Michael Koch writes: I think Andrew's solution might be viable. His solution is already used in SWT. SWT runs with SUN JRE so it seems to be portable enough for us. Well, not exactly: I'm suggesting that we wrap all those longs in an opaque type. But otherwise, yes. Can you give an example on how to do this ? Sure. Instead of putting a native pointer in a long or in a byte[], you just declare a class with a single field that contains the pointer. Everyone who needs a pointer makes a suitably named subclass, so they'll know what they're pointing to. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes:
Andrew Haley wrote:
Sure. Instead of putting a native pointer in a long or in a byte[],
you just declare a class with a single field that contains the
pointer. Everyone who needs a pointer makes a suitably named
subclass, so they'll know what they're pointing to.
How is that more efficient than a byte array?
It's a heck of a lot *better* than a byte array. Whether it's more
efficient or not depends on the design of your VM.
class SomeRandomLibraryClass
{
static
{
System.loadLibrary(SomeRandomJNILibrary);
}
private RandomPointer extends nativePointer;
private native RandomPointer initNativeData(...);
private native void someNativeMethod(RandomPointer nativePointer, ...);
public SomeRandomLibraryClass(...)
{
...
nativePointer = initNativeData(...);
}
public void someMethod(...)
{
someNativeMethod(nativePointer, ...) /** THE REAL CALL **/
}
}
In the C code, we'll have:
JNIEXPORT void JNICALL
Java_somepackage_someNativeMethod
(JNIEnv *env, jobject this, jbyteArray nativePointer, ...)
{
void *ptr;
(*env)-GetByteArrayRegion(env, nativePointer, 0, sizeof(void *), (jbyte *)
ptr);
Danger, Will Robinson! This is not legal C You can *not* take
the address of a pointer and cast it to a jbyte*.
Try finding a legal way to do this.
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
For one thing, you have not shown me *your* native part.
Second, see below.
Andrew Haley wrote:
JNIEXPORT void JNICALL
Java_somepackage_someNativeMethod
(JNIEnv *env, jobject this, jbyteArray nativePointer, ...)
{
void *ptr;
(*env)-GetByteArrayRegion(env, nativePointer, 0, sizeof(void *), (jbyte *) ptr);
Danger, Will Robinson! This is not legal C You can *not* take
the address of a pointer and cast it to a jbyte*.
What's wrong with it? What do you think you are doing, every time you use
malloc(), e.g.:
buffer = (some cast) malloc(...);
All I am doing is the same, and is perfectly legal ANSI/ISO C. Prove me wrong
with a specific ISO C specification clause, if you claim otherwise.
Etienne
--
Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/
SableVM: http://www.sablevm.org/
SableCC: http://www.sablecc.org/
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon writes:
For one thing, you have not shown me *your* native part.
Second, see below.
Andrew Haley wrote:
JNIEXPORT void JNICALL
Java_somepackage_someNativeMethod
(JNIEnv *env, jobject this, jbyteArray nativePointer, ...)
{
void *ptr;
(*env)-GetByteArrayRegion(env, nativePointer, 0, sizeof(void *), (jbyte *)
ptr);
Danger, Will Robinson! This is not legal C You can *not* take
the address of a pointer and cast it to a jbyte*.
What's wrong with it? What do you think you are doing, every time you use
malloc(), e.g.:
buffer = (some cast) malloc(...);
malloc() returns a char*, not a jbyte*.
All I am doing is the same, and is perfectly legal ANSI/ISO C.
Sigh. No it isn't, and this code will break with gcc.
Prove me wrong with a specific ISO C specification clause, if you
claim otherwise.
I spent a long time working on and supporting gcc, and this is the
rule I've had to refer people to more times than any other. It's
amazing how many programmers don't know the specfication.
All that C guarantees is that you can convert a pointer to a different
pointer type and then back again, and the result will compare equal to
the original pointer. Also, when a pointer to an object is converted
to a pointer to a character type, the result points to the lowest
addressed byte of the object. Successive increments of the result, up
to the size of the object, yield pointers to the remaining bytes of
the object. Except for this, you may not convert pointers of
different type and dereference them.
Scetion 6.2.2.3, Para 7.
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: Prove me wrong with a specific ISO C specification clause, if you claim otherwise. I spent a long time working on and supporting gcc, and this is the rule I've had to refer people to more times than any other. It's amazing how many programmers don't know the specfication. You guys are going off into the weeds. All that's required from C for the byte[] strategy to work is that this works: void *ptr; char *buf; ptr = any valid pointer value buf = malloc(sizeof(ptr)); memcpy(buf, ptr, sizeof(ptr)); ... then sometime later ... memcpy(ptr, buf, sizeof(ptr)); continue using the fully restored ptr I'm not a specophile but I would guess that C does at least guarantee this. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Archie Cobbs writes: Andrew Haley wrote: Prove me wrong with a specific ISO C specification clause, if you claim otherwise. I spent a long time working on and supporting gcc, and this is the rule I've had to refer people to more times than any other. It's amazing how many programmers don't know the specfication. You guys are going off into the weeds. All that's required from C for the byte[] strategy to work is that this works: void *ptr; char *buf; ptr = any valid pointer value buf = malloc(sizeof(ptr)); memcpy(buf, ptr, sizeof(ptr)); ... then sometime later ... memcpy(ptr, buf, sizeof(ptr)); continue using the fully restored ptr I'm not a specophile but I would guess that C does at least guarantee this. It does. I'm not trying to get into a debate about micro- optimization: given the overhead of JNI, it's probably not important. What I care about is that the code should be legal, maintainable, and efficient -- in that order. Having spent a lot of time working on SWT, the ability to subclass a native pointer is IMO very important indeed. It would have saved me a huge amount of time if native pointers had types. This is more important than quibbles about efficiency. However, if someone posts code that I don't believe is legal, I will point it out. I mean, I can't help it. :-) Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
I should have compiled with -pedantic, of course... I've included a few fixes
in the attachment.
malloc() returns a char*, not a jbyte*.
[#1] byte
addressable unit of data storage large enough to hold any
member of the basic character set of the execution
environment
[#2] NOTE 1 It is possible to express the address of each
individual byte of an object uniquely.
[#3] NOTE 2 A byte is composed of a contiguous sequence of
bits, the number of which is implementation-defined. The
least significant bit is called the low-order bit; the most
significant bit is called the high-order bit.
3.5
[#1] character
bit representation that fits in a byte *
Do we actually have to deal with platforms that have non 8-bit chars?
I guess quite a few other things/algorithms in the class library would
break if it is so...
It's fine to be pedantic, but up to a point...
FYI: The JNI specification guarantees that jbyte is an 8-bit signed value.
Etienne
--
Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/
SableVM: http://www.sablevm.org/
SableCC: http://www.sablecc.org/
#include jni.h
#include stdio.h
#include stdlib.h
/*
* Class: JNITest
* Method:getPtr
* Signature: ()[B
*/
JNIEXPORT jbyteArray JNICALL
Java_JNITest_getPtr (JNIEnv * env, jclass class)
{
int *int_ptr;
jbyteArray nativePtr;
int_ptr = (int *) malloc (sizeof (int));;
*int_ptr = 0x47;
nativePtr = (*env)-NewByteArray (env, (jint) sizeof (int *));
if (nativePtr == NULL)
return NULL;
(*env)-SetByteArrayRegion (env, nativePtr, 0, (int) sizeof (int *),
(jbyte *) int_ptr);
return nativePtr;
}
/*
* Class: JNITest
* Method:testPtr
* Signature: ([B)V
*/
JNIEXPORT void JNICALL
Java_JNITest_testPtr (JNIEnv * env, jclass class, jbyteArray nativePtr)
{
int *int_ptr;
(*env)-GetByteArrayRegion (env, nativePtr, 0, (jint) sizeof (int *),
(jbyte *) int_ptr);
printf (value = %x\n, *int_ptr);
}
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Well if we're being pedantic and imprecise to boot ... malloc() returns a char*, not a jbyte*. malloc() returns void* not char*. It hasn't returned char* since pre ANSI C. Hence the pointer returned by malloc can be legally converted to any real pointer type with clearly defined semantics. Taking the address of a void* and pretending that it is the address of a jbyte* is not legal. However, it is legal to convert between an int and a pointer type and a pointer type and an int. If the address represented by the int is incorrectly aligned for the pointer type then the result is undefined. Otherwise the result is implementation specific. Hence you can write legal but (by definition) non-portable code to do what you want. David Holmes ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: FYI: The JNI specification guarantees that jbyte is an 8-bit signed value. Hmmm... Thinking about all this mess of non-specified C byte length... Can JNI actually be implemented on a 16-bit per byte system? Anybody has a reasonable answer? To consider: 5.2.4.2.1 Sizes of integer types limits.h [#1] The values given below shall be replaced by constant expressions suitable for use in #if preprocessing directives... ... -- number of bits for smallest object that is not a bit- field (byte) CHAR_BIT 8 ... So, what how would one define jbyte on a platform where CHAR_BIT is defined as 16? JNI depends on having such a type. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Hi, On Mon, 2004-03-29 at 16:47, Michael Koch wrote: Am Montag, 29. März 2004 17:12 schrieb Andrew Haley: Archie Cobbs writes: [...] Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. Indeed, but so does long. I suppose it's possible that on some weird platform a pointer mat not fit in long. In gcj was have a class RawData, which we know points to something that isn't an Object and isn't gcable. Yes, gcj has gnu.gcj.RawData. Classpath has gnu.classpath.RawData for the same purpose. It would be really nice to unify this. I tried once but I saw its a lot of work needed to change gnu.gcj.RawData to gnu.classpath.RawData (or gnu.lang.RawData or whatever). To be honest I think we should not have RawData in Classpath. It is used in VMClass, but it seems no VM implementor actually likes how that is setup now. And it is used in some of the new nio implementation classes. But I have a hard time seeing how our reference JNI implementation will make use of it. All places where it is used have unimplemented native jni parts. How are you planning to use this? Maybe we could add state (a Object or byte[] field to the class) that holds the real data (== address). And some access method to get add the address it represents (that then might be optimized by a particular VM). But that would mean that it isn't the same type as the one from libgcj (or Ptr from kaffe) since those classes are not actually accessed through normal java code or through JNI, but handled specially through CNI or Kaffe native/vm code. As Andrew says we could maybe just use long if we want to store an address. Andrew also worked hard on getting SWT (Eclipse) 64 bit clean. What I have seen from the (huge) patch this was mainly done by turning addresses as stored by JNI from int to long. BTW, was this work ever accepted into Eclipse? Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Hi, On Mon, 2004-03-29 at 18:42, Etienne Gagnon wrote: Etienne Gagnon wrote: vmData = new byte[PTR_SIZE]; or vmData = new RawData(); or whatever. So what's the problem, with this? And for those who want to do: vmData = [internalVMpointer] They can deal with it in many ways, such as: 1- make sure [internalVMpointer] points to a non-GC'ed memory, so that GC can identify it as special; 2- Set their internal compiler/interpreter to deal specially with all accesses to vmData (and use flow analysis to track the value propagation). I really think Object is the most general approach, as Classpath has decided to have a unique, good-for-all-vms source base approach. I think I agree when it comes to specific VM state in the VM-interface classes since those should not even leak into the rest of the code at all. For general state as hold for some native JNI call (which GNU Classpath also provides for java.net, java.nio, java.io, java.util, etc) I don't think this is the right approach though. There a combination of VMClass-shadow-object with possibly a long to hold native JNI (address) state seems a better approach. There providing a VMClass-shadow-object also allows VMs that don't use out JNI/Posix/C implementation for these methods to completely override/replace these classes with their own implementation (and defines a clear interface that has to be provided). Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Mark Wielaard writes: As Andrew says we could maybe just use long if we want to store an address. Andrew also worked hard on getting SWT (Eclipse) 64 bit clean. What I have seen from the (huge) patch this was mainly done by turning addresses as stored by JNI from int to long. BTW, was this work ever accepted into Eclipse? It's in Eclipse 3, but it's not totally stable yet. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=37775 for all the gory details. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Sonntag, 4. April 2004 16:15 schrieb Mark Wielaard: Hi, On Mon, 2004-03-29 at 16:47, Michael Koch wrote: Am Montag, 29. März 2004 17:12 schrieb Andrew Haley: Archie Cobbs writes: [...] Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. Indeed, but so does long. I suppose it's possible that on some weird platform a pointer mat not fit in long. In gcj was have a class RawData, which we know points to something that isn't an Object and isn't gcable. Yes, gcj has gnu.gcj.RawData. Classpath has gnu.classpath.RawData for the same purpose. It would be really nice to unify this. I tried once but I saw its a lot of work needed to change gnu.gcj.RawData to gnu.classpath.RawData (or gnu.lang.RawData or whatever). To be honest I think we should not have RawData in Classpath. It is used in VMClass, but it seems no VM implementor actually likes how that is setup now. And it is used in some of the new nio implementation classes. But I have a hard time seeing how our reference JNI implementation will make use of it. All places where it is used have unimplemented native jni parts. How are you planning to use this? Maybe we could add state (a Object or byte[] field to the class) that holds the real data (== address). And some access method to get add the address it represents (that then might be optimized by a particular VM). But that would mean that it isn't the same type as the one from libgcj (or Ptr from kaffe) since those classes are not actually accessed through normal java code or through JNI, but handled specially through CNI or Kaffe native/vm code. As Andrew says we could maybe just use long if we want to store an address. Andrew also worked hard on getting SWT (Eclipse) 64 bit clean. What I have seen from the (huge) patch this was mainly done by turning addresses as stored by JNI from int to long. BTW, was this work ever accepted into Eclipse? The RawData pointer will get the real address to a native memory region. It can be directly used and has always the pointer size of the platform. There will never be problems on platforms with more then 64 bit. Its really simply. Its just not done in classpath yet because I developed NIO on gcj so far. This hopefully changes now. (JamVM, be my friend ...). Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Mark == Mark Wielaard [EMAIL PROTECTED] writes: Mark I admit not to have looked into or thought about java.lang.ref support Mark yet. How many VMs based on GNU Classpath properly implement those? libgcj claims to :-). We had to hack Reference.java a little. http://gcc.gnu.org/java/compare/java.lang.ref.Reference.diff I don't remember the rationale for all the changes at this moment, but I could dig them up if someone really needs to know. Some of them can't be understood without also looking at how we implement the native parts of this code. I wouldn't expect a VM to need any changes outside of Reference. Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Jeroen Frijters writes: Etienne Gagnon wrote: Mark Wielaard wrote: I would like the vmdata field type then to be VMClass not Object. I disagree, as it imposes a restriction on what vmData actually is. The most obvious implementation of vmData is to be a byte[] instance holding the byte of a native pointer to an internal VM non-moveable data structure. I'm glad to see we agree (although I don't think it's at all obvious that it should be a byte[], not all VMs use native code). Eeeh. I can't imagine that either. If there's a strong argument for holding native pointer in a byte[] ? Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
On Sun, 2004-03-28 at 23:53, Archie Cobbs wrote: Mark Wielaard wrote: I had hoped that the VM interface for Class, Object, Thread and Throwable was usable for most VMs. What isn't in your case? JamVM 1.1.1 uses no special versions of these classes (i.e. it uses the VMClass, VMObject and VMThread, VMThrowable, etc., classes). JamVM 1.1.2 also uses VMRuntime, however, it has it's own version of ClassLoader, to fix the findLoadedClass problem. When the VM interface consisted of static methods on a VM class I was happy to adopt it (even an interpreter can inline the call, but JamVM doesn't do it at the moment). However, I didn't like the move to shadow objects in VMThread and VMClass (VMClass in particular was a lot of trouble to implement). However, it seemed pointless to use some of the interface and not others -- either all or nothing -- I came close to using nothing :) I am now more or less convinced that in some cases we should even mark it as 'Object'. Again marking it as VMWhatever I had hoped that the compiler/VM would easily be able to handle it specially. See RawData in gcj. I would prefer the internal VM data field to be of type Object (this is what I use in JamVM). This means it should be the width of a native pointer field, whether the machine is 32 or 64 bit. - I prefer to avoid the extra method call overhead for important methods like Object.wait() imposed by the VMFoo split. In Classpath every Object.wait() requires calling VMObject.wait(). That makes the code separation cleaner but doesn't make implementation any easier - it's just as easy to implement Object.wait() as a native method as it is to implement VMObject.wait() as a native method. See above. And the environment is always free to overwrite specific special cases (like Object and Class will almost always be I guess). But not JamVM. Inlining of final calls is something I intend to look at soon :) - My Thread class uses private objects to implement sleep() and join() in terms of Object.wait(). The VM notify()'s this object when the thread exits. This means all the complexity of sleeping (and handling Thread.interrupt()) can be put in Object.wait() and not duplicated elsewhere. Could you post your versions? It might be interesting to see whether we can adopt this approach as default in the vm/reference implementation. VMThread now does have a lot of native methods. But I believe I discussed some of these issues with Jeroen and if I remember correctly there were some subtle issues with just doing everything as wrappers over Object.wait()/notify(). I would be very unhappy about this. Sleep in JamVM is done by waiting on an internal monitor, which is what Object.wait() maps down to (the monitor associated with the Object) and joining threads wait on the vmthread associated with the thread, which is notified on exit. So all code is done in terms of monitor wait, which handles interrupt, so JamVM doesn't duplicate code either. JamVM uses thin locks which map down to a real monitor in the presence of contention. Join/sleep by definition requires a real monitor, so I side step the redundant monitor lookup. The current approach allows the VM a lot of freedom. Trying to do too much in Java removes this, and in my opinion forces many VM implementors to abandon the VM interface altogether. - I folded VMThrowable into Throwable to get rid of VMThrowable. Same reasons as mentioned above. I can't see the point of this. Exceptions are slow anyway -- what's the cost of an extra indirection to walking the call stack? - Classpath's reflection implementation is/was incomplete and required modifications to work. I also modifed these classes to work with JamVM. Hope these don't sound like complaints - using Classpath has worked out great and I'm very appreciative of it!! I would second this. I've tried to use the VM interface as much as possible, to stay true to the goals of Classpath. Rob. _ Express yourself with cool emoticons - download MSN Messenger today! http://www.msn.co.uk/messenger ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Could you post your versions? It might be interesting to see whether we can adopt this approach as default in the vm/reference implementation. VMThread now does have a lot of native methods. But I believe I discussed some of these issues with Jeroen and if I remember correctly there were some subtle issues with just doing everything as wrappers over Object.wait()/notify(). The current approach allows the VM a lot of freedom. Trying to do too much in Java removes this, and in my opinion forces many VM implementors to abandon the VM interface altogether. You were talking about the VMThread implementation. I jumped to the conclusion that you meant Thread. Sorry. Rob. _ Express yourself with cool emoticons - download MSN Messenger today! http://www.msn.co.uk/messenger ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Answering Mark's question: Why does Jikes RVM override 11 non-VMFoo classes? (1) Native methods: For us, native methods are (1) lower performance and (2) can't be used early in the VM booting process. This is the primary reason for java.lang.Object, java.lang.reflect.Field, java.lang.reflect.Method, java.lang.reflect.Constructor. (2) java.lang.ref.*. Very little code in these classes that isn't VM specific. Don't see any reason why classpath should be expected to provide them, as they are really unlikely to be useful to anyone. Yes, Jikes RVM does implement this stuff...you have to do at least weak soft references if you want to run larger server-side applications. (3) java.lang.Class. If the Class/VMClass redesign with a VM-specific instance of type Object goes through, then we might be able to use it. (4) java.lang.Throwable. I don't see any hope for us ever using the classpath implementation. Handling exceptions is pretty delicate, and one of the implications of our entire VM being written in Java is that it is even more delicate than in a C based VM. We have lots of Jikes-RVM specific stuff here to deal with decent stack trace printing early in VM booting in other cases where the VM is sick or in a funny state and operating purely in Java won't work. (5) java.lang.Thread. We do m-n threading and some other strange stuff. Our thread class is kind of strange and really intertwined with other aspects of the VM. If we were starting from scratch, I think we could use the Thread/VMThread currently in classpath, but it would be a lot of work to retrofit. --dave___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Archie Cobbs writes: Andrew Haley wrote: I would like the vmdata field type then to be VMClass not Object. I disagree, as it imposes a restriction on what vmData actually is. The most obvious implementation of vmData is to be a byte[] instance holding the byte of a native pointer to an internal VM non-moveable data structure. I'm glad to see we agree (although I don't think it's at all obvious that it should be a byte[], not all VMs use native code). Eeeh. I can't imagine that either. If there's a strong argument for holding native pointer in a byte[] ? Here's my thinking on this topic. Warning: may not apply to you :-) Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. Indeed, but so does long. I suppose it's possible that on some weird platform a pointer mat not fit in long. In gcj was have a class RawData, which we know points to something that isn't an Object and isn't gcable. I remember kaffe had a kaffe.util.Pointer reference type that did this, and was specially recognized by the GC. Slightly better than Object imho. Yes, but you have to be very careful that such instances never leak into application code. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
I would be interested in a quick poll of VM implementors using Classpath: do you care more about having fewer diffs with stock Classpath or modifying optimizing your VM's core classes to eke out optimal performance? Both of course ;-) More seriously, I'm a little cautious when thinking about introducing an extra level of object instances in performance critical classes (or classes which have a lot of instances at runtime), but an extra level of static (or final virtual) calls should never be an issue. If you're in the performance game, your inliner has to eat up this kind of stuff with no problems, otherwise you have absolutely no chance of dealing with the application code. Effective automatic object-inlining (to optimize away levels of redundant objects) is still very much bleeding-edge research. Effective automatic inlining of static and virtual function calls is old, well-known technology and you have to do it and do it well to be in the OO performance game. --dave ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Mark Wielaard wrote: environment. By marking the VMWhatever classes as package local, final and naming them specially I had kind of hoped that Compilers/VMs would easily inline/optimize away extra calls and/or inline VM-specific fields in the non-VM specific instance. Maybe something for the future, but maybe it just won't happen. Right now JC doesn't inline, but hopefully soon it will. So then my reason for eliminating extra method calls will go away. Again, just goes to show that priorities are different for different VMs and even different versions of the same VM. - My Thread class uses private objects to implement sleep() and join() in terms of Object.wait(). The VM notify()'s this object when the thread exits. This means all the complexity of sleeping (and handling Thread.interrupt()) can be put in Object.wait() and not duplicated elsewhere. Could you post your versions? It might be interesting to see whether we can adopt this approach as default in the vm/reference implementation. VMThread now does have a lot of native methods. But I believe I discussed some of these issues with Jeroen and if I remember correctly there were some subtle issues with just doing everything as wrappers over Object.wait()/notify(). I am probably screwing up on the subtle issues :-) I did things this way to get Thread.interrupt() working and didn't think too hard about it. I'd be interested in hearing more about the subtle issues. FWIW here it is: http://cvs.sourceforge.net/viewcvs.py/*checkout*/jcvm/jcvm/java/java/lang/Thread.java?rev=1.2 -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Am Montag, 29. März 2004 17:12 schrieb Andrew Haley: Archie Cobbs writes: Andrew Haley wrote: I would like the vmdata field type then to be VMClass not Object. I disagree, as it imposes a restriction on what vmData actually is. The most obvious implementation of vmData is to be a byte[] instance holding the byte of a native pointer to an internal VM non-moveable data structure. I'm glad to see we agree (although I don't think it's at all obvious that it should be a byte[], not all VMs use native code). Eeeh. I can't imagine that either. If there's a strong argument for holding native pointer in a byte[] ? Here's my thinking on this topic. Warning: may not apply to you :-) Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. Indeed, but so does long. I suppose it's possible that on some weird platform a pointer mat not fit in long. In gcj was have a class RawData, which we know points to something that isn't an Object and isn't gcable. Yes, gcj has gnu.gcj.RawData. Classpath has gnu.classpath.RawData for the same purpose. It would be really nice to unify this. I tried once but I saw its a lot of work needed to change gnu.gcj.RawData to gnu.classpath.RawData (or gnu.lang.RawData or whatever). Michael ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Robert Lougher wrote: - My Thread class uses private objects to implement sleep() and join() in terms of Object.wait(). The VM notify()'s this object when the thread exits. This means all the complexity of sleeping (and handling Thread.interrupt()) can be put in Object.wait() and not duplicated elsewhere. Could you post your versions? It might be interesting to see whether we can adopt this approach as default in the vm/reference implementation. VMThread now does have a lot of native methods. But I believe I discussed some of these issues with Jeroen and if I remember correctly there were some subtle issues with just doing everything as wrappers over Object.wait()/notify(). I would be very unhappy about this. Sleep in JamVM is done by waiting on an internal monitor, which is what Object.wait() maps down to (the monitor associated with the Object) and joining threads wait on the vmthread associated with the thread, which is notified on exit. So all code is done in terms of monitor wait, which handles interrupt, so JamVM doesn't duplicate code either. JamVM uses thin locks which map down to a real monitor in the presence of contention. Join/sleep by definition requires a real monitor, so I side step the redundant monitor lookup. Hmm.. sounds like JamVM does the same thing as I described pretty much. Why then unhappy ? I must be missing something. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: I would like the vmdata field type then to be VMClass not Object. I disagree, as it imposes a restriction on what vmData actually is. The most obvious implementation of vmData is to be a byte[] instance holding the byte of a native pointer to an internal VM non-moveable data structure. I'm glad to see we agree (although I don't think it's at all obvious that it should be a byte[], not all VMs use native code). Eeeh. I can't imagine that either. If there's a strong argument for holding native pointer in a byte[] ? Here's my thinking on this topic. Warning: may not apply to you :-) Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. I remember kaffe had a kaffe.util.Pointer reference type that did this, and was specially recognized by the GC. Slightly better than Object imho. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Archie Cobbs wrote: Eeeh. I can't imagine that either. If there's a strong argument for holding native pointer in a byte[] ? Object is good because it is automatically the size of a pointer on any platform. However, it has one significant disadvantage, which is that you must special case all such fields in your garbage collector (unless you have a conservative collector). byte[] avoids this problem. OK. It seems people are missing an important point: This is polymorphic Java code. If you declare: Object vmData; You can later write: vmData = new byte[PTR_SIZE]; or vmData = new RawData(); or whatever. So what's the problem, with this? Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: vmData = new byte[PTR_SIZE]; or vmData = new RawData(); or whatever. So what's the problem, with this? And for those who want to do: vmData = [internalVMpointer] They can deal with it in many ways, such as: 1- make sure [internalVMpointer] points to a non-GC'ed memory, so that GC can identify it as special; 2- Set their internal compiler/interpreter to deal specially with all accesses to vmData (and use flow analysis to track the value propagation). I really think Object is the most general approach, as Classpath has decided to have a unique, good-for-all-vms source base approach. I was suggesting an template-based approach to solve the issue, but it was rejected as it was not pure Java. Object vmData IS pure Java, and it also conforms to Java's view of generecity (e.g. all Collection containers store and retrieve generic Objects). Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Andrew Haley wrote: [] I suppose it's possible that on some weird platform a pointer mat not fit in long. In gcj was have a class RawData, which we know points to something that isn't an Object and isn't gcable. Jikes RVM abstracts this away with a similar class, VM_Address. And, as you said, Andrew, it would be very bad if it leaked into application code. One day -- I have difficulty imagining it -- we are going to have processors that use an address space more than 64 bits wide. Then again, in 1984 the VAX's 32-bit (4 GB) address space seemed huge. Just as C is still in common use today, I expect that there will still be running Java code out there when we go over 64 bits of address space. I further expect that a descendant of at least one of the Free Java projects we discuss on this list will be out there running Java code. -- Steven Augart Jikes RVM, a free, open source, Virtual Machine: http://oss.software.ibm.com/jikesrvm ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Hi! We have our own version of the classes in java.lang.* so this dicussion is not vital for us. For us it is important that after the initial memory allocation there is no further memory allocation. So if we need to store native information we need to put it into the Java heap. To do this we allocate the required memory in the Java heap and mark it as a Java object of type Object. This allows us to handle this object like every other Java object. So we can store native information and connect it to a Java object and the garbage collector just frees the memory when the corresponding Java object is gone. So we would rather use Object references in such places than byte[]. Byte arrays give programmers on the Java side the feeling the can work with the content where they really should not touch it. Cheers, ingo -- Ingo Prötel [EMAIL PROTECTED] aicas GmbHhttp://www.aicas.com Haid-und-Neu-Str. 18phone +49 721 663 968-32 76131 Karlsruhe fax +49 721 663 968-93 Germany ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
--On Samstag, 27. März 2004 21:00 -0500 Etienne Gagnon [EMAIL PROTECTED] wrote: Note to Patrik: I thought you would have noticed, by now, that I was only trying to tell about a subset of the VMs that use Classpath without using 10 words. (See other messages for a better description). I used normal as in normally built. I did not intend to say better or worse. Hi Etienne, I want to apologize to you if my comment offended you; it was not meant that way. -Patrik ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Etienne Gagnon wrote: Hi all, As most people dislike the m4 approach without even having a look at it (just remembering m4 as used in auto* stuff...), I will no push for this further. I have no time for religious wars. I'd like to say that I wouldn't mind the m4 approach since I'm interested in sharing as much as possible of the necessary class library work between different VMs [1]. Alas, Kaffe is not ready for that yet, and it may take another few months till we switch completely to GNU Classpath for our class library. But then I'd be glad to do the guinea pig bit for such an integration. I also think providing as much as possible in pure java is the way to go, and such I guess I agree with most of what's been said here. cheers, dalibor topic [1] Other parts, too, of course. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Hi, On Thu, 2004-03-25 at 21:50, Jeroen Frijters wrote: David Lichteblau wrote: BTW, on the topic of VM* classes: Has there been agreement on java.lang.VMClass? The proposal was to make its methods static (and possibly add an Object vmdata field to java.lang.Class instead). I don't think anybody objected to this proposal. I'd definitely would like the vmdata field though. I would like the vmdata field type then to be VMClass not Object. So if I remember the discussion correctly the idea then is to not call the methods on the VMClass object but on static VMClass methods. Do these methods then get as parameter the Class object itself or the vmdata object? And more importantly, who wants to prototype this so we can have a real discussion about implementation issues? Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: Classpath build process and VM-specific issues
Hi, On Thu, 2004-03-25 at 22:54, Jeroen Frijters wrote: No, if you somehow get a Configuration.java (e.g. manually create it), you can build Classpath by simply running jikes (or another Java compiler). BTW. If someone could come up with something better for (most of) the Configuration.java.in thing that would be nice. Most of the things in there could be provided by the execution environment during runtime. Only the DEBUG flag seems really compile-time critical since it can remove some dead code. And maybe hard-coding the version number is a good thing. But the rest seems easily provided by the runtime during the bootstrap process. Volunteers for thinking/coming up with a better solution? Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath build process and VM-specific issues
Hi, On Sat, 2004-03-27 at 00:56, Steven Augart wrote: Etienne Gagnon wrote: [Some VMs, like JikesRVM tend to completely replace some base classes by their own; SableVM does not]. Jikes RVM's rvmrt.jar overrides exactly eleven non-VM* classes from a default Classpath installation's glibj.zip: java.lang.Class java.lang.Object java.lang.Thread java.lang.Throwable java.lang.ref.PhantomReference java.lang.ref.Reference java.lang.ref.SoftReference java.lang.ref.WeakReference java.lang.reflect.Constructor java.lang.reflect.Field java.lang.reflect.Method I like to know what prevents JikesRVM currently from sharing these classes. I hope to go to a model like SableVM apparently has were none of the core classes need to be overridden (even though some VMs might still opt to do it anyway). I had hoped that the VM interface for Class, Object, Thread and Throwable was usable for most VMs. What isn't in your case? I admit not to have looked into or thought about java.lang.ref support yet. How many VMs based on GNU Classpath properly implement those? java.lang.reflect should be refactored. I don't think there is a real reason for a VM to have specific versions of those (given properly designed VM interfaces of course). But I don't have had time yet to work on it. Hopefully some of the work done by the SableVM hackers will help here. Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
