Re: [kaffe] Compare kaffe and IBMJava2-131 Java Virtual Machine.
hi Greg, --- Greg Wooledge [EMAIL PROTECTED] wrote: I also had to add java/io/CharArrayWriter to essential.files to get it to build the javalib. I used jikes, so I didn't notice that. Then I stopped it by pressing Ctrl-C in the window where I had started it. I don't know if this was supposed to cause some sort of extra information to be printed in the output file; if you need me to do it again and terminate Freenet in some other way, let me know. My code plants a hook into Runtime.exit in order to print out the object counter table. So exiting via Ctrl-C is a short-cut that jumps over my code. We'll need to do this again, unfortunately. If there is no way the freenet daemon can not be told to 'exit', then I can ammend the patch with a thread that regularly dumps the information from the trace map. the patch should print its output to System.out when the program calls Runtime.exit(). cheers, dalibor topic __ Do you Yahoo!? Yahoo! Web Hosting - establish your business online http://webhosting.yahoo.com ___ kaffe mailing list [EMAIL PROTECTED] http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] Jit3 for IA-64
Hi! A bit late but here comes the requested ChangeLog entries. Svante - Original Message - From: Dalibor Topic [EMAIL PROTECTED] To: Svante Arvedahl [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, February 24, 2003 8:46 PM Subject: Re: [kaffe] Jit3 for IA-64 Hi Svante, --- Svante Arvedahl [EMAIL PROTECTED] wrote: BTW: any news from the JIT front? Yes, I finished my master's thesis in december where I ported the jit3 to IA-64. A patch and my report can be found at http://hem.passagen.se/svaar968/ that's very cool, thanks! patch (in config/ia64, somewhere). The patch applies to Gwenole's CVS snapshot from august of 2001. Could you post a ChangeLog entry for the modified files? cheers, dalibor topic __ Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, more http://taxes.yahoo.com/ File: config/ia64/fixme.txt Change: New file. Accounts things to be fixed in the ia64 jit3 backend. File: config/ia64/heuristic2/datastructures.c Change: New file. Primitive functions used in Arvedahl's instruction scheduling heuristic File: config/ia64/heuristic2/datastructures.h Change: New file. Datastructures used in Arvedahl's instruction scheduling heuristic. Based on a java implementation by Samuel Sanseri -- [EMAIL PROTECTED] File: config/ia64/heuristic2/heuristic2.c Change: New file. Functions used in Arvedahl's instruction scheduling heuristic. File: config/ia64/heuristic2/heuristic2.h Change: New file. Primitive functions used in Arvedahl's instruction scheduling heuristic. File: config/ia64/heuristic2/heuristic2_defines.h Change: New file. Primitive functions used in Arvedahl's instruction scheduling heuristic. Based on a java implementation by Samuel Sanseri -- [EMAIL PROTECTED] File: config/ia64/heuristic2/heuristic2_macros.h Change: New file. Macros used in Arvedahl's instruction scheduling heuristic. Based on a java implementation by Samuel Sanseri -- [EMAIL PROTECTED] File: config/ia64/heuristic2/ia64_isa.defs Change: New file. The IA-64 version of the ppc defs file, originally developed by the Flux group at the University of Utah. To be run through mnemonicizer.awk. File: config/ia64/heuristic2/ia64_isa.h Change: New file. Macros used in Arvedahl's instruction scheduling heuristic. Produced by the script mnemonicizer.awk. File: config/ia64/heuristic2/mnemonicizer.awk Change: New file. The IA-64 version of the ppc mnemonicizer, originally developed by the Flux group at the University of Utah. Interprets the defs file ia64_isa.defs, filled with definitions for the IA-64 architecture. File: config/ia64/heuristic2/schedule.c Change: New file. Functions used in Arvedahl's instruction scheduling heuristic. Based on a java implementation by Samuel Sanseri -- [EMAIL PROTECTED] File: config/ia64/itanium.h Change: New file. Itanium specific JIT configuration information. File: config/ia64/jit.h Change: New file. Common IA-64 JIT configuration information. File: config/ia64/jit3-ia64.def Change: New file. IA-64 instruction definitions. File: config/ia64/jit3-icode.h Change: New file. Defines the instructions which are present on the ia64. File: config/ia64/linux/jit-md.c Change: New file. The functions in this file are ugly hacks used in stack unwinding. They should be replaced by proper stack unwinding. Basically they search the memory for frames they recognize. File: config/ia64/linux/jit3-md.h Change: New file. Linux IA64 JIT configuration information. File: config/ia64/linux/md.h Change: Added include of jit3-md.h when the jit is used (TRANSLATOR is defined). File: config/ia64/sysdepCallMethod.h Change: Added support for the stack unwinding hack in ia64/linux/jit-md.c. File: config/ia64/trampolines.c Change: New file. Contains the ia64_do_fixup_trampoline used in the jit3 backend. File: developers/gdbinit1 Change: Added a few gdb debug features. File: include/jtypes.h.in Change: Ensures that jword gets the appropriate size depending on size of voidp. File: kaffe/kaffe/main.c Change: Added a vmdebug flag: TIME, that uses clock() to measure the time between entering and exiting function main2. File: kaffe/kaffevm/classMethod.c Change: Oops, no changes made here... File: kaffe/kaffevm/debug.c Change: Added two vmdebug flags: TIME, which shows time between entry and exit of main2, and VLIW which prints VLIW instruction packing information. File: kaffe/kaffevm/debug.h Change: Added two vmdebug flags: TIME, which shows time between entry and exit of main2, and VLIW which prints VLIW instruction packing information. File: kaffe/kaffevm/gc.h Change: Added a gc alloc type: GC_ALLOC_VLIW. File: kaffe/kaffevm/gcFuncs.c Change: Added a gc alloc type: vliw (GC_ALLOC_VLIW). File: kaffe/kaffevm/jit3/basecode.c Change: Changed all u[x].value.i to u[x].value.w to get the appropriate size, since this field is often used for pointers. File:
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx and -ss switches.)
hi Mark,
--- Mark J Roberts [EMAIL PROTECTED] wrote:
I am sick and tired of manually overriding the heap
size limit in
order to run Freenet without hitting the arbitrary
default 64MB
limit.
I think that's due to a bug in kaffe ('s class
library) as JDK seems to be able to run Freenet in
30M, or so, according to what Greg posted here. I'm
trying to track that one down with Greg, but it would
be helpful if you joined in, as a Freenet developer.
I've looked at the gcstats outputs provided by Greg,
and most of the Objects lying around are either
java.lang.Objects or java.util.HashMap$Entries, with a
bunch of freenet.fs.dir.* Objects following. Grepping
throuh the source indicates that java.lang.Objects are
used by freenet for synchronization, right?
My prime suspect at the moment are the
HashMap.Entries, so I posted a patch to track down
who's creating all these HashMap.Entries, hoping that
would provide some clue where they are used, and why
they don't go away ;)
The idea is to create a stack trace for every
constructed instance of HashMap.Entry and map the
trace to the number of instances created with the same
stack trace. So the HashMap.Entry constructor
increases the counter for its stack trace, or inserts
it into the map.
The patch has (at least) one problem, it relies on
Runtime.exit() to print the map of stack traces.
Apparently it seems to be hard to get freenet to
exit(). So I'm not sure how to proceed from here: if
it is possible to tell a freenet node to shut down
through exit(), that would be helpful, otherwise I
could add a thread that prints out the map in regular
intervals.
The other course of action would be to replace kaffe's
HashMap (if that's the culprit) with another
implementation, let's say Classpath's, and see if that
yields any benefits.
I just don't understand why these options are even
there, or why
they are not unlimited by default. It is _breaking_
applications.
In theory, they are not necessary. In practice, it's
rather unconvenient to have a maximum memory setting
higher that the amount of available RAM. In that case,
the gc might spend a lot of time asking the virtual
memory manager to shuffle pages around, and that
degrades performance severely. The gc kicks in when a
certain percentage of memory is used.
The 64Mb limit is the same that Sun's using on the JDK
1.4.1 for example. See
http://java.sun.com/j2se/1.4/docs/tooldocs/linux/java.html
under -Xmx . So if you're getting problems because
your application is running out of memory on kaffe,
but works fine on Sun's JDK, then that's a bug in
kaffe, in my opinion. Setting -mx to unlimited by
default would just mask it ;)
Additionally, here's a blurb on usefullness of the
java heap setting from the Java Server documentation (
http://salamander.merit.edu/system/doc/administration/performance.html
):
The heap limit helps determine when a garbage
collection (GC) will take place. You can set the limit
high to avoid a GC, but then you could end up paging.
Generally, we have found that it is better to take the
hit from a GC fairly often and to avoid paging. The
other thing to keep in mind about garbage collection
is that the entire server stops while it is going. So
if you have an occasional, very long GC, the clients
will hang for that time. That leads to more
variability in service quality than more frequent but
smaller GCs.
In short: let's fix the bug ;)
cheers,
dalibor topic
__
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com
___
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] java.lang.ArrayIndexOutOfBoundsException thrown by Stack.push()
Hi Mark, --- Mark J Roberts [EMAIL PROTECTED] wrote: Stack.push() should not be throwing this exception. Looks like an obvious bug. Any ideas? Looks like a race condition bug to me. Could you make Vector.addElement synchronized, and see if that helps? My assumption is that between the size() call in addElement and the call to insertElementAt the Vector size changes so we get the bug. cheers, dalibor topic __ Do you Yahoo!? Yahoo! Web Hosting - establish your business online http://webhosting.yahoo.com ___ kaffe mailing list [EMAIL PROTECTED] http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
[kaffe] Kaffe CVS: kaffe stack
CVSROOT:/cvs/kaffe Module name:kaffe Changes by: stack 03/03/14 08:24:28 Modified files: . : ChangeLog kaffe/kaffevm/jit3: machine.c Added files: test/regression: FloatAlias.j Log message: 2003-03-14 Tim Stack [EMAIL PROTECTED] * kaffe/kaffevm/jit3/machine.c: Aggressively spill Rreadonce registers in slotAlias, otherwise they might not get spilled at all. * test/regression/FloatAlias.j: Test aliased float problems on x86. (Not integrated into the test suite yet.) Reported by: Ito Kazumitsu [EMAIL PROTECTED] ___ kaffe mailing list [EMAIL PROTECTED] http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] Simple program shows strange results
hi,
In message Re: [kaffe] Simple program shows strange results
on 03/03/05, Ito Kazumitsu [EMAIL PROTECTED] writes:
I have made a simpler test program:
This strange problem occurs with float and double, but not
with int or byte.
$ cat Test.java.m4
public class Test {
public static void main(String[] args) {
T x = C1;
if (x C2) {
System.err.println(xC2: + x);
x = C2;
}
System.err.println(x);
}
}
CVS update and try this again. (Thanks for the test case!)
If you're interested, the causes were:
I modified move_float() in icode.c to look like
move_int/move_ref/etc since it was generating inefficient code on
the PowerPC.
Unfortunately, the x86 jitter depended on the way the old
move_float() worked, so it was not working properly when a slot
was aliased. For example, in the following bytecode:
ldc 10.0
fstore_1
fload_1
Kaffe optimizes away the store and subsequent load since it is
potentially redundant. Unfortunately, because of the bizarro way
the x86 FPU works, this does not work and the 10.0 value never
gets written to the proper place on the stack. Therefore, the
floating point number that gets printed is just whatever garbage
is on the stack at the time.
Solution:
Modify slotAlias() in machine.c to aggressively spill Rreadonce
registers (the x86 float register). This method works and seems
to generate the same code as before on x86 (as far as I know),
without screwing over CPUs with more sensible FPUs.
thanks,
tim stack
___
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx and -ss switches.)
Dalibor Topic: The patch has (at least) one problem, it relies on Runtime.exit() to print the map of stack traces. Apparently it seems to be hard to get freenet to exit(). So I'm not sure how to proceed from here: if it is possible to tell a freenet node to shut down through exit(), that would be helpful, otherwise I could add a thread that prints out the map in regular intervals. I'll hack something together and reply with that stack trace map as soon as I have the time. I suspected that Sun's implementation also had the memory limit. I haven't used a proprietary JVM in ages, so I can't say whether Kaffe is much worse. Freenet is a really bloated program no matter how you run it, in any case. It's entirely reasonable that a user would _want_ his OS to page out data. The working set nearly always constitutes only a small share of a program's allocated memory. I don't see why the frequency of garbage collection relates to this--that blurb explicitly suggests that reducing its frequency through allowing more slack is a false optimization. What's more, if any JVM were crazy enough to wait until the heap limit is hit before invoking the GC (helps determine when a garbage collection (GC) will take place), even lightweight programs will consume an average of half the heap limit in memory. So I'm not at all persuaded that Sun is doing anything remotely sane with that limit, or that Kaffe needs to fall into the same trap. ___ kaffe mailing list [EMAIL PROTECTED] http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx and -ss switches.)
Mark J Roberts: I'll hack something together and reply with that stack trace map as soon as I have the time. I called System.exit() when Freenet was using over 150MB of memory. The output is attached. log.bz2 Description: Binary data
Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx and -ss switches.)
On Fri, Mar 14, 2003 at 10:30:14PM -0600, Mark J Roberts wrote: Mark J Roberts: I'll hack something together and reply with that stack trace map as soon as I have the time. I called System.exit() when Freenet was using over 150MB of memory. The output is attached. It looks like you have some sort of infinite looping problem. You should check and not create a Throwable if the object being tracked is a Throwable :) -- Matthew Toseland [EMAIL PROTECTED]/[EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://80-192-4-36.cable.ubr09.na.blueyonder.co.uk:8889/0bIANZmR~K8/ ICTHUS. pgp0.pgp Description: PGP signature
[: Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx and -ss switches.)]
- Forwarded message from -
To: Dalibor Topic [EMAIL PROTECTED]
Subject: Re: [kaffe] What is the rationale for Kaffe's VM memory limits? (The -mx
and -ss switches.)
On Fri, Mar 14, 2003 at 05:55:21AM -0800, Dalibor Topic wrote:
hi Mark,
--- Mark J Roberts [EMAIL PROTECTED] wrote:
I am sick and tired of manually overriding the heap
size limit in
order to run Freenet without hitting the arbitrary
default 64MB
limit.
I think that's due to a bug in kaffe ('s class
library) as JDK seems to be able to run Freenet in
30M, or so, according to what Greg posted here. I'm
trying to track that one down with Greg, but it would
be helpful if you joined in, as a Freenet developer.
I've looked at the gcstats outputs provided by Greg,
and most of the Objects lying around are either
java.lang.Objects or java.util.HashMap$Entries, with a
bunch of freenet.fs.dir.* Objects following. Grepping
throuh the source indicates that java.lang.Objects are
used by freenet for synchronization, right?
My prime suspect at the moment are the
HashMap.Entries, so I posted a patch to track down
who's creating all these HashMap.Entries, hoping that
would provide some clue where they are used, and why
they don't go away ;)
The idea is to create a stack trace for every
constructed instance of HashMap.Entry and map the
trace to the number of instances created with the same
stack trace. So the HashMap.Entry constructor
increases the counter for its stack trace, or inserts
it into the map.
Well, my immediate suspicion would be that: every once in a while,
freenet writes out the datastore index. In order to minimize the time
the datastore is locked for, we clone the index in a lock, and then
write it out from the clone. HashMap.clone() clones the individual items
as well as the table, so we end up with a whole lot of rather short
lived objects. If some way could be found to make the clone() lazily
copy (refcounts?), this would greatly reduce the GC/allocation stress.
Freenet on Kaffe still suffers from mysterious unexplained pauses in
execution that cause much higher routingTime's (a very bad thing, it
causes the node to assume it is overloaded and reject requests) than
Sun; maybe this is caused by a combination of this and the fact that
Kaffe uses a monolithic GC that locks the whole VM.
The patch has (at least) one problem, it relies on
Runtime.exit() to print the map of stack traces.
Apparently it seems to be hard to get freenet to
exit(). So I'm not sure how to proceed from here: if
it is possible to tell a freenet node to shut down
through exit(), that would be helpful, otherwise I
could add a thread that prints out the map in regular
intervals.
The other course of action would be to replace kaffe's
HashMap (if that's the culprit) with another
implementation, let's say Classpath's, and see if that
yields any benefits.
I just don't understand why these options are even
there, or why
they are not unlimited by default. It is _breaking_
applications.
In theory, they are not necessary. In practice, it's
rather unconvenient to have a maximum memory setting
higher that the amount of available RAM. In that case,
the gc might spend a lot of time asking the virtual
memory manager to shuffle pages around, and that
degrades performance severely. The gc kicks in when a
certain percentage of memory is used.
The 64Mb limit is the same that Sun's using on the JDK
1.4.1 for example. See
http://java.sun.com/j2se/1.4/docs/tooldocs/linux/java.html
under -Xmx . So if you're getting problems because
your application is running out of memory on kaffe,
but works fine on Sun's JDK, then that's a bug in
kaffe, in my opinion. Setting -mx to unlimited by
default would just mask it ;)
Additionally, here's a blurb on usefullness of the
java heap setting from the Java Server documentation (
http://salamander.merit.edu/system/doc/administration/performance.html
):
The heap limit helps determine when a garbage
collection (GC) will take place. You can set the limit
high to avoid a GC, but then you could end up paging.
Generally, we have found that it is better to take the
hit from a GC fairly often and to avoid paging. The
other thing to keep in mind about garbage collection
is that the entire server stops while it is going. So
if you have an occasional, very long GC, the clients
will hang for that time. That leads to more
variability in service quality than more frequent but
smaller GCs.
In short: let's fix the bug ;)
cheers,
dalibor topic
__
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com
___
kaffe mailing list
[EMAIL PROTECTED]
http://kaffe.org/cgi-bin/mailman/listinfo/kaffe
--
Matthew Toseland
[EMAIL PROTECTED]/[EMAIL PROTECTED]
Full time freenet hacker.
http://freenetproject.org/
Freenet
[kaffe] Freenet slowness dominated by Sun's slow implementation of modPow()?
It looks like the dominating factor in the crypto in authorizeTime is BigInteger.modPow() (a JVM-provided method, which really ought to be fast...). I'm seeing an average time for modPow() of 1412ms (it seems to be increasing...). With Sun 1.4. Now, with Kaffe, which uses libgmp, averages are closer to 59ms-75ms. I will run it on Kaffe overnight to see what happens. Possibilities: A) Fix remaining Kaffe problems (kaffe has monolithic GC, causing longish delays from time to time which lock the whole VM, but there is a kaffe derivative that uses Boehm incremental GC which could be merged; kaffe seems to get longer lock times suggesting maybe its locking is very heavy...), bundle Kaffe with Freenet (even on the Win32 version - this should be possible, I believe there is a port). Grumble if running a Sun JVM, but still run. B) Call out to an external, platform specific helper app if available (grumble loudly if it isn't there). With times of a second or more, this is probably still faster than using Sun's slow code. C) Any other suggestions? We really should do something about this is 0.5.2 - shaving 900ms off average authorizeTime's/connectingTime's is not something to be ignored. BTW, the theory: Kaffe uses libgmp, which is very very fast. Sun uses some apparently badly written in-house BigInteger code. -- Matthew Toseland [EMAIL PROTECTED]/[EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://80-192-4-36.cable.ubr09.na.blueyonder.co.uk:8889/J3Q~LkZ7ezk/ ICTHUS. pgp0.pgp Description: PGP signature
