RE : RE : memory-leaks in servlets, tool for tracing ?

2003-11-06 Thread Laurent Michenaud
When does a class reload occur ?

When u update a JSP ?
When u update a class ?
When u update a lib ?
When u do reload in the manager webapps ? = yes :)

Anything else ?

-Message d'origine-
De : Christopher Schultz [mailto:[EMAIL PROTECTED] 
Envoyé : mercredi 5 novembre 2003 23:31
À : Tomcat Users List
Objet : Re: RE : memory-leaks in servlets, tool for tracing ?

Laurent,
 What about classes with static method and/or static attributes ?
 
 Are they deleted from the old webapp ?

I don't believe that the VM ever releases resources taken up by Class 
objects (I think this includes static resources for a class). There used 
to be a VM option, -noclassgc, that was often used so that instance-less 
Classes wouldn't get GC'd. Almost everyone who ran Java apps turned that 
option on.

I'm not even sure if that option is still available. They may have 
simply eliminated the GC'ing of Class objects altogether.

I know that in my stock tc 4.1.27 on Sun's 1.4.2 VM, that after a 
re-load, all the old Class objects stick around. It's quite 
disconcerting because the heap grows by around 600 classes every time a 
context-reload occurs. The solution, of course, is not to enable 
context-reloading on production :)

-chris


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: RE : memory-leaks in servlets, tool for tracing ?

2003-11-06 Thread Shapira, Yoav

Howdy,

context-reload occurs. The solution, of course, is not to enable
context-reloading on production :)

This is very good advice.

Yoav Shapira



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: RE : RE : memory-leaks in servlets, tool for tracing ?

2003-11-06 Thread Christopher Schultz
Laurent,
When does a class reload occur ?
I was talking about a context re-load, which dumps everything in the 
context and re-starts it in a new ClassLoader.

When u update a JSP ?
Yes, the class gets re-loaded, but not the whole context. This is much 
less of a problem.

When u update a class ?
Sometimes is depende on the class you update. If you update a servlet 
that's mapped in web.xml, then Tomcat will definately re-load the 
context. Sometimes it seems like if I update a bean class that Tomcat 
doesn't re-load. I don't know it's checking technique.

When u update a lib ?
I don't think it re-loads the context for a lib update.

When u do reload in the manager webapps ? = yes :)
Of course!

It will also re-load if web.xml changes, and I've seen it re-load 
sometimes if anything directly in the WEB-INF directory changes (i.e. I 
have struts-config.xml in my current application, and if I change that, 
sometimes it reloads).

-chris

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


Re: memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Dirk Griesbach
Christopher,

thanks for your comprehensive response !
See more comments down ...

- Original Message -
From: Christopher Schultz [EMAIL PROTECTED]
To: Tomcat Users List [EMAIL PROTECTED]
Sent: Wednesday, October 29, 2003 3:29 PM
Subject: Re: memory-leaks in servlets, tool for tracing ?


 Grisi,

  our TC-based webapplication performs well but the java-processes
  concerned are showing increasing memory usage over time. For tracing
  we already stripped the app down to the very basic to get a clue.
  Wasn't successful enough.

 Have you looked at the memory over a long time, including several large
 GCs? I've had to do this in the past, and it's no fun without a
 profiler.

No, it isn't indeed. I've been checking LINUX' 'top' over a period of time.
A mem-hole is apparent. Unfortunately our app makes use of a
(self-developed)
JNI-Module (C) which makes it even more difficult because the C-code also
runs
in the Java-context.

Unfortunately, profiling tools can get expensive. Anyone know
 of any decent OSS ones out there?

'JProbe' and 'OptimizeIt!' are really no bargains !

 When I've done this sans profiler in the past, I did two things:

 1. Turn on verbose GC for the VM (dumps GC stats to stdout)
 2. Write a program to parse the GC lines and graph them over time

 We got a curve that looked like this:

   - max memory
  /\ /
 used memory-/\ /  \___/
  /\ /  \___/
 /  \___/
 --/ ^ OutOfMemoryError

 This indicated that we were really screwing up somewhere.

 Had the curve looked like this, we would be happy:

  - max memory
   /\   /\   /\
  /  \ /  \ /  \
 /\   /\   /\
/  \_/  \_/  \
 _/


 ... or even with high-frequency perturbations in there (usually from
 minor GCs happening periodically).

 It often helps to set the initial heap size and the maximum heap size to
 the same value (usually something like 256MB, 512MB, or 1024MB). Just
 remember that the higher it is, the longer you'll have to wait for a
 full GC.

 GCs don't always free everything they can. If they did, they'd take
 forever. It's only when the VM gets near its maximum heap size that the
 GC panics and goes on a collection rampage.

 If you ever get an OutOfMemoryError, go and get a thread-dump. On UNIX,
 you can sent the VM a STOP signal using kill or by pressing CTRL-\ if
 the VM is running in a terminal.

Good idea, I also achieved this by narrowing available mem on the machine
until java
gave up quitting with an 'out-of-mem' err.
The problem is that the dump written only contains the hex-adresses of the
modules being involved.
No possibilty to trace those adressesdown  to the functions or even modules
involved.
Even more difficult: the root cause given by this dump is not the
function(s)/method(s) causing the
memory problems.


 You'll get lots of good information including the number of threads and
 where they are. You might find that there are threads there that you
 thought had terminated long ago. Old active threads are always a source
 of tied-up memory.

  Does anybody's got experience with a profiling toolkit which she/he
  can suggest ?

There is a tool provided by SUN itself called 'HAT' (HeapAnalyzingTool)
http://java.sun.com/people/billf/heap/
but this seems to be only applicable to java programs, not servlets.
With Tomcat there were errors. May be in different cases this would help.


 I have some experience with Borland's OptimizeIt, and I've even recently
 installed it on Linux and run Tomcat 4.1 through it. I was able to
 determine something about the VM and Tomcat that might help you. I
 thought I had a memory leak, too.

In our case it was helpful to operate java with the -Xincgc option that
causes the
GC to collect smaller amounts of mem; but does this more often. It gives a
more
accurate impression in the current memory status.

What we did is the following:
- separate the JNI-module and write a little main()-frame around it to run
it alone.
- compile this with the 'dmalloc'-library to identify mem not being free'd
there.
- running the Servlet in one scenario (without JNI) and examine the few
methods being touched.
- rewrite java-code being in question.
- test it on mem-behavior.
- iterate with the next scenario etc.

Could be more satisfying, this kind of work, but it should be
straight-forward.

 It turns out that when you recompile your servlets and Tomcat does an
 auto-reload (I have it configured that way), a new ClassLoader gets
 installed to hold the classes loaded for the new context. However, the
 classes loaded from the old context stick around because the JVM doesn't
 want to discard java.lang.Class objects in case they're useful in the
 future.

 This increases the number of java.lang.Class objects by about 600 every
 time I recompiled. After many compile-deploy-reload cycles, my VM was
 hogging all my memory and lots

Re: memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Christopher Schultz
Grisi,

our TC-based webapplication performs well but the java-processes
concerned are showing increasing memory usage over time. For tracing
we already stripped the app down to the very basic to get a clue.
Wasn't successful enough.
Have you looked at the memory over a long time, including several large
GCs? I've had to do this in the past, and it's no fun without a
profiler.
No, it isn't indeed. I've been checking LINUX' 'top' over a period of time.
A mem-hole is apparent. Unfortunately our app makes use of a
(self-developed)
JNI-Module (C) which makes it even more difficult because the C-code also
runs in the Java-context.
Ooh, JNI is always a wild-card. I wrote some JNI code that we were using 
within a servlet container as well. I tested the hell out of it, 
especially with threaded use (that's very important). I could only get 
it to fail when it had trouble writing to files at very high thread 
rates (I made each thread write to files on a floppy disk, which would 
run out of space and cause an I/O error).

I'd highly recommend testing your native library separately from the use 
of JNI and the servlet. Try setting up a test harness for the native 
code to see if there are easier memory leaks to find. Adding JNI and the 
VM on top of all that it tough.

Some profiles claim to allow you to watch native code as well as Java 
code (which sounds really cool!), but again they are expensive. You can 
use gdb and clever use of tests that are likely to break your code 
without getting Java involved.


Unfortunately, profiling tools can get expensive. Anyone know
of any decent OSS ones out there?
I don't. Anyone have any ideas? Some intern at my old company wrote one 
from scratch, apparently, so I'm guessing they aren't TOO hard. He used 
the JVMDP or whatever, which is a debugging protocol that you use to 
attach to a running VM process.

'JProbe' and 'OptimizeIt!' are really no bargains !
Well, that depends on how much money you are going to spend to find the 
problem otherwise. How much do you (and your team) cost per hour to your 
employer? If it's going to take you a week (40 hours * x team members), 
then how does that compare to the cost of an OptimizeIt license...?

The problem is that the [thread] dump written only contains the hex-adresses of the
modules being involved.
No possibilty to trace those adressesdown  to the functions or even modules
involved.
Yeah, that's a problem. You'd have to do a lot of investigation to 
figure out what's actually being executed.

Even more difficult: the root cause given by this dump is not the
function(s)/method(s) causing the
memory problems.
Well, once you get an OutOfMemoryException, you pretty much can't trust 
anything the VM says after that. What you're seeing is just a guide; 
they might be total lies. :)

There is a tool provided by SUN itself called 'HAT' (HeapAnalyzingTool)
http://java.sun.com/people/billf/heap/
but this seems to be only applicable to java programs, not servlets.
With Tomcat there were errors. May be in different cases this would help.
Well, Tomcat is a Java program, so you should be able to use it, right? 
It's just a matter of figuring out how to hook it into your startup 
process. I hacked through the OptimizeIt instructions and created an ant 
target to start my Tomcat instance running inside of OptimizeIt. Very sexy.

In our case it was helpful to operate java with the -Xincgc option that
causes the
GC to collect smaller amounts of mem; but does this more often. It gives a
more
accurate impression in the current memory status.
What we did is the following:
- separate the JNI-module and write a little main()-frame around it to run
it alone.
- compile this with the 'dmalloc'-library to identify mem not being free'd
there.
- running the Servlet in one scenario (without JNI) and examine the few
methods being touched.
- rewrite java-code being in question.
- test it on mem-behavior.
- iterate with the next scenario etc.
Could be more satisfying, this kind of work, but it should be
straight-forward.
Well, that's the approach I'd take without a profiler.

Good luck. Let us know what you find.

-chris

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


RE: memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Shapira, Yoav

Howdy,

our TC-based webapplication performs well but the java-processes
concerned are showing increasing memory usage over time. For tracing

This basic idea that memory usage should be constant over time is very
common and usually wrong.  The JVM will use as much memory as it needs
without GC'ing until it has to.  So if you give it 1GB for an app that
needs 10MB, and keep that app running for long, you'd see memory usage
increasing and increasing until GC is fired.  Even then, the heap will
stay where it was, only the %free will be much higher.  The better
solution here is not to give your app so much memory, make GC run more
often, and track the %free instead of just total process image memory
via the top command.  You need to find it if there are memory leaks, of
if the memory increase is legitimate, e.g. due to high usage.

I'd highly recommend testing your native library separately from the
use

That's a very good suggestion.

 Unfortunately, profiling tools can get expensive. Anyone know
of any decent OSS ones out there?

The question comes up every couple of months on this list, search the
archives.  Personally, I think OptimizeIt is worth its price without a
doubt.

I don't. Anyone have any ideas? Some intern at my old company wrote one
from scratch, apparently, so I'm guessing they aren't TOO hard. He used
the JVMDP or whatever, which is a debugging protocol that you use to
attach to a running VM process.

It's easy to write bad profilers, each EXTREMELY difficult to write
profilers with low overhead.  It's the JVMPI (Java Virtual Machine
Profiling Interface) that you must implement.

Yoav Shapira



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Anthony E. Carlos
Yoav,

You make a great point about how the app should stabilize it's memory 
usage over time. However, I've got a question about memory usage when I 
stop (via Tomcat manager) and reload a webapp via a WAR file. If I 
understand your point, and I'm close to the max heap size, shouldn't GC 
free up the memory from the old webapp? In my case, GC happens more 
frequently, but doesn't do a great job (not even close to freeing up 
the memory footprint of my webapp). Eventually, I run into out of 
memory problems. Shouldn't a reload of a webapp cause a relase of the 
resources from the old webapp?

-Anthony Carlos

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


RE: memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Shapira, Yoav

Howdy,

You make a great point about how the app should stabilize it's memory
usage over time. However, I've got a question about memory usage when I
stop (via Tomcat manager) and reload a webapp via a WAR file. If I
understand your point, and I'm close to the max heap size, shouldn't GC
free up the memory from the old webapp? In my case, GC happens more
frequently, but doesn't do a great job (not even close to freeing up
the memory footprint of my webapp). Eventually, I run into out of
memory problems. Shouldn't a reload of a webapp cause a relase of the
resources from the old webapp?

A reload should release old resources when you restart the webapp, yes.
But it's your responsibility to ensure there's nothing in your
application which would stay in memory after a webapp restart.  For
example, classes in common/lib aren't affected by a restart, so if you
app puts stuff there, and then you restart your app, the stuff you put
in common/lib will still be there.

Yoav Shapira



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE : memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Laurent Michenaud
What about classes with static method and/or static attributes ?

Are they deleted from the old webapp ?

-Message d'origine-
De : Shapira, Yoav [mailto:[EMAIL PROTECTED] 
Envoyé : mercredi 5 novembre 2003 15:57
À : Tomcat Users List
Objet : RE: memory-leaks in servlets, tool for tracing ?


Howdy,

You make a great point about how the app should stabilize it's memory
usage over time. However, I've got a question about memory usage when I
stop (via Tomcat manager) and reload a webapp via a WAR file. If I
understand your point, and I'm close to the max heap size, shouldn't GC
free up the memory from the old webapp? In my case, GC happens more
frequently, but doesn't do a great job (not even close to freeing up
the memory footprint of my webapp). Eventually, I run into out of
memory problems. Shouldn't a reload of a webapp cause a relase of the
resources from the old webapp?

A reload should release old resources when you restart the webapp, yes.
But it's your responsibility to ensure there's nothing in your
application which would stay in memory after a webapp restart.  For
example, classes in common/lib aren't affected by a restart, so if you
app puts stuff there, and then you restart your app, the stuff you put
in common/lib will still be there.

Yoav Shapira



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE : RE : memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Laurent Michenaud
It would be good if some of you writes an kind of howTo
that shows :
- a webapp with a memory leak.
- the use of a profiler program ( with screenshots ) to detect the memory leak.

-Message d'origine-
De : Laurent Michenaud 
Envoyé : mercredi 5 novembre 2003 16:56
À : Tomcat Users List
Objet : RE : memory-leaks in servlets, tool for tracing ?

What about classes with static method and/or static attributes ?

Are they deleted from the old webapp ?

-Message d'origine-
De : Shapira, Yoav [mailto:[EMAIL PROTECTED] 
Envoyé : mercredi 5 novembre 2003 15:57
À : Tomcat Users List
Objet : RE: memory-leaks in servlets, tool for tracing ?


Howdy,

You make a great point about how the app should stabilize it's memory
usage over time. However, I've got a question about memory usage when I
stop (via Tomcat manager) and reload a webapp via a WAR file. If I
understand your point, and I'm close to the max heap size, shouldn't GC
free up the memory from the old webapp? In my case, GC happens more
frequently, but doesn't do a great job (not even close to freeing up
the memory footprint of my webapp). Eventually, I run into out of
memory problems. Shouldn't a reload of a webapp cause a relase of the
resources from the old webapp?

A reload should release old resources when you restart the webapp, yes.
But it's your responsibility to ensure there's nothing in your
application which would stay in memory after a webapp restart.  For
example, classes in common/lib aren't affected by a restart, so if you
app puts stuff there, and then you restart your app, the stuff you put
in common/lib will still be there.

Yoav Shapira



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: RE : RE : memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Shapira, Yoav

Howdy,
That'd be good, and I think there are some efforts in this area.  (Is Peter Lin's 
apache performance book available for online purchase?)  You can google for them.  
Another one with a specific app is a good idea, but requires a lot of time ;)

Yoav Shapira
Millennium ChemInformatics


-Original Message-
From: Laurent Michenaud [mailto:[EMAIL PROTECTED]
Sent: Wednesday, November 05, 2003 10:59 AM
To: Tomcat Users List
Subject: RE : RE : memory-leaks in servlets, tool for tracing ?

It would be good if some of you writes an kind of howTo
that shows :
- a webapp with a memory leak.
- the use of a profiler program ( with screenshots ) to detect the memory
leak.

-Message d'origine-
De : Laurent Michenaud
Envoyé : mercredi 5 novembre 2003 16:56
À : Tomcat Users List
Objet : RE : memory-leaks in servlets, tool for tracing ?

What about classes with static method and/or static attributes ?

Are they deleted from the old webapp ?

-Message d'origine-
De : Shapira, Yoav [mailto:[EMAIL PROTECTED]
Envoyé : mercredi 5 novembre 2003 15:57
À : Tomcat Users List
Objet : RE: memory-leaks in servlets, tool for tracing ?


Howdy,

You make a great point about how the app should stabilize it's memory
usage over time. However, I've got a question about memory usage when I
stop (via Tomcat manager) and reload a webapp via a WAR file. If I
understand your point, and I'm close to the max heap size, shouldn't GC
free up the memory from the old webapp? In my case, GC happens more
frequently, but doesn't do a great job (not even close to freeing up
the memory footprint of my webapp). Eventually, I run into out of
memory problems. Shouldn't a reload of a webapp cause a relase of the
resources from the old webapp?

A reload should release old resources when you restart the webapp, yes.
But it's your responsibility to ensure there's nothing in your
application which would stay in memory after a webapp restart.  For
example, classes in common/lib aren't affected by a restart, so if you
app puts stuff there, and then you restart your app, the stuff you put
in common/lib will still be there.

Yoav Shapira



This e-mail, including any attachments, is a confidential business
communication, and may contain information that is confidential,
proprietary and/or privileged.  This e-mail is intended only for the
individual(s) to whom it is addressed, and may not be saved, copied,
printed, disclosed or used by anyone else.  If you are not the(an) intended
recipient, please immediately delete this e-mail from your computer system
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: RE : memory-leaks in servlets, tool for tracing ?

2003-11-05 Thread Christopher Schultz
Laurent,
What about classes with static method and/or static attributes ?

Are they deleted from the old webapp ?
I don't believe that the VM ever releases resources taken up by Class 
objects (I think this includes static resources for a class). There used 
to be a VM option, -noclassgc, that was often used so that instance-less 
Classes wouldn't get GC'd. Almost everyone who ran Java apps turned that 
option on.

I'm not even sure if that option is still available. They may have 
simply eliminated the GC'ing of Class objects altogether.

I know that in my stock tc 4.1.27 on Sun's 1.4.2 VM, that after a 
re-load, all the old Class objects stick around. It's quite 
disconcerting because the heap grows by around 600 classes every time a 
context-reload occurs. The solution, of course, is not to enable 
context-reloading on production :)

-chris

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


RE: memory-leaks in servlets, tool for tracing ?

2003-10-29 Thread Shapira, Yoav

Howdy,
I like OptimizeIt's heap snapshots.

Yoav Shapira
Millennium ChemInformatics


-Original Message-
From: Dirk Griesbach [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 29, 2003 4:03 AM
To: Tomcat Users List
Subject: memory-leaks in servlets, tool for tracing ?

hi everybody,

our TC-based webapplication performs well but the java-processes
concerned
are showing increasing memory usage over time. For tracing we already
stripped the app down to the very basic to get a clue. Wasn't
successful
enough.

Does anybody's got experience with a profiling toolkit which she/he can
suggest ?

grisi



This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: memory-leaks in servlets, tool for tracing ?

2003-10-29 Thread Christopher Schultz
Grisi,

our TC-based webapplication performs well but the java-processes
concerned are showing increasing memory usage over time. For tracing
we already stripped the app down to the very basic to get a clue.
Wasn't successful enough.
Have you looked at the memory over a long time, including several large 
GCs? I've had to do this in the past, and it's no fun without a 
profiler. Unfortunately, profiling tools can get expensive. Anyone know 
of any decent OSS ones out there?

When I've done this sans profiler in the past, I did two things:

1. Turn on verbose GC for the VM (dumps GC stats to stdout)
2. Write a program to parse the GC lines and graph them over time
We got a curve that looked like this:

  - max memory
/\ /
used memory-/\ /  \___/
/\ /  \___/
   /  \___/
--/ ^ OutOfMemoryError
This indicated that we were really screwing up somewhere.

Had the curve looked like this, we would be happy:

 - max memory
 /\   /\   /\
/  \ /  \ /  \
   /\   /\   /\
  /  \_/  \_/  \
_/
... or even with high-frequency perturbations in there (usually from 
minor GCs happening periodically).

It often helps to set the initial heap size and the maximum heap size to 
the same value (usually something like 256MB, 512MB, or 1024MB). Just 
remember that the higher it is, the longer you'll have to wait for a 
full GC.

GCs don't always free everything they can. If they did, they'd take 
forever. It's only when the VM gets near its maximum heap size that the 
GC panics and goes on a collection rampage.

If you ever get an OutOfMemoryError, go and get a thread-dump. On UNIX, 
you can sent the VM a STOP signal using kill or by pressing CTRL-\ if 
the VM is running in a terminal.

You'll get lots of good information including the number of threads and 
where they are. You might find that there are threads there that you 
thought had terminated long ago. Old active threads are always a source 
of tied-up memory.

Does anybody's got experience with a profiling toolkit which she/he
can suggest ?
I have some experience with Borland's OptimizeIt, and I've even recently
installed it on Linux and run Tomcat 4.1 through it. I was able to
determine something about the VM and Tomcat that might help you. I
thought I had a memory leak, too.
It turns out that when you recompile your servlets and Tomcat does an
auto-reload (I have it configured that way), a new ClassLoader gets
installed to hold the classes loaded for the new context. However, the
classes loaded from the old context stick around because the JVM doesn't
want to discard java.lang.Class objects in case they're useful in the
future.
This increases the number of java.lang.Class objects by about 600 every
time I recompiled. After many compile-deploy-reload cycles, my VM was
hogging all my memory and lots of CPU. Perhaps this is your problem, too?
-chris

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]