Hello,
we are using Tomcat 5.0.27 to provide the eLearning environment of our University.
Our system configuration is as follows:
SUN Fire 240 with 2 cpus at 1.28 GHz (Sparc-3) 8 GB RAM Solaris 9 (with actual patches)
All requests are handled via Apache 2.0.52 with mod_ssl and using mod_jk2.
The system works well most of the time, but seemingly randomly the catalina process starts to use up
more and more cpu performance without actually doing anything useful! Memory usage stays normal/stable,
so we analyzed the process and found out the following:
- The cpu performance is used up primarily by the thread "VM Thread".
- This Thread is running nothing else but "synchronization primitives", which where called very fast an in high numbers without reading or writing anything (as if it is in some kind of endless loop of doing nothing).
- The effect (and an symptome which can be monitored repeatedly) is that the number of threads that the catalina process is using rises with every new request, because no thread gets enough cpu power to come to an end. "VM Thread" is taking away anything after a short period of time.
To come to this information, we did the following:
We monitored the machine with prstat -L to see when the cpu usage of the catalina process rose to a unusual high value (5% at average, rising up to about 60% when VM Thread got into its frenzy). So we could see that one single lwp was going into the high cpu usage.
As soon as we could see the rise in cpu usage we used pstack to find out which thread is responsible for the lwp-id that was shown by prstat -L.
Then, we used kill -3 to create a thread-dump of the catalina-process and searched for the thread-number that was stated by pstack.
That thread-dump showed us that the thread called "VM Thread" seems to be the problematic thread.
As far as we found out that problems occured several times with Tomcat 5 for other users in the world. However, so far no one offered an explanation what is going wrong there and what could be done to avoid that problem.
We now have prepared a monitoring service that kills and restarts the catalina engine as soon as the problem occurs, but that solution is not a good one.
Any suggestions of how to solve the problem are very, very welcome. If you need more information or have any idea how we should analyze the situation more properly, I would be happy to hear of it.
Thank you in advance,
Oliver Schoenwald FernUniversitaet Hagen Germany
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
