We're experiencing a problem on both production and test systems that has only started to occur in the last few weeks (no new version of any software have been put live to anyone's knowledge) that has us quite stumped.
We are seeing tomcat starting up additional copies of itself, each new copy is allocating a chunk of storage, it only starts 1 thread, nothing gets written to any logs and no CPU is being listed as having been used. This keeps happening, and if it's not monitored the system memory all gets taken up and we start to get SIGBUS errors. We've tried turning all logging on, there don't appear to be any software related events at the heart of the issue, and we can't see any system triggers. When we list the system tasks we see that the ghost process PPID is the same as the PID of the still running main tomcat instance. Here is list of what happened when it wasn't picked up for a while. tomcat 21115 21114 0 14:46:44 ? 193:04 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 773 21115 0 17:29:27 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 4270 21115 0 18:29:29 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 5988 21115 0 18:59:32 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 27209 21115 0 01:01:02 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 13155 21115 0 05:32:09 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 16785 21115 0 06:34:52 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 12948 21115 0 20:58:50 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 20054 21115 0 23:00:21 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti tomcat 23730 21115 0 00:03:47 ? 0:00 /usr/java/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat -5.5.27-ti The main process 21115 starts off fine, and is even still running with no problems the next day, but we keep getting these ghost processes (for want of a better description). We have multiple tomcat servers running on the same system, but this is the only one experiencing the problem, all of the others are unaffected. The web apps themselves are pretty standard, servlets, JSPs, web services, Oracle DB connection and a JMX link to an Apache MQ server for monitoring purposes. This isn't even a heavily used application. The web has come up empty for us, we can't find any information on this fault at all, so would really appreciate any help you may be able to give us. Regards Dave
