Tomcat CPU 100%
Hi all, We have a java web application (deployed on Tomcat 5.5 with java 1.4) that has never had CPU problems. Some days ago we deployed a new version, that had non significant differences, and since this deploy we got frequent CPU 100% usage. users could not work and we needed to rollback to previuos version. The unique structural difference is the upgrade of the following libraries, used to generate jasperReports: - commons-beanutils-1.5 -- 1.7 - groovy-all-1.0-beta-10 -- groovy-all-1.0 - jasperreports-1.2.0 -- 1.3.3 - jdt-compiler -- 3.1.1 - added jasperreports-extensions-1.3.1 with the -Xrs option we could not get info about the hanging thread, so yesterday we deployed the new version, with old libraries and without -Xrs option. Now we get no CPU usage problem. Can you help us understand what the real problem could be? Thanks in advance -- View this message in context: http://www.nabble.com/Tomcat-CPU-100--tp22617067p22617067.html Sent from the Tomcat - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat CPU 100%
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 MSerraInsiel, On 3/20/2009 5:40 AM, MSerraInsiel wrote: Hi all, We have a java web application (deployed on Tomcat 5.5 with java 1.4) that has never had CPU problems. Take thread dumps while the CPU is pegged. You didn't mention your Java version, so here goes: - - If you have Java 1.5+, you should have the 'jstack' program. Run that targeting the pid of the top-level Java process and you should get a complete thread dump. OR - - On UNIX, send a SIGSTOP to the main java process and look at the output in catalina.out. OR - - On Windows, you'll need to press CTRL-\ when running Tomcat from the command-line. I'm not sure how else to get a thread dump from java on Windows, so if you're running it as a service, you might have to stop Tomcat, re-start it from the command line (startup.bat) and then press CTRL-\ after your CPU goes high. Take thread dumps a few seconds apart, then examine them. Threads that say BLOCKED are not using any CPU (at least, not at the time the thread dump was taken). Anything that says RUNNABLE should be inspected to see what it's doing. Post your thread dumps to the list if you can't figure out what's going on. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAknDqpsACgkQ9CaO5/Lv0PAWEwCfcQ3n9YoOFKElIWqdgjf01WHZ U8EAn32cdrguG0RUOTGoUCm6vJZV9ozr =woZD -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat CPU 100%
java is 1.4.2, the SO is Linux, but we could not dump the comsuming thread because of teh -Xrs option. Now we removed it, but we have old libraries and no CPU problem. In an identical test environment the same problem does not arise, even with -Xrs option and new libraries version. Any ideas? Thanks Christopher Schultz-2 wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 MSerraInsiel, On 3/20/2009 5:40 AM, MSerraInsiel wrote: Hi all, We have a java web application (deployed on Tomcat 5.5 with java 1.4) that has never had CPU problems. Take thread dumps while the CPU is pegged. You didn't mention your Java version, so here goes: - - If you have Java 1.5+, you should have the 'jstack' program. Run that targeting the pid of the top-level Java process and you should get a complete thread dump. OR - - On UNIX, send a SIGSTOP to the main java process and look at the output in catalina.out. OR - - On Windows, you'll need to press CTRL-\ when running Tomcat from the command-line. I'm not sure how else to get a thread dump from java on Windows, so if you're running it as a service, you might have to stop Tomcat, re-start it from the command line (startup.bat) and then press CTRL-\ after your CPU goes high. Take thread dumps a few seconds apart, then examine them. Threads that say BLOCKED are not using any CPU (at least, not at the time the thread dump was taken). Anything that says RUNNABLE should be inspected to see what it's doing. Post your thread dumps to the list if you can't figure out what's going on. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAknDqpsACgkQ9CaO5/Lv0PAWEwCfcQ3n9YoOFKElIWqdgjf01WHZ U8EAn32cdrguG0RUOTGoUCm6vJZV9ozr =woZD -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- View this message in context: http://www.nabble.com/Tomcat-CPU-100--tp22617067p22621751.html Sent from the Tomcat - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat CPU 100%
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 MSerraInsiel, On 3/20/2009 10:48 AM, MSerraInsiel wrote: java is 1.4.2, the SO is Linux, but we could not dump the consuming thread because of the -Xrs option. Now we removed it, but we have old libraries and no CPU problem. In an identical test environment the same problem does not arise, even with -Xrs option and new libraries version. Presumably, a set of upgraded libraries is the goal. Are you willing to reproduce the circumstances again and take thread dumps? If not, and you have an identical test environment where everything works fine, I'd say you have a new production environment :) - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAknDrmIACgkQ9CaO5/Lv0PBWpwCgi6fOvUR6nZwZW4LVu84/sbcm eNwAn1ZAQuWKQdwSgPvQnCS/ZFpssq4n =aRpq -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat CPU 100%
MSerraInsiel wrote: Hi all, We have a java web application (deployed on Tomcat 5.5 with java 1.4) that has never had CPU problems. Some days ago we deployed a new version, that had non significant differences, and since this deploy we got frequent CPU 100% usage. users could not work and we needed to rollback to previuos version. The unique structural difference is the upgrade of the following libraries, used to generate jasperReports: - commons-beanutils-1.5 -- 1.7 - groovy-all-1.0-beta-10 -- groovy-all-1.0 Latest stable Groovy is 1.6. I found 1.0 doing all sorts of things I'd rather it didn't, later versions are considerably improved. p - jasperreports-1.2.0 -- 1.3.3 - jdt-compiler -- 3.1.1 - added jasperreports-extensions-1.3.1 with the -Xrs option we could not get info about the hanging thread, so yesterday we deployed the new version, with old libraries and without -Xrs option. Now we get no CPU usage problem. Can you help us understand what the real problem could be? Thanks in advance - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat CPU 100%
Hi, Andrea, Look at this: http://iusr.spaces.live.com/blog/cns!69F4725ED815E770!408.entry A summary of mine after resolving the same problem. It's quite clear from the thread dump to tell which thread is running into a JSP page. Cheers, Jerome On Wed, May 7, 2008 at 10:19 PM, Andrea Di Muro [EMAIL PROTECTED] wrote: Hello everyone, I have a Tomcat server with several domains configured and sometimes I can see that the tomcat process uses 100-190% (dual processor) and hangs up. I think that one of my customer has uploaded a jsp page that makes an infinite loop. How can I check which page causes the problem? Thank you Andrea Di Muro Technical Developer [EMAIL PROTECTED] Headquarter: Viale dei Mille, 111 - 50131 Firenze Telefono +39 055 40 89 084 Siena: Via Senese 155 - 53036 Poggibonsi Telefono +39 0577 985 562 - Fax +39 0577 991 591 http://www.virtualcom.it Informativa Privacy (ex D.lgs 196/03) Le informazioni contenute nel presente documento e relativi allegati possono essere riservate e sono destinate esclusivamente alla persona od alla Società indicata come destinatario. La diffusione e la distribuzione del presente documento a soggetti diversi da quelli indicati, od in generale qualsivoglia utilizzo illecito dei dati ivi contenuti, è proibita sia ai sensi dell'art. 616 del Codice Penale che dal D. Lgs 196/03 in materia di protezione dei dati personali (Privacy). Se avete ricevuto per errore questo documento siete pregati di distruggerlo e di comunicarcelo prontamente tramite e-mail o fax. -- Houston, we have a problem.
Tomcat CPU 100%
Hello everyone, I have a Tomcat server with several domains configured and sometimes I can see that the tomcat process uses 100-190% (dual processor) and hangs up. I think that one of my customer has uploaded a jsp page that makes an infinite loop. How can I check which page causes the problem? Thank you Andrea Di Muro Technical Developer mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] Headquarter: Viale dei Mille, 111 - 50131 Firenze Telefono +39 055 40 89 084 Siena: Via Senese 155 - 53036 Poggibonsi Telefono +39 0577 985 562 - Fax +39 0577 991 591 http://www.virtualcom.it/ http://www.virtualcom.it Informativa Privacy (ex D.lgs 196/03) Le informazioni contenute nel presente documento e relativi allegati possono essere riservate e sono destinate esclusivamente alla persona od alla Società indicata come destinatario. La diffusione e la distribuzione del presente documento a soggetti diversi da quelli indicati, od in generale qualsivoglia utilizzo illecito dei dati ivi contenuti, è proibita sia ai sensi dellart. 616 del Codice Penale che dal D. Lgs 196/03 in materia di protezione dei dati personali (Privacy). Se avete ricevuto per errore questo documento siete pregati di distruggerlo e di comunicarcelo prontamente tramite e-mail o fax.
Re: Tomcat CPU 100%
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrea, Andrea Di Muro wrote: | Hello everyone, I have a Tomcat server with several domains configured | and sometimes I can see that the tomcat process uses 100-190% (dual | processor) and hangs up. | I think that one of my customer has uploaded a jsp page that makes an | infinite loop. | How can I check which page causes the problem? Take a thread dump (QUIT signal on *NIX, CTRL-BREAK on windows console, jstack in recent JVM implementations, ... some monitoring programs have this ability, too) and look at what is running. You may have to run a couple of them and compare. The thread that is always running the same thing is likely to be the one that is causing problems. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkghzzIACgkQ9CaO5/Lv0PCRCACeLmJJR58cXsSGjln5FeVdlfa0 y/UAn0Q75rwJnb3sHWTY2qw+nOWbMa9E =GVY9 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
R: Tomcat CPU 100%
Andrea Di Muro wrote: | Hello everyone, I have a Tomcat server with several domains configured | and sometimes I can see that the tomcat process uses 100-190% (dual | processor) and hangs up. | I think that one of my customer has uploaded a jsp page that makes an | infinite loop. | How can I check which page causes the problem? Take a thread dump (QUIT signal on *NIX, CTRL-BREAK on windows console, jstack in recent JVM implementations, ... some monitoring programs have this ability, too) and look at what is running. You may have to run a couple of them and compare. The thread that is always running the same thing is likely to be the one that is causing problems. - -chris Tomcat it's running on a Linux Server, what is the exact command to take the thread dump? kill -s QUIT tomcat_pid is right? By using this command will Tomcat shut down or just print the threa dump on the standard output? And how can I interpretate the output to see which page is causing the problem? Thank you - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: R: Tomcat CPU 100%
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrea, Andrea Di Muro wrote: | Tomcat is running on a Linux Server, what is the exact command to take the | thread dump? | kill -s QUIT tomcat_pid is right? Yes. The thread dump will be printed on stdout, and probably go into catalina.out. | By using this command will Tomcat shut down or just print the threa dump on | the standard output? Only the thread dump -- Tomcat will not shutdown. | And how can I interpret the output to see which page is causing the | problem? A thread dump is like a stack trace, except that it performs a stack trace on /all/ live threads. It will tell you what each thread is doing. If you see a few threads that are running methods like _jsp_service, then those threads are running JSPs. The name of the JSP should be evident from the class and method name printed in the trace for that particular thread. You could observe several thread dumps to see if one particular thread (or several) are always executing the same method. If so, that JSP is probably the one that is hanging. Good luck, - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgh3xAACgkQ9CaO5/Lv0PBBbACgnRz2dQEdZH/kwzUXa/E/UjGE n58AoIpHKTrJowHF+DmUwztHEfRF4ppa =B4jY -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: R: Tomcat CPU 100%
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrea, Andrea Di Muro wrote: | Tomcat is running on a Linux Server, what is the exact command to take the | thread dump? | kill -s QUIT tomcat_pid is right? Yes. The thread dump will be printed on stdout, and probably go into catalina.out. | By using this command will Tomcat shut down or just print the threa dump on | the standard output? Only the thread dump -- Tomcat will not shutdown. | And how can I interpret the output to see which page is causing the | problem? A thread dump is like a stack trace, except that it performs a stack trace on /all/ live threads. It will tell you what each thread is doing. If you see a few threads that are running methods like _jsp_service, then those threads are running JSPs. The name of the JSP should be evident from the class and method name printed in the trace for that particular thread. You could observe several thread dumps to see if one particular thread (or several) are always executing the same method. If so, that JSP is probably the one that is hanging. Another (perhaps Linux-specific?) way to get some information is to run ps -fLp 12345 (where 12345 should be the PID of your Tomcat process). This will show one line for each thread in the Tomcat process, and also will show the amount of CPU time used by each thread. If you run this a couple of times, you should be able to pick up threads with significant increase in the used CPU time. These can then be correlated with the thread dump. F.ex. in my toy machine, the thread taking the most of CPU time is: tomcat7035 1 7870 0 40 Feb13 ?01:08:37 /usr/lib/jvm/java/bin ... where 7035 is the process id of Tomcat, and 7870 is the thread id. The thread dump then tells what that thread is running - however, the thread dump lists threads by hexadecimal id, and the above is decimal, so a conversion is needed: 7870=0x1ebe . Now to look for that 1ebe in the thread dump: ContainerBackgroundProcessor[StandardEngine[Catalina]] daemon prio=1 tid=0x08366728 nid=0x1ebe waiting on condition [0xb135c000..0xb135cfa0] at java.lang.Thread.sleep(Native Method) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1597) at java.lang.Thread.run(Thread.java:595) so, on this machine (a toy, as I said), the thread with most accumulated CPU usage has been ContainerBackgroundProcessor - which was sleeping at the time of this thread dump. For active request processing threads the dump will differ a lot from the above, and will tell in which class/method the thread is executing. -- ..Juha - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
