RE: Tomcat using near 100% of CPU on Windows 2008
Thanks to everyone who helped with their suggestions! We have switched the numbers, as you pointed out. Howard's suggestion to add the following command line options in tomcat7w.exe: -Xmx4096m -XX:MaxPermSize=512m -XX:+UseTLAB -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnableddid the trick. The CPU utilization by Tomcat is now really-really low. We'll keep an eye on it, but so far-so good. Cheers,Dima Date: Wed, 30 Jan 2013 13:06:15 -0500 Subject: Re: Tomcat using near 100% of CPU on Windows 2008 From: smithh032...@gmail.com To: users@tomcat.apache.org On Wed, Jan 30, 2013 at 12:27 PM, Dima Iva ots...@live.com wrote: We have a Windows 2008 R2 virtual machine on an ESXi server. It only runs one application that requires Apache, Tomcat, Java and PostgreSQL. We have the latest versions of these programs. The machine has 1TB of disk space and 8GB of RAM. When I start the Tomcat 7 service, it starts using around 95% of CPU in a span of a minute, after which the total utilization of CPU by the machine is 100%. Process Explorer shows that the number of threads for this process goes up to 91 and one of them shows over 15,000,000 Cycles Delta and uses all the CPU. While the machine is stalled by Tomcat, the commit charge for the machine is at 12% and physical memory usage at 25%, only the CPU is maxed out. I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables, but it doesn't seem to help. I believe there is a memory leak here, but not sure how to pinpoint it using free tools and patch this leak. I am not one of the expert Tomcat users or committers, but I am using TomEE via Tomcat 7.0.35 Windows service (tomcat7w.exe) on Windows Server 2008 R2 with 32GB RAM and 1TB hard drive too (after migrating from Windows Server 2003 4GB RAM). Maybe I have not monitored it closely, but i have not seen the 100% CPU spike that you have experienced and I have found tomee/tomcat to be stable on Windows Server 2008 R2. Instead of using CATALINA_OPTS=..., I specify the following java commandline options in the tomcat7 service dialog window (tomcat7/bin/tomcat7w.exe): -Xmx4096m -XX:MaxPermSize=512m -XX:+UseTLAB -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
RE: Tomcat using near 100% of CPU on Windows 2008
From: Dima Iva [mailto:ots...@live.com] Subject: Tomcat using near 100% of CPU on Windows 2008 I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables That's clearly wrong; you made the maximum smaller than the minimum. What makes you think you have a heap size problem? Also note that setting environment variables when running tomcat7.exe is pointless. Use the control panel (tomcat7w.exe) to set options for the Tomcat service. I believe there is a memory leak here Why? The last file shows what kind of information Process Explorer lists for the 91 threads that are part of the tomcat7.exe process, including the first one that is eating up all the CPU. Process Explorer is pretty much useless for a Java program; take several Java thread dumps and see what's going on. http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F http://wiki.apache.org/tomcat/HowTo#How_do_I_read_a_Java_thread_dump_.3F - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat using near 100% of CPU on Windows 2008
On Wed, Jan 30, 2013 at 12:27 PM, Dima Iva ots...@live.com wrote: We have a Windows 2008 R2 virtual machine on an ESXi server. It only runs one application that requires Apache, Tomcat, Java and PostgreSQL. We have the latest versions of these programs. The machine has 1TB of disk space and 8GB of RAM. When I start the Tomcat 7 service, it starts using around 95% of CPU in a span of a minute, after which the total utilization of CPU by the machine is 100%. Process Explorer shows that the number of threads for this process goes up to 91 and one of them shows over 15,000,000 Cycles Delta and uses all the CPU. While the machine is stalled by Tomcat, the commit charge for the machine is at 12% and physical memory usage at 25%, only the CPU is maxed out. I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables, but it doesn't seem to help. I believe there is a memory leak here, but not sure how to pinpoint it using free tools and patch this leak. I am not one of the expert Tomcat users or committers, but I am using TomEE via Tomcat 7.0.35 Windows service (tomcat7w.exe) on Windows Server 2008 R2 with 32GB RAM and 1TB hard drive too (after migrating from Windows Server 2003 4GB RAM). Maybe I have not monitored it closely, but i have not seen the 100% CPU spike that you have experienced and I have found tomee/tomcat to be stable on Windows Server 2008 R2. Instead of using CATALINA_OPTS=..., I specify the following java commandline options in the tomcat7 service dialog window (tomcat7/bin/tomcat7w.exe): -Xmx4096m -XX:MaxPermSize=512m -XX:+UseTLAB -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Re: Tomcat using near 100% of CPU on Windows 2008
On 1/30/2013 10:04 AM, Caldarale, Charles R wrote: From: Dima Iva [mailto:ots...@live.com] Subject: Tomcat using near 100% of CPU on Windows 2008 I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables That's clearly wrong; you made the maximum smaller than the minimum. What makes you think you have a heap size problem? I don't know about heap size, but I did manage to receive his logs. Hint, most of the time the list strips attachments. If you need to include text, then it's best to include it inline (and remove sensitive data). In probe.log, I saw the following: 28 Jan 2013 12:39:38 [scheduler_Worker-8] ERROR org.quartz.core.JobRunShell - Job DEFAULT.connectorStatsJobDetail threw an unhandled Exception: java.lang.OutOfMemoryError: PermGen space After that, I would think all bets are off. It looks like you are running (from a quick glance at the logs) a Cocoon plus Solr application? Maybe you do need to increase PermGen space. Also note that setting environment variables when running tomcat7.exe is pointless. Use the control panel (tomcat7w.exe) to set options for the Tomcat service. +1 I believe there is a memory leak here Why? The last file shows what kind of information Process Explorer lists for the 91 threads that are part of the tomcat7.exe process, including the first one that is eating up all the CPU. Process Explorer is pretty much useless for a Java program; take several Java thread dumps and see what's going on. http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F - Chuck . . . . just my two cents /mde/ - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [OT] Tomcat using near 100% of CPU on Windows 2008
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 1/30/13 1:04 PM, Caldarale, Charles R wrote: From: Dima Iva [mailto:ots...@live.com] Subject: Tomcat using near 100% of CPU on Windows 2008 I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables That's clearly wrong; you made the maximum smaller than the minimum. What makes you think you have a heap size problem? He's got a negative heap, of course. He needs to use anti-objects instead of objects. Make sure not to mix the two or they will annihilate each other. Also note that setting environment variables when running tomcat7.exe is pointless. Use the control panel (tomcat7w.exe) to set options for the Tomcat service. +1 Take a thread dump, get a heap summary. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlEJaBgACgkQ9CaO5/Lv0PC79QCgnDP9tooAxQtjSUna3ylX+EKJ G2oAn2aMRAz2Ihe46BAT24qKGWO6czBV =wydt -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
