Tomcat hanging unexpectedly [URGENT]
Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush
Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat hanging unexpectedly [URGENT]
Em 12/02/2013 15:53, Cool Techi escreveu: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Would it have exhausted database connections limit? Edson Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
I am guessing it would when more users come into the system, but why would this cause a jump in memory utilization in tomcat and then hanging. If you could throw more light at this? Also, I am not sure if these are pointing to the problem. -Ayush Date: Tue, 12 Feb 2013 16:00:12 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] Em 12/02/2013 15:53, Cool Techi escreveu: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Would it have exhausted database connections limit? Edson Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat hanging unexpectedly [URGENT]
As I said, everything said before getting more details would be mere speculation. I'm just brainstorming based on information you gave so far. I would recommend to check a heap dump and profiling using VisualJVM of application while running. It would give a better idea. Regards, Edson Em 12/02/2013 16:18, Cool Techi escreveu: I am guessing it would when more users come into the system, but why would this cause a jump in memory utilization in tomcat and then hanging. If you could throw more light at this? Also, I am not sure if these are pointing to the problem. -Ayush Date: Tue, 12 Feb 2013 16:00:12 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] Em 12/02/2013 15:53, Cool Techi escreveu: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Would it have exhausted database connections limit? Edson Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Oh dear. Are you doing your app logging to a database? Is it the same database that your app uses? Is it the same connection pool that your app uses? p Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- [key:62590808] - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
I am not logging to database, these are my appenders,
appender name=errorfile class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/errorVer.log /
param name=threshold value=error /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS}
%5p %c{1}:%L - %m%n /
/layout
/appender
appender name=debugfile
class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/debugVer.log /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS}
%5p %c{1}:%L - %m%n /
/layout
filter class=org.apache.log4j.varia.LevelRangeFilter
param name=LevelMin value=debug /
param name=LevelMax value=debug /
/filter
/appender
regards,
Ayush
Date: Tue, 12 Feb 2013 18:38:24 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote:
I am new to this, I have just installed YourKit and see a lot of threads
blocked,but currently my response are fine, the blocked thread are showing
the following
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
Does any of these sound like reason. It would be great to get pointed in
the right direction.
Regards,Ayush
Oh dear.
Are you doing your app logging to a database?
Is it the same database that your app uses?
Is it the same connection pool that your app uses?
p
Date: Tue, 12 Feb 2013 15:35:40 -0200
From: edsonrich...@hotmail.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago.
It can be caused by almost anything, and my last suspect would be Tomcat
it self.
Most of time, the cause is some programming error, like:
- missing step out of recursive loop
- loading too many objects is memory (for example, when using JPA where
graph of objects are all connected - so, when you load one object, all
database is read into memory)
- some deadlock in multi-threaded application...
How to discover:
1) Try to get a memory dump and check how your memory is being used
2) Get a heap dump (stack trace, as well) or currently running
application when it is hang, and check for deadlocks
3) Use VisualVM to get a big picture of what the virtual machine is doing
4) Put the application running under profiler (like JRockit Mission
Control, or NetBeans profiler)
These are some ideas...
Regards,
Edson
Em 12/02/2013 15:27, Cool Techi escreveu:
Hi,
We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32
installed on the server.
The
server seems to be running smoothly most of the time, but suddenly the
memory used goes very high and tomcat stops responding, the logs don't
show any out of memory or thread exceptions.
What could be causing this?
Regards,
Ayush
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
[key:62590808]
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 18:42, Cool Techi wrote:
I am not logging to database, these are my appenders,
You posted the below in a previous message. It was in a single
unintelligible line. Perhaps you could clarify what you intended to
post and try again?
org.apache.log4j.Category.callAppenders
com.mysql.jdbc.SingleByteCharsetConverter.getInstance
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
p
appender name=errorfile class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/errorVer.log /
param name=threshold value=error /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd
HH:mm:ss,SSS} %5p %c{1}:%L - %m%n /
/layout
/appender
appender name=debugfile
class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/debugVer.log /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd
HH:mm:ss,SSS} %5p %c{1}:%L - %m%n /
/layout
filter class=org.apache.log4j.varia.LevelRangeFilter
param name=LevelMin value=debug /
param name=LevelMax value=debug /
/filter
/appender
regards,
Ayush
Date: Tue, 12 Feb 2013 18:38:24 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote:
I am new to this, I have just installed YourKit and see a lot of threads
blocked,but currently my response are fine, the blocked thread are showing
the following
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
Does any of these sound like reason. It would be great to get pointed in
the right direction.
Regards,Ayush
Oh dear.
Are you doing your app logging to a database?
Is it the same database that your app uses?
Is it the same connection pool that your app uses?
p
Date: Tue, 12 Feb 2013 15:35:40 -0200
From: edsonrich...@hotmail.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago.
It can be caused by almost anything, and my last suspect would be Tomcat
it self.
Most of time, the cause is some programming error, like:
- missing step out of recursive loop
- loading too many objects is memory (for example, when using JPA where
graph of objects are all connected - so, when you load one object, all
database is read into memory)
- some deadlock in multi-threaded application...
How to discover:
1) Try to get a memory dump and check how your memory is being used
2) Get a heap dump (stack trace, as well) or currently running
application when it is hang, and check for deadlocks
3) Use VisualVM to get a big picture of what the virtual machine is doing
4) Put the application running under profiler (like JRockit Mission
Control, or NetBeans profiler)
These are some ideas...
Regards,
Edson
Em 12/02/2013 15:27, Cool Techi escreveu:
Hi,
We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32
installed on the server.
The
server seems to be running smoothly most of the time, but suddenly the
memory used goes very high and tomcat stops responding, the logs don't
show any out of memory or thread exceptions.
What could be causing this?
Regards,
Ayush
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
[key:62590808]
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
[key:62590808]
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
Sorry if what I wrote was confusing, actually I have installed YourKit profiler
and am profiling my tomcat. On checking or blocking threads I get to see the
following,
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
--org.apache.log4j.Category.callAppenders
--com.mysql.jdbc.SingleByteCharsetConverter.getInstance
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
--ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
As I am new to this I am not sure what might be causing this, is it log4j
or something else like dbcp pool.
Help appreciated.
Ayush.
Date: Tue, 12 Feb 2013 18:45:03 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 18:42, Cool Techi wrote:
I am not logging to database, these are my appenders,
You posted the below in a previous message. It was in a single
unintelligible line. Perhaps you could clarify what you intended to
post and try again?
org.apache.log4j.Category.callAppenders
com.mysql.jdbc.SingleByteCharsetConverter.getInstance
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
p
appender name=errorfile
class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/errorVer.log /
param name=threshold value=error /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd
HH:mm:ss,SSS} %5p %c{1}:%L - %m%n /
/layout
/appender
appender name=debugfile
class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/debugVer.log /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd
HH:mm:ss,SSS} %5p %c{1}:%L - %m%n /
/layout
filter class=org.apache.log4j.varia.LevelRangeFilter
param name=LevelMin value=debug /
param name=LevelMax value=debug /
/filter
/appender
regards,
Ayush
Date: Tue, 12 Feb 2013 18:38:24 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote:
I am new to this, I have just installed YourKit and see a lot of threads
blocked,but currently my response are fine, the blocked thread are
showing the following
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
Does any of these sound like reason. It would be great to get pointed in
the right direction.
Regards,Ayush
Oh dear.
Are you doing your app logging to a database?
Is it the same database that your app uses?
Is it the same connection pool that your app uses?
p
Date: Tue, 12 Feb 2013 15:35:40 -0200
From: edsonrich...@hotmail.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago.
It can be caused by almost anything, and my last suspect would be Tomcat
it self.
Most of time, the cause is some programming error, like:
- missing step out of recursive loop
- loading too many objects is memory (for example, when using JPA where
graph of objects are all connected - so, when you load one object, all
database is read into memory)
- some deadlock in multi-threaded application...
How to discover:
1) Try to get a memory dump and check how your memory is being used
2) Get a heap dump (stack trace, as well) or currently running
application when it is hang, and check for deadlocks
3) Use VisualVM to get a big picture of what the virtual machine is doing
4) Put the application running under profiler (like JRockit Mission
Control, or NetBeans profiler)
These are some ideas...
Regards,
Edson
Em 12/02/2013 15:27, Cool Techi escreveu:
Hi,
We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32
installed on the server.
The
server seems to be running smoothly most of the time, but suddenly the
memory used goes very high and tomcat stops responding, the logs don't
show any out of memory or thread exceptions.
What could be causing this?
Regards,
Ayush
Re: Tomcat hanging unexpectedly [URGENT]
Em 12/02/2013 16:50, Cool Techi escreveu:
Sorry if what I wrote was confusing, actually I have installed YourKit profiler
and am profiling my tomcat. On checking or blocking threads I get to see the
following,
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
--org.apache.log4j.Category.callAppenders
--com.mysql.jdbc.SingleByteCharsetConverter.getInstance
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
--ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
As I am new to this I am not sure what might be causing this, is it log4j
or something else like dbcp pool.
The cause is unlikely to be log4j or dbcp pool it self (they may be
involved as a bridge between the code causing the problem and the thread
that has been stuck).
QuartzScheduler would be blocked by your application (for instance,
opening a transaction that would lock one or more tables, and then
forget to properly release by correctly closing the resources).
As I already said, having more information, like a heap dump or a stack
trace would help to identify the cause.
Seems too little information to deal with such big problem.
Regards,
Edson
Help appreciated.
Ayush.
Date: Tue, 12 Feb 2013 18:45:03 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 18:42, Cool Techi wrote:
I am not logging to database, these are my appenders,
You posted the below in a previous message. It was in a single
unintelligible line. Perhaps you could clarify what you intended to
post and try again?
org.apache.log4j.Category.callAppenders
com.mysql.jdbc.SingleByteCharsetConverter.getInstance
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
p
appender name=errorfile class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/errorVer.log /
param name=threshold value=error /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS} %5p
%c{1}:%L - %m%n /
/layout
/appender
appender name=debugfile class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/debugVer.log /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS} %5p
%c{1}:%L - %m%n /
/layout
filter class=org.apache.log4j.varia.LevelRangeFilter
param name=LevelMin value=debug /
param name=LevelMax value=debug /
/filter
/appender
regards,
Ayush
Date: Tue, 12 Feb 2013 18:38:24 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote:
I am new to this, I have just installed YourKit and see a lot of threads
blocked,but currently my response are fine, the blocked thread are showing the
following
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
Does any of these sound like reason. It would be great to get pointed in the
right direction.
Regards,Ayush
Oh dear.
Are you doing your app logging to a database?
Is it the same database that your app uses?
Is it the same connection pool that your app uses?
p
Date: Tue, 12 Feb 2013 15:35:40 -0200
From: edsonrich...@hotmail.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago.
It can be caused by almost anything, and my last suspect would be Tomcat
it self.
Most of time, the cause is some programming error, like:
- missing step out of recursive loop
- loading too many objects is memory (for example, when using JPA where
graph of objects are all connected - so, when you load one object, all
database is read into memory)
- some deadlock in multi-threaded application...
How to discover:
1) Try to get a memory dump and check how your memory is being used
2) Get a heap dump (stack trace, as well) or currently running
application when it is hang, and check for deadlocks
3) Use VisualVM to get a big picture of what the virtual machine is doing
4) Put the application running under profiler (like JRockit Mission
Control, or NetBeans profiler)
These are some ideas...
Regards,
Edson
Em 12/02/2013 15:27, Cool Techi escreveu:
Hi,
We
Re: Tomcat hanging unexpectedly [URGENT]
On 2/12/2013 10:50 AM, Cool Techi wrote:
Sorry if what I wrote was confusing, actually I have installed YourKit profiler
and am profiling my tomcat. On checking or blocking threads I get to see the
following,
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
--org.apache.log4j.Category.callAppenders
--com.mysql.jdbc.SingleByteCharsetConverter.getInstance
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
--org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
--ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
As I am new to this I am not sure what might be causing this, is it log4j
or something else like dbcp pool.
Help appreciated.
Ayush.
It's most likely your application.
Post a complete thread dump of at least one of the blocked threads. See
the following on how to do this:
http://wiki.apache.org/tomcat/HowTo#If_you_are_running_on_Microsoft_Windows
I'm guessing the following:
1. Quartz scheduler writes to a database
2. Some exception happens
3. The exception is logged
4. The connection isn't cleaned up with a finally block
5. Bad things happen . . . .
Or
1. Quartz scheduler writes to a database
2. A log4j entry is written
3. The connection isn't cleaned up
4. Bad things happen . . .
Or
Make up your scenario here . . .
See my previous post on setting up logAbandoned, etc. in a test environment.
Also, first things first, post a complete thread dump of the blocked thread.
. . . . just my two cents.
/mde/
Date: Tue, 12 Feb 2013 18:45:03 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 18:42, Cool Techi wrote:
I am not logging to database, these are my appenders,
You posted the below in a previous message. It was in a single
unintelligible line. Perhaps you could clarify what you intended to
post and try again?
org.apache.log4j.Category.callAppenders
com.mysql.jdbc.SingleByteCharsetConverter.getInstance
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()
org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()
ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
p
appender name=errorfile class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/errorVer.log /
param name=threshold value=error /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS} %5p
%c{1}:%L - %m%n /
/layout
/appender
appender name=debugfile
class=org.apache.log4j.DailyRollingFileAppender
param name=File value=D:/Logs/debugVer.log /
layout class=org.apache.log4j.PatternLayout
param name=ConversionPattern value=%d{-MMM-dd HH:mm:ss,SSS} %5p
%c{1}:%L - %m%n /
/layout
filter class=org.apache.log4j.varia.LevelRangeFilter
param name=LevelMin value=debug /
param name=LevelMax value=debug /
/filter
/appender
regards,
Ayush
Date: Tue, 12 Feb 2013 18:38:24 +
From: p...@pidster.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
On 12/02/2013 17:53, Cool Techi wrote:
I am new to this, I have just installed YourKit and see a lot of threads
blocked,but currently my response are fine, the blocked thread are showing the
following
Blocker Thread QuartzSchedular_Worker-6 native ID ..
On clicking on the Blocker thread it shows me the following,
org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle
Does any of these sound like reason. It would be great to get pointed in the
right direction.
Regards,Ayush
Oh dear.
Are you doing your app logging to a database?
Is it the same database that your app uses?
Is it the same connection pool that your app uses?
p
Date: Tue, 12 Feb 2013 15:35:40 -0200
From: edsonrich...@hotmail.com
To: users@tomcat.apache.org
Subject: Re: Tomcat hanging unexpectedly [URGENT]
There is a similar thread published a week ago.
It can be caused by almost anything, and my last suspect would be Tomcat
it self.
Most of time, the cause is some programming error, like:
- missing step out of recursive loop
- loading too many objects is memory (for example, when using JPA where
graph of objects are all connected - so, when you load one object, all
database is read into memory)
- some deadlock in multi-threaded application...
How to discover:
1) Try to get a memory dump and check how your memory is being used
2) Get a heap dump (stack trace, as well) or currently running
application when it is hang, and check for deadlocks
3) Use VisualVM to get a big picture of what
Re: Tomcat hanging unexpectedly [URGENT]
On 2/12/2013 10:38 AM, Pid wrote: On 12/02/2013 17:53, Cool Techi wrote: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenders com.mysql.jdbc.SingleByteCharsetConverter.getInstance org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Oh dear. Are you doing your app logging to a database? Is it the same database that your app uses? Is it the same connection pool that your app uses? p This sounds like a lot of things. However, without borrowing your crystal ball, I can't tell anything about the Quartz scheduler. A couple of things: 1. Upgrade your Java to at least the latest 1.6.0_xx version The one you are using has known server (and client) security issues. 2. Upgrade your Tomcat to at least the latest 6.xx version There are many improvements and bug fixes. 3. [URGENT] I understand that this problem is urgent for you. However, the people here contribute time on a volunteer basis (I'm taking a break from writing documentation for a new system). We'll try to help as time allows, but if you need an SLA then you should purchase commercial support. 4. Possible root cause I'm guessing (based on your partial listing of classes in a thread dump) is that you either have a logging problem, a database connectivity problem, or a Quartz scheduler problem. Or all of the above. If there was one problem, then I (guessing) would look to see if you're logging via a Quartz scheduler job through log4j to a MySQL database. Now that I've got my guess out of the way, here are some pointers. a. Thread dumps You said you've taken them, but have only posted snippets. Post a complete thread dump inline (mailing list strips attachments). For your own use, take several 3-10 seconds apart to see what's changing. b. Check your database It sounds like you have a connection leak. When this happens, how many active connections do you have to your MySQL database? c. Connection leak As noted above, you may have a connections leak. Post your Resource definition (hopefully it's in context.xml) with passwords and usernames changed. While you're at it, read: http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html#JDBC_Data_Sources In particular, note removeAbandoned and logAbandoned. These will impact performance, so do it on a test system. Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get a heap dump (stack trace, as well) or currently running application when it is hang, and check for deadlocks 3) Use VisualVM to get a big picture of what the virtual machine is doing + 1 here Run this on a test environment, load up the system, and do heap dumps and thread dumps. 4) Put the application running under profiler (like JRockit Mission Control, or NetBeans profiler) These are some ideas... Regards, Edson Em 12/02/2013 15:27, Cool Techi escreveu: Hi, We are using tomcat 6.0.26 on windows 2008 web server, with Jdk 1.6.32 installed on the server. The server seems to be running smoothly most of the time, but suddenly the memory used goes very high and tomcat stops responding, the logs don't show any out of memory or thread exceptions. What could be causing this? Regards, Ayush Oh, and please do not top post. I had to bounce back and forth through the message to figure out what's been said and where to reply. . . . just my two cents /mde/ - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
Thanks Everyone for the help, I will enable thread dump for my Tomcat, just reading up on how to get that done. My database connection string is given below. Resource name=jdbc/bm type=javax.sql.DataSource maxActive=250 maxIdle=5 username=root testWhileIdle=true removeAbandonedTimeout=60 maxWait=-1 removeAbandoned=true validationQuery=select 1 driverClassName=com.mysql.jdbc.Driver password=xxx minEvictableIdleTimeMillis=3 timeBetweenEvictionRunsMillis=30 url=jdbc:mysql://localhost:3306/brandmanagement?autoReconnect=trueamp;useUnicode=trueamp;characterEncoding=utf-8/ Resource name=jdbc/s360 type=javax.sql.DataSource maxActive=250 maxIdle=5 username=root testWhileIdle=true removeAbandonedTimeout=60 maxWait=-1 removeAbandoned=true validationQuery=select 1 driverClassName=com.mysql.jdbc.Driver password=xxx minEvictableIdleTimeMillis=3 timeBetweenEvictionRunsMillis=30 url=jdbc:mysql://localhost:3306/brandmanagement?autoReconnect=trueamp;useUnicode=trueamp;characterEncoding=utf-8/ I use quartz schedular(spring based) to perform some tasks on reqular intervals, Log4j is used in those methods, so will cross check those methods again. Really appreciate your time and help. Regards, Ayush Date: Tue, 12 Feb 2013 11:36:52 -0800 From: its_toas...@yahoo.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] On 2/12/2013 10:38 AM, Pid wrote: On 12/02/2013 17:53, Cool Techi wrote: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenders com.mysql.jdbc.SingleByteCharsetConverter.getInstance org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Oh dear. Are you doing your app logging to a database? Is it the same database that your app uses? Is it the same connection pool that your app uses? p This sounds like a lot of things. However, without borrowing your crystal ball, I can't tell anything about the Quartz scheduler. A couple of things: 1. Upgrade your Java to at least the latest 1.6.0_xx version The one you are using has known server (and client) security issues. 2. Upgrade your Tomcat to at least the latest 6.xx version There are many improvements and bug fixes. 3. [URGENT] I understand that this problem is urgent for you. However, the people here contribute time on a volunteer basis (I'm taking a break from writing documentation for a new system). We'll try to help as time allows, but if you need an SLA then you should purchase commercial support. 4. Possible root cause I'm guessing (based on your partial listing of classes in a thread dump) is that you either have a logging problem, a database connectivity problem, or a Quartz scheduler problem. Or all of the above. If there was one problem, then I (guessing) would look to see if you're logging via a Quartz scheduler job through log4j to a MySQL database. Now that I've got my guess out of the way, here are some pointers. a. Thread dumps You said you've taken them, but have only posted snippets. Post a complete thread dump inline (mailing list strips attachments). For your own use, take several 3-10 seconds apart to see what's changing. b. Check your database It sounds like you have a connection leak. When this happens, how many active connections do you have to your MySQL database? c. Connection leak As noted above, you may have a connections leak. Post your Resource definition (hopefully it's in context.xml) with passwords and usernames changed. While you're at it, read: http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html#JDBC_Data_Sources In particular, note removeAbandoned and logAbandoned. These will impact performance, so do it on a test system. Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover
Re: Tomcat hanging unexpectedly [URGENT]
As is usual - long reply at the end. On 2/12/2013 11:48 AM, Cool Techi wrote: Thanks Everyone for the help, I will enable thread dump for my Tomcat, just reading up on how to get that done. My database connection string is given below. Resource name=jdbc/bm type=javax.sql.DataSource maxActive=250 maxIdle=5 username=root testWhileIdle=true removeAbandonedTimeout=60 maxWait=-1 removeAbandoned=true validationQuery=select 1 driverClassName=com.mysql.jdbc.Driver password=xxx minEvictableIdleTimeMillis=3 timeBetweenEvictionRunsMillis=30 url=jdbc:mysql://localhost:3306/brandmanagement?autoReconnect=trueamp;useUnicode=trueamp;characterEncoding=utf-8/ Resource name=jdbc/s360 type=javax.sql.DataSource maxActive=250 maxIdle=5 username=root testWhileIdle=true removeAbandonedTimeout=60 maxWait=-1 removeAbandoned=true validationQuery=select 1 driverClassName=com.mysql.jdbc.Driver password=xxx minEvictableIdleTimeMillis=3 timeBetweenEvictionRunsMillis=30 url=jdbc:mysql://localhost:3306/brandmanagement?autoReconnect=trueamp;useUnicode=trueamp;characterEncoding=utf-8/ I use quartz schedular(spring based) to perform some tasks on reqular intervals, Log4j is used in those methods, so will cross check those methods again. Really appreciate your time and help. Regards, Ayush Date: Tue, 12 Feb 2013 11:36:52 -0800 From: its_toas...@yahoo.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] On 2/12/2013 10:38 AM, Pid wrote: On 12/02/2013 17:53, Cool Techi wrote: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenders com.mysql.jdbc.SingleByteCharsetConverter.getInstance org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool() org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Does any of these sound like reason. It would be great to get pointed in the right direction. Regards,Ayush Oh dear. Are you doing your app logging to a database? Is it the same database that your app uses? Is it the same connection pool that your app uses? p This sounds like a lot of things. However, without borrowing your crystal ball, I can't tell anything about the Quartz scheduler. A couple of things: 1. Upgrade your Java to at least the latest 1.6.0_xx version The one you are using has known server (and client) security issues. 2. Upgrade your Tomcat to at least the latest 6.xx version There are many improvements and bug fixes. 3. [URGENT] I understand that this problem is urgent for you. However, the people here contribute time on a volunteer basis (I'm taking a break from writing documentation for a new system). We'll try to help as time allows, but if you need an SLA then you should purchase commercial support. 4. Possible root cause I'm guessing (based on your partial listing of classes in a thread dump) is that you either have a logging problem, a database connectivity problem, or a Quartz scheduler problem. Or all of the above. If there was one problem, then I (guessing) would look to see if you're logging via a Quartz scheduler job through log4j to a MySQL database. Now that I've got my guess out of the way, here are some pointers. a. Thread dumps You said you've taken them, but have only posted snippets. Post a complete thread dump inline (mailing list strips attachments). For your own use, take several 3-10 seconds apart to see what's changing. b. Check your database It sounds like you have a connection leak. When this happens, how many active connections do you have to your MySQL database? c. Connection leak As noted above, you may have a connections leak. Post your Resource definition (hopefully it's in context.xml) with passwords and usernames changed. While you're at it, read: http://tomcat.apache.org/tomcat-6.0-doc/jndi-resources-howto.html#JDBC_Data_Sources In particular, note removeAbandoned and logAbandoned. These will impact performance, so do it on a test system. Date: Tue, 12 Feb 2013 15:35:40 -0200 From: edsonrich...@hotmail.com To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] There is a similar thread published a week ago. It can be caused by almost anything, and my last suspect would be Tomcat it self. Most of time, the cause is some programming error, like: - missing step out of recursive loop - loading too many objects is memory (for example, when using JPA where graph of objects are all connected - so, when you load one object, all database is read into memory) - some deadlock in multi-threaded application... How to discover: 1) Try to get a memory dump and check how your memory is being used 2) Get
Re: Tomcat hanging unexpectedly [URGENT]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Pid, On 2/12/13 1:38 PM, Pid wrote: On 12/02/2013 17:53, Cool Techi wrote: I am new to this, I have just installed YourKit and see a lot of threads blocked,but currently my response are fine, the blocked thread are showing the following Blocker Thread QuartzSchedular_Worker-6 native ID .. Tomcat does not use Quartz for anything, so this is looking like a problem elsewhere. We still might be able to help. On clicking on the Blocker thread it shows me the following, org.apache.log4j.Category.callAppenderscom.mysql.jdbc.SingleByteCharsetConverter.getInstanceorg.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject()org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.addObjectToPool()ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Yuck, I don't know what to make of that line, but this looks suspicious: org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrorObject() and so does this one: ogr.apache.tomcat.dbcp.pool.impl.GenericObjectPool.getNumIdle Did you manually type-in all of those? Does any of these sound like reason. It would be great to get pointed in the right direction. Oh dear. Are you doing your app logging to a database? Is it the same database that your app uses? Is it the same connection pool that your app uses? Yup, that sounds like a recipe for deadlock if it is indeed happening. Ayush, can you try reducing your connection pool's maxActive to 1 in a dev/test environment and try running your application? If it deadlocks (which it will if you try to log to the db while holding a db connection in the same thread), then you have a lot of work to do. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlEay5wACgkQ9CaO5/Lv0PAxiACaA08FSybmZXG9dk+pQXZHXziq 5IsAn1jtL3XpnVArB5vXn6AmkcwB2tKc =0/94 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat hanging unexpectedly [URGENT]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ayush, On 2/12/13 2:48 PM, Cool Techi wrote: Thanks Everyone for the help, I will enable thread dump for my Tomcat, just reading up on how to get that done. My database connection string is given below. You don't need to enable a thread dump on your system: just actively take one when your webapp locks-up. http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlEazKQACgkQ9CaO5/Lv0PCDcQCggBafYfas6Ti1uZfn5veAINrN z+cAmwdkfP9pv0aWD2U9xx3BEgCa/ZBd =wOpn -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Tomcat hanging unexpectedly [URGENT]
Thanks Everyone for all the help, it turns out that multiple instance of the schedular was getting fired, since I had created multiple appbases for multiple domains pointing to the same code base. This was causing a deadlock. I am still monitoring the application, a few things are not happening as expected as tomcat still hangs once in 12 hrs, but I know where to look. Regards, Ayush Date: Tue, 12 Feb 2013 18:13:40 -0500 From: ch...@christopherschultz.net To: users@tomcat.apache.org Subject: Re: Tomcat hanging unexpectedly [URGENT] -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ayush, On 2/12/13 2:48 PM, Cool Techi wrote: Thanks Everyone for the help, I will enable thread dump for my Tomcat, just reading up on how to get that done. My database connection string is given below. You don't need to enable a thread dump on your system: just actively take one when your webapp locks-up. http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEAREIAAYFAlEazKQACgkQ9CaO5/Lv0PCDcQCggBafYfas6Ti1uZfn5veAINrN z+cAmwdkfP9pv0aWD2U9xx3BEgCa/ZBd =wOpn -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
