Andrew,
The performance data that we have is for the backend servers. We are
monitoring the normal things cpu, memory, load swap, context switches,
threads etc etc on the system but nothing specific to the tomcat service,
just the overall health of the system. It is not easy to put a lot of
profiling on a production server, it may be a good thing to do to find out
the issue but not something that anyone wants on in production.
Cataline.out from one of the crashes is below :
2007-12-05 13:42:50 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
at org.apache.tomcat.util.buf.ByteChunk.toString(ByteChunk.java:457)
at org.apache.tomcat.util.buf.MessageBytes.toString(MessageBytes.java
:192)
at org.apache.tomcat.util.http.MimeHeaders.getHeader(MimeHeaders.java
:293)
at org.apache.coyote.Request.getHeader(Request.java:345)
at org.apache.coyote.tomcat4.CoyoteAdapter.postParseRequest(
CoyoteAdapter.java:201)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java
:150)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:799)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection
(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java
:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)
2007-12-05 13:43:55 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
2007-12-05 13:44:08 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
2007-12-05 13:44:21 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
2007-12-05 13:44:21 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
2007-12-05 13:44:34 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
2007-12-05 13:44:46 CoyoteAdapter An exception or error occurred in the
container during the request processing
java.lang.OutOfMemoryError: Java heap space
Sun 1.4.2 Jvm
A stack trace was not done yet.
backend stats for cpu/memory/disk look normal we collect them every 15
minutes.
The thing I am wondering also is why it shuts down cleanly sometimes. Is
there something internally that triggers that with this version of Tomcat,
would an out of heap memory situation cause that. Maybe its in one of the
logs but I am still going through all of them. There are multiple megs of
them.
Sean
On Dec 6, 2007 11:23 AM, Andrew Miehs <[EMAIL PROTECTED]> wrote:
>
> On 06/12/2007, at 5:12 PM, Peter Crowther wrote:
>
> >> From: Sean Carnes [mailto:[EMAIL PROTECTED]
> >> The back-end
> >> servers seem to be responding in a timely fashion right now. We have
> >> performance data from the time period and nothing seems
> >> abnormal.
>
> I unfortunately missed the first part of this thread ...
>
> If you are having problems and your performance data show nothing
> abnormal,
> then you either do not have enough data or you do not have a
> problem.. :-)
>
> What errors are you seeing? (What is in catalina.out)?
> Are you running out of threads? (you are probably runing JVM 1.42 based
> on the version of tomcat you are running - Sun and IBM JVM 1.42 used to
> respond with Out of memory when you were out of threads)...
>
> Have you done a stack trace on the tomcat?
>
> Do you have disk performance stats from the backend as well as CPU,
> and load?
>
> At the moment, it could be anything, but if you say you have 50% more
> users,
> this something could very easily be the effect of a little more load,
> which
> brings the whole thing to a standstill
>
>
> Cheers
>
> Andrew
>
>
>
--
"Any sufficiently advanced technology is indistinguishable from magic."
- Arthur C. Clarke
