subject:"RE\: 100% usage and hanging on startup of Spring\/Hibernate app in Tomcat 5.5 on Ubuntu 7"

Hmm, I'm going to try again with more explicit PermGen space as it seems to
be out of it.
- Bradley

On Thu, Sep 3, 2009 at 3:51 PM, Bradley Wagner 
bradley.wag...@hannonhill.com wrote:

 And again in a new place on subsequent attempt. I've attached the logging
 this time because the amount of logging between the last Tomcat line and the
 GC line was significant!
 - Bradley


 On Thu, Sep 3, 2009 at 1:49 PM, Caldarale, Charles R 
 chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Maybe it can't connect when it's hung.

 Quite possible; GC stops all threads, so the JMX agent may well be
 suspended.

 You might want to try turning on pretty much all of the GC-related trace
 flags:

 -XX:+PrintGC
 -XX:+PrintGCDetails
 -XX:+PrintGCTimeStamps
 -XX:+PrintGCTaskTimeStamps
 -XX:+PrintReferenceGC
 -XX:+TraceGen0Time
 -XX:+TraceGen1Time
 -XX:+PrintTenuringDistribution
 -XX:+PrintHeapAtGC
 -XX:+TraceParallelOldGCTasks
 -XX:+PrintParallelOldGCPhaseTimes

 (All on one line, of course.)  This will produce a *lot* of output.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 Hmm, I'm going to try again with more explicit PermGen space as it
 seems to be out of it.

Try it, but I'd be surprised if it had any effect.  The values shown are for 
the current size of PermGen, not the maximum to which it may expand.  That's 
usually 64m, unless you've changed it.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

This is odd. After setting permgen to 128m with -XX:MaxPermSize=128m I'm
seeing it hang again. What's odd is that the PermGen space appears to be
totally used up at 26MB and not expanding, unless I'm reading it wrong.
Heap after GC invocations=10 (full 0):
 PSYoungGen  total 47296K, used 2831K [0xb087, 0xb415,
0xb415)
  eden space 36992K, 0% used [0xb087,0xb087,0xb2c9)
  from space 10304K, 27% used [0xb374,0xb3a03e30,0xb415)
  to   space 10624K, 0% used [0xb2c9,0xb2c9,0xb36f)
 PSOldGentotal 466048K, used 16623K [0x9415, 0xb087,
0xb087)
  object space 466048K, 3% used [0x9415,0x9518bea8,0xb087)
 PSPermGen   total 26240K, used 26114K [0x8c15, 0x8daf,
0x9415)
  object space 26240K, 99% used [0x8c15,0x8dad0a50,0x8daf)
}
VM-Thread 9737526 9770673 9771284
GC-Thread 0 entries: 8
[ old-to-young-roots-task 9750971 9766007 ]
[ thread-roots-task 9766013 9766016 ]
[ scavenge-roots-task 9766017 9766024 ]
[ scavenge-roots-task 9766024 9766028 ]
[ scavenge-roots-task 9766029 9766032 ]
[ scavenge-roots-task 9766033 9766504 ]
[ scavenge-roots-task 9766505 9766506 ]
[ steal-task 9766507 9770612 ]
GC-Thread 1 entries: 19
[ old-to-young-roots-task 9737771 9756587 ]
[ serial-old-to-young-roots-task 9756590 9762460 ]
[ scavenge-roots-task 9762462 9762465 ]
[ scavenge-roots-task 9762466 9762481 ]
[ thread-roots-task 9762482 9762500 ]
[ thread-roots-task 9762506 9762516 ]
[ thread-roots-task 9762516 9762534 ]
[ thread-roots-task 9762535 9762552 ]
[ thread-roots-task 9762553 9762558 ]
[ thread-roots-task 9762558 9762565 ]
[ thread-roots-task 9762565 9762567 ]
[ thread-roots-task 9762567 9762569 ]
[ thread-roots-task 9762570 9762572 ]
[ thread-roots-task 9762572 9762574 ]
[ thread-roots-task 9762574 9762580 ]
[ thread-roots-task 9762580 9762584 ]
[ thread-roots-task 9762585 9770048 ]
[ steal-task 9770050 9770591 ]
[ waitfor-barrier-task 9770592 9770632 ]
{Heap before GC invocations=11 (full 0):
 PSYoungGen  total 47296K, used 39823K [0xb087, 0xb415,
0xb415)
  eden space 36992K, 100% used [0xb087,0xb2c9,0xb2c9)
  from space 10304K, 27% used [0xb374,0xb3a03e30,0xb415)
  to   space 10624K, 0% used [0xb2c9,0xb2c9,0xb36f)
 PSOldGentotal 466048K, used 16623K [0x9415, 0xb087,
0xb087)
  object space 466048K, 3% used [0x9415,0x9518bea8,0xb087)
 PSPermGen   total 26624K, used 26570K [0x8c15, 0x8db5,
0x9415)
  object space 26624K, 99% used [0x8c15,0x8db42ab0,0x8db5)
10.307: [GC


On Thu, Sep 3, 2009 at 3:58 PM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Hmm, I'm going to try again with more explicit PermGen space as it
  seems to be out of it.

 Try it, but I'd be surprised if it had any effect.  The values shown are
 for the current size of PermGen, not the maximum to which it may expand.
  That's usually 64m, unless you've changed it.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 What's odd is that the PermGen space appears to be totally used 
 up at 26MB and not expanding, unless I'm reading it wrong.

You're reading it wrong.  That's just the amount used of the current PermGen 
size; it's nowhere near your max, so that's not the problem.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 And again in a new place on subsequent attempt.

There's nothing unusual that I can see in the traces, so it looks like you will 
have to try random GC settings until something works (as Mark suggested some 
time ago).  Try turning these tags on, one at a time:

-XX:+UseParallelGC
-XX:+UseParallelOldGC
-XX:+UseSerialGC
-XX:+DoEscapeAnalysis
-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC

The last one is somewhat experimental, as the parameter implies.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Ok, I'll try that next. Two things:
- Did some searching, but couldn't find how to determine the default GC
method for my Java version
- Would you recommend I move this problem elsewhere, given that this is
likely not related to Tomcat at all. I don't want to spam this list.

Thanks,
Bradley

On Thu, Sep 3, 2009 at 4:24 PM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  And again in a new place on subsequent attempt.

 There's nothing unusual that I can see in the traces, so it looks like you
 will have to try random GC settings until something works (as Mark suggested
 some time ago).  Try turning these tags on, one at a time:

 -XX:+UseParallelGC
 -XX:+UseParallelOldGC
 -XX:+UseSerialGC
 -XX:+DoEscapeAnalysis
 -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC

 The last one is somewhat experimental, as the parameter implies.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 - Did some searching, but couldn't find how to determine the default GC
 method for my Java version

You appear to be running the parallel GC; that's one of the things JConsole 
would tell you.  There are probably other ways to determine the setting, but 
unfortunately I've forgotten them.  Lambda Probe does display the heap types, 
but you have to have Tomcat running to use it...

 - Would you recommend I move this problem elsewhere, given that this is
 likely not related to Tomcat at all. I don't want to spam this list.

That's why people have delete keys.  Just add [OT] to the subject line, if you 
think it's appropriate.  Whatever is eventually discovered may well help other 
Tomcat users.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread Mark Thomas

Bradley Wagner wrote:
 Ok, I'll try that next. Two things:
 - Did some searching, but couldn't find how to determine the default GC
 method for my Java version

http://blogs.sun.com/watt/resource/jvm-options-list.html is a useful
reference

 - Would you recommend I move this problem elsewhere, given that this is
 likely not related to Tomcat at all. I don't want to spam this list.

No need. GC freezes are close to the most common issue I see with
unstable production Tomcat systems so I think it is very relevant.

Mark




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread Ziggy

I noticed that you mention that you are using the community version of the
Java runtime environment which i think is what you get when you do an
apt-get.

I have never honestly been able to use this version of the JVM successfully.
I had a similar problem recently where i was running an application (That
uses Hibernate) using this version of the JDK and was always getting memory
exceptions.

What i did was to download sun's version of the JDK and that seem to have
solved the problem. Did you try to use the SUN's version of the JDK from the
SUN website? It might be usefull to try the SUN's version just to rule out
that the JDK version/release is not the problem.



On Wed, Sep 2, 2009 at 11:26 PM, Mark Thomas ma...@apache.org wrote:

 Bradley Wagner wrote:
  Any thoughts on which GC parameters I should change.

 See what the default collector is for your platform and switch to s
 different one.

 Mark




 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Sorry, are you saying that the JRE I'm getting when doing apt-get install
sun-java6-jdk is not actually a Sun certified version and that I should be
downloading it directly from their website instead?

On Thu, Sep 3, 2009 at 3:50 AM, Ziggy zigg...@gmail.com wrote:

 I noticed that you mention that you are using the community version of the
 Java runtime environment which i think is what you get when you do an
 apt-get.

 I have never honestly been able to use this version of the JVM
 successfully.
 I had a similar problem recently where i was running an application (That
 uses Hibernate) using this version of the JDK and was always getting memory
 exceptions.

 What i did was to download sun's version of the JDK and that seem to have
 solved the problem. Did you try to use the SUN's version of the JDK from
 the
 SUN website? It might be usefull to try the SUN's version just to rule out
 that the JDK version/release is not the problem.



 On Wed, Sep 2, 2009 at 11:26 PM, Mark Thomas ma...@apache.org wrote:

  Bradley Wagner wrote:
   Any thoughts on which GC parameters I should change.
 
  See what the default collector is for your platform and switch to s
  different one.
 
  Mark
 
 
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
  For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 are you saying that the JRE I'm getting when doing apt-get
 install sun-java6-jdk is not actually a Sun certified version
 and that I should be downloading it directly from their website
 instead?

It's certainly something to try.  You can find out exactly what you have 
installed by doing java -version (without the quotes).

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Yea, I don't think it's the Open JDK, though that was an option. java
-version gives:
java version 1.6.0_14
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)

- Bradley

On Thu, Sep 3, 2009 at 10:19 AM, Ziggy zigg...@gmail.com wrote:

 What do you get when you run java -version?

 I think the linux distributions include the open jdk version see
 http://openjdk.java.net/




 On Thu, Sep 3, 2009 at 3:13 PM, Caldarale, Charles R 
 chuck.caldar...@unisys.com wrote:

   From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
   Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
   in Tomcat 5.5 on Ubuntu 7
  
   are you saying that the JRE I'm getting when doing apt-get
   install sun-java6-jdk is not actually a Sun certified version
   and that I should be downloading it directly from their website
   instead?
 
  It's certainly something to try.  You can find out exactly what you have
  installed by doing java -version (without the quotes).
 
   - Chuck
 
 
  THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
  MATERIAL and is thus for use only by the intended recipient. If you
 received
  this in error, please contact the sender and delete the e-mail and its
  attachments from all computers.
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
  For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread ramzi khlil

It provides you the version of installed java.

On Thu, Sep 3, 2009 at 10:19 AM, Ziggy zigg...@gmail.com wrote:

 What do you get when you run java -version?

 I think the linux distributions include the open jdk version see
 http://openjdk.java.net/




 On Thu, Sep 3, 2009 at 3:13 PM, Caldarale, Charles R 
 chuck.caldar...@unisys.com wrote:

From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
   Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
   in Tomcat 5.5 on Ubuntu 7
  
   are you saying that the JRE I'm getting when doing apt-get
   install sun-java6-jdk is not actually a Sun certified version
   and that I should be downloading it directly from their website
   instead?
 
  It's certainly something to try.  You can find out exactly what you have
  installed by doing java -version (without the quotes).
 
   - Chuck
 
 
  THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
  MATERIAL and is thus for use only by the intended recipient. If you
 received
  this in error, please contact the sender and delete the e-mail and its
  attachments from all computers.
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
  For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread Ziggy

Ok my mistake. Thats the official SUN version.

The linux distribution i use downloads the openjdk version and i had to
manually download the official one to stop the memory errors.



On Thu, Sep 3, 2009 at 3:29 PM, ramzi khlil ramzi.atv...@gmail.com wrote:

 It provides you the version of installed java.

 On Thu, Sep 3, 2009 at 10:19 AM, Ziggy zigg...@gmail.com wrote:

  What do you get when you run java -version?
 
  I think the linux distributions include the open jdk version see
  http://openjdk.java.net/
 
 
 
 
  On Thu, Sep 3, 2009 at 3:13 PM, Caldarale, Charles R 
  chuck.caldar...@unisys.com wrote:
 
 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
Subject: Re: 100% usage and hanging on startup of Spring/Hibernate
 app
in Tomcat 5.5 on Ubuntu 7
   
are you saying that the JRE I'm getting when doing apt-get
install sun-java6-jdk is not actually a Sun certified version
and that I should be downloading it directly from their website
instead?
  
   It's certainly something to try.  You can find out exactly what you
 have
   installed by doing java -version (without the quotes).
  
- Chuck
  
  
   THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE
 PROPRIETARY
   MATERIAL and is thus for use only by the intended recipient. If you
  received
   this in error, please contact the sender and delete the e-mail and its
   attachments from all computers.
  
  
   -
   To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
   For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 java version 1.6.0_14
 Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
 Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)

That's a real one.  This is getting strange.  Can you run JConsole against the 
Tomcat JVM and see what the state of the various heap parts are when it gets 
into the hung state?

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread Ziggy

What do you get when you run java -version?

I think the linux distributions include the open jdk version see
http://openjdk.java.net/




On Thu, Sep 3, 2009 at 3:13 PM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  are you saying that the JRE I'm getting when doing apt-get
  install sun-java6-jdk is not actually a Sun certified version
  and that I should be downloading it directly from their website
  instead?

 It's certainly something to try.  You can find out exactly what you have
 installed by doing java -version (without the quotes).

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Is JConsole something that comes bundled with the JRE or JDK or a separate
application. I have limited ability to install other things on this machine
because it belongs to a client.
Thanks,
Bradley

On Thu, Sep 3, 2009 at 10:32 AM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  java version 1.6.0_14
  Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
  Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)

 That's a real one.  This is getting strange.  Can you run JConsole against
 the Tomcat JVM and see what the state of the various heap parts are when it
 gets into the hung state?

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Probably unrelated, but... when setting JAVA_HOME for Tomcat in Ubuntu
what's the difference between:
- /usr/lib/jvm/java-6-sun, and
- /usr/lib/jvm/java-6-sun/jre

It doesn't seem to have any effect on my problem, but does it make a
difference? I believe I have the full JDK installed so maybe the second path
only points to the binaries for the JRE?

- Bradley

On Thu, Sep 3, 2009 at 10:31 AM, Ziggy zigg...@gmail.com wrote:

 Ok my mistake. Thats the official SUN version.

 The linux distribution i use downloads the openjdk version and i had to
 manually download the official one to stop the memory errors.



 On Thu, Sep 3, 2009 at 3:29 PM, ramzi khlil ramzi.atv...@gmail.com
 wrote:

  It provides you the version of installed java.
 
  On Thu, Sep 3, 2009 at 10:19 AM, Ziggy zigg...@gmail.com wrote:
 
   What do you get when you run java -version?
  
   I think the linux distributions include the open jdk version see
   http://openjdk.java.net/
  
  
  
  
   On Thu, Sep 3, 2009 at 3:13 PM, Caldarale, Charles R 
   chuck.caldar...@unisys.com wrote:
  
  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate
  app
 in Tomcat 5.5 on Ubuntu 7

 are you saying that the JRE I'm getting when doing apt-get
 install sun-java6-jdk is not actually a Sun certified version
 and that I should be downloading it directly from their website
 instead?
   
It's certainly something to try.  You can find out exactly what you
  have
installed by doing java -version (without the quotes).
   
 - Chuck
   
   
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE
  PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
   received
this in error, please contact the sender and delete the e-mail and
 its
attachments from all computers.
   
   
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-03 Thread Mark Thomas

Bradley Wagner wrote:
 Probably unrelated, but... when setting JAVA_HOME for Tomcat in Ubuntu
 what's the difference between:
 - /usr/lib/jvm/java-6-sun, and
 - /usr/lib/jvm/java-6-sun/jre
 
 It doesn't seem to have any effect on my problem, but does it make a
 difference? I believe I have the full JDK installed so maybe the second path
 only points to the binaries for the JRE?

JVM hangs with the garbage collector are a fairly regular occurrence.
Have you tried changing the GC settings yet?

Mark



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 Is JConsole something that comes bundled with the JRE or JDK or a
 separate application.

It's part of the JDK, but not the JRE.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 Probably unrelated, but... when setting JAVA_HOME for Tomcat in Ubuntu
 what's the difference between:
 - /usr/lib/jvm/java-6-sun, and
 - /usr/lib/jvm/java-6-sun/jre

Every Sun JDK has a JRE inside it; the first directory above is for the full 
JDK, the second for just the JRE subset, as you surmised.  JConsole should be 
located in /usr/lib/jvm/java-6-sun/bin.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Hmm, so JConsole:
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html appears
to need a GUI which I don't have in this case.
Should I just start switching through the different GC types and see if it
makes a difference?

On Thu, Sep 3, 2009 at 11:02 AM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Probably unrelated, but... when setting JAVA_HOME for Tomcat in Ubuntu
  what's the difference between:
  - /usr/lib/jvm/java-6-sun, and
  - /usr/lib/jvm/java-6-sun/jre

 Every Sun JDK has a JRE inside it; the first directory above is for the
 full JDK, the second for just the JRE subset, as you surmised.  JConsole
 should be located in /usr/lib/jvm/java-6-sun/bin.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 Hmm, so JConsole appears to need a GUI which I don't have in this case.

You can run it remotely by setting the appropriate command-line options.

 Should I just start switching through the different GC types and see if
 it makes a difference?

If you can't get more detailed information about what's going on inside the 
heap, then yes.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Ok, I've tried to get JConsole working but have been unsuccessful. I've set
it up to connect remotely using JMX over port 8081. The program is starting
and I've configured the JMX passwords.
However, when I try to connect to the hung program using username:
controlRole and my password: password nothing pops up. I see the following
catalina.out:

2009-09-03 12:18:04,295 INFO  [XmlWebApplicationContext] : Bean factory for
application context
[org.springframework.web.context.support.xmlwebapplicationcont...@1f1680f]:
org.springframework.beans.factory.support.defaultlistablebeanfact...@45378f
[GC [PSYoungGen: 50993K-7222K(51008K)] 51106K-8565K(517056K), 0.0402520
secs] [Times: user=0.03 sys=0.01, real=0.04 secs]
[GC


However, it is is listing my application name in JConsole's list of windows.
Maybe it can't connect when it's hung.

- Bradley

On Thu, Sep 3, 2009 at 11:17 AM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Hmm, so JConsole appears to need a GUI which I don't have in this case.

 You can run it remotely by setting the appropriate command-line options.

  Should I just start switching through the different GC types and see if
  it makes a difference?

 If you can't get more detailed information about what's going on inside the
 heap, then yes.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 Maybe it can't connect when it's hung.

Quite possible; GC stops all threads, so the JMX agent may well be suspended.

You might want to try turning on pretty much all of the GC-related trace flags:

-XX:+PrintGC
-XX:+PrintGCDetails
-XX:+PrintGCTimeStamps
-XX:+PrintGCTaskTimeStamps
-XX:+PrintReferenceGC
-XX:+TraceGen0Time
-XX:+TraceGen1Time
-XX:+PrintTenuringDistribution
-XX:+PrintHeapAtGC
-XX:+TraceParallelOldGCTasks
-XX:+PrintParallelOldGCPhaseTimes

(All on one line, of course.)  This will produce a *lot* of output.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Ok, hung in a different place, but hung nonetheless with all of that logging
enabled:
Here is the result:

2009-09-03 15:47:11,328 INFO  [SettingsFactory] : Default entity-mode: pojo
2009-09-03 15:47:11,328 INFO  [SettingsFactory] : Named query checking :
enabled
2009-09-03 15:47:11,355 INFO  [SessionFactoryImpl] : building session
factory
{Heap before GC invocations=10 (full 0):
 PSYoungGen  total 47808K, used 43957K [0xb087, 0xb415,
0xb415)
  eden space 37504K, 100% used [0xb087,0xb2d1,0xb2d1)
  from space 10304K, 62% used [0xb2d1,0xb335d6f0,0xb372)
  to   space 9984K, 0% used [0xb379,0xb379,0xb415)
 PSOldGentotal 466048K, used 11257K [0x9415, 0xb087,
0xb087)
  object space 466048K, 2% used [0x9415,0x94c4e4f0,0xb087)
 PSPermGen   total 26240K, used 26158K [0x9015, 0x91af,
0x9415)
  object space 26240K, 99% used [0x9015,0x91adb890,0x91af)
10.247: [GC10.277: [SoftReference, 0 refs, 0.100 secs]10.277:
[WeakReference, 210 refs, 0.530 secs]10.277: [FinalReference, 124 refs,
0.580 secs]10.277: [PhantomReference, 0 refs, 0.070 secs]10.277:
[JNI Weak Reference, 0.010 secs]
Desired survivor size 10616832 bytes, new threshold 1 (max 15)
 [PSYoungGen: 43957K-2894K(47488K)] 55214K-19098K(513536K), 0.0304750
secs] [Times: user=0.01 sys=0.05, real=0.03 secs]
Heap after GC invocations=10 (full 0):
 PSYoungGen  total 47488K, used 2894K [0xb087, 0xb415,
0xb415)
  eden space 37504K, 0% used [0xb087,0xb087,0xb2d1)
  from space 9984K, 28% used [0xb379,0xb3a639d0,0xb415)
  to   space 10368K, 0% used [0xb2d1,0xb2d1,0xb373)
 PSOldGentotal 466048K, used 16203K [0x9415, 0xb087,
0xb087)
  object space 466048K, 3% used [0x9415,0x95122ed8,0xb087)
 PSPermGen   total 26240K, used 26158K [0x9015, 0x91af,
0x9415)
  object space 26240K, 99% used [0x9015,0x91adb890,0x91af)
}
VM-Thread 10246673 10276996 10277531
GC-Thread 0 entries: 17
[ old-to-young-roots-task 10246926 10272563 ]
[ scavenge-roots-task 10272571 10272574 ]
[ scavenge-roots-task 10272574 10272590 ]
[ thread-roots-task 10272591 10272612 ]
[ thread-roots-task 10272613 10272618 ]
[ thread-roots-task 10272623 10272631 ]
[ thread-roots-task 10272631 10272654 ]
[ thread-roots-task 10272655 10272660 ]
[ thread-roots-task 10272660 10272667 ]
[ thread-roots-task 10272667 10272669 ]
[ thread-roots-task 10272669 10272672 ]
[ thread-roots-task 10272672 10272677 ]
[ thread-roots-task 10272678 10272679 ]
[ thread-roots-task 10272680 10272686 ]
[ thread-roots-task 10272687 10272690 ]
[ thread-roots-task 10272691 10276295 ]
[ steal-task 10276297 10276969 ]
GC-Thread 1 entries: 10
[ old-to-young-roots-task 10246939 10269250 ]
[ serial-old-to-young-roots-task 10269253 10274454 ]
[ thread-roots-task 10274457 10274461 ]
[ scavenge-roots-task 10274462 10274473 ]
[ scavenge-roots-task 10274474 10274478 ]
[ scavenge-roots-task 10274478 10274483 ]
[ scavenge-roots-task 10274483 10274919 ]
[ scavenge-roots-task 10274920 10274921 ]
[ steal-task 10274922 10276966 ]
[ waitfor-barrier-task 10276967 10276991 ]
{Heap before GC invocations=11 (full 0):
 PSYoungGen  total 47488K, used 40398K [0xb087, 0xb415,
0xb415)
  eden space 37504K, 100% used [0xb087,0xb2d1,0xb2d1)
  from space 9984K, 28% used [0xb379,0xb3a639d0,0xb415)
  to   space 10368K, 0% used [0xb2d1,0xb2d1,0xb373)
 PSOldGentotal 466048K, used 16203K [0x9415, 0xb087,
0xb087)
  object space 466048K, 3% used [0x9415,0x95122ed8,0xb087)
 PSPermGen   total 26752K, used 26626K [0x9015, 0x91b7,
0x9415)
  object space 26752K, 99% used [0x9015,0x91b50878,0x91b7)
10.754: [GC

Once again, it terminates in the middle of GC logging.

- Bradley

On Thu, Sep 3, 2009 at 1:49 PM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Maybe it can't connect when it's hung.

 Quite possible; GC stops all threads, so the JMX agent may well be
 suspended.

 You might want to try turning on pretty much all of the GC-related trace
 flags:

 -XX:+PrintGC
 -XX:+PrintGCDetails
 -XX:+PrintGCTimeStamps
 -XX:+PrintGCTaskTimeStamps
 -XX:+PrintReferenceGC
 -XX:+TraceGen0Time
 -XX:+TraceGen1Time
 -XX:+PrintTenuringDistribution
 -XX:+PrintHeapAtGC
 -XX:+TraceParallelOldGCTasks
 -XX:+PrintParallelOldGCPhaseTimes

 (All on one line, of course.)  This will produce a *lot* of output.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

And again in a new place on subsequent attempt. I've attached the logging
this time because the amount of logging between the last Tomcat line and the
GC line was significant!
- Bradley

On Thu, Sep 3, 2009 at 1:49 PM, Caldarale, Charles R 
chuck.caldar...@unisys.com wrote:

  From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
  Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
  in Tomcat 5.5 on Ubuntu 7
 
  Maybe it can't connect when it's hung.

 Quite possible; GC stops all threads, so the JMX agent may well be
 suspended.

 You might want to try turning on pretty much all of the GC-related trace
 flags:

 -XX:+PrintGC
 -XX:+PrintGCDetails
 -XX:+PrintGCTimeStamps
 -XX:+PrintGCTaskTimeStamps
 -XX:+PrintReferenceGC
 -XX:+TraceGen0Time
 -XX:+TraceGen1Time
 -XX:+PrintTenuringDistribution
 -XX:+PrintHeapAtGC
 -XX:+TraceParallelOldGCTasks
 -XX:+PrintParallelOldGCPhaseTimes

 (All on one line, of course.)  This will produce a *lot* of output.

  - Chuck


 THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
 MATERIAL and is thus for use only by the intended recipient. If you received
 this in error, please contact the sender and delete the e-mail and its
 attachments from all computers.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org


2009-09-03 15:47:31,314 INFO  [TableMetadata] : columns: [xmldata, id, version]
{Heap before GC invocations=16 (full 0):
 PSYoungGen  total 47168K, used 39534K [0xb087, 0xb415, 0xb415)
  eden space 36352K, 100% used [0xb087,0xb2bf,0xb2bf)
  from space 10816K, 29% used [0xb2bf,0xb2f0b9a8,0xb368)
  to   space 10496K, 0% used [0xb371,0xb371,0xb415)
 PSOldGentotal 466048K, used 27270K [0x9415, 0xb087, 0xb087)
  object space 466048K, 5% used [0x9415,0x95bf1b80,0xb087)
 PSPermGen   total 28032K, used 27963K [0x9015, 0x91cb, 0x9415)
  object space 28032K, 99% used [0x9015,0x91c9eed0,0x91cb)
30.488: [GC30.496: [SoftReference, 0 refs, 0.100 secs]30.496: 
[WeakReference, 60 refs, 0.240 secs]30.496: [FinalReference, 0 refs, 
0.060 secs]30.496: [PhantomReference, 0 refs, 0.060 secs]30.496: [JNI 
Weak Reference, 0.020 secs]
Desired survivor size 10747904 bytes, new threshold 1 (max 15)
 [PSYoungGen: 39534K-1648K(47744K)] 66805K-32097K(513792K), 0.0086770 secs] 
[Times: user=0.00 sys=0.01, real=0.01 secs] 
Heap after GC invocations=16 (full 0):
 PSYoungGen  total 47744K, used 1648K [0xb087, 0xb415, 0xb415)
  eden space 37248K, 0% used [0xb087,0xb087,0xb2cd)
  from space 10496K, 15% used [0xb371,0xb38ac140,0xb415)
  to   space 10496K, 0% used [0xb2cd,0xb2cd,0xb371)
 PSOldGentotal 466048K, used 30449K [0x9415, 0xb087, 0xb087)
  object space 466048K, 6% used [0x9415,0x95f0c480,0xb087)
 PSPermGen   total 28032K, used 27963K [0x9015, 0x91cb, 0x9415)
  object space 28032K, 99% used [0x9015,0x91c9eed0,0x91cb)
}
VM-Thread 30487334 30495913 30496387
GC-Thread 0 entries: 24
[ old-to-young-roots-task 30487618 30491453 ]
[ serial-old-to-young-roots-task 30491456 30491653 ]
[ scavenge-roots-task 30491654 30491657 ]
[ scavenge-roots-task 30491657 30491677 ]
[ thread-roots-task 30491678 30491698 ]
[ thread-roots-task 30491699 30491704 ]
[ thread-roots-task 30491705 30491712 ]
[ thread-roots-task 30491713 30491732 ]
[ thread-roots-task 30491732 30491741 ]
[ thread-roots-task 30491742 30491748 ]
[ thread-roots-task 30491748 30491750 ]
[ thread-roots-task 30491751 30491752 ]
[ thread-roots-task 30491754 30491755 ]
[ thread-roots-task 30491756 30491757 ]
[ thread-roots-task 30491758 30491764 ]
[ thread-roots-task 30491764 30491770 ]
[ thread-roots-task 30491770 30492076 ]
[ thread-roots-task 30492077 30492079 ]
[ scavenge-roots-task 30492080 30492089 ]
[ scavenge-roots-task 30492089 30492091 ]
[ scavenge-roots-task 30492091 30492094 ]
[ scavenge-roots-task 30492095 30492512 ]
[ scavenge-roots-task 30492512 30492514 ]
[ steal-task 30492515 30495882 ]
GC-Thread 1 entries: 3
[ old-to-young-roots-task 30487602 30494947 ]
[ steal-task 30494952 30495881 ]
[ waitfor-barrier-task 30495882 30495897 ]
{Heap before GC invocations=17 (full 0):
 PSYoungGen  total 47744K, used 38896K [0xb087, 0xb415, 0xb415)
  eden space 37248K, 100% used [0xb087,0xb2cd,0xb2cd)
  from space 10496K, 15% used [0xb371,0xb38ac140,0xb415)
  to   space 10496K, 0% used [0xb2cd,0xb2cd,0xb371)
 PSOldGentotal 466048K, used 30449K [0x9415, 0xb087, 0xb087)

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-02 Thread Bradley Wagner

Thanks for the clarification on the initial heap size. I've set both Xms and
Xmx to 512m for the following test.
I was able to reproduce the problem immediately with:
- Xms and Xmx both set to 512m
- Java 1.6 with no JPDA running
- XX:+PrintGCDetails enabled

I've attached the thread dump which I took on the 3rd or 4th attempt after
the process hung. They were all the same on each measure.

I've also attached the catalina.out log which shows the GC basically
freezing.

One thing that it is a little odd is that the Catalina log is basically in
the middle of (or possibly JUST finished) some kind of hibernate
initialization but the blocked thread (the only thread doing anything but
sleeping) doesn't really indicate this.

- Bradley

On Mon, Aug 31, 2009 at 6:19 PM, Leon Rosenberg
rosenberg.l...@googlemail.com wrote:

Hello,
This indeed sounds like you have problems with one of the spaces,
please add following option to the JAVA_OPTS (or whatever you are
using) to seubmit jvm parameters:
-XX:+PrintGCDetails

And, no tomcat's initial heap size don't default to 64Mb. The JVM
allocateds a heap dependent on your computer type, for example the 1.5
VM would identify a 2 core 2 GB machine as 'server' and allocated more
memory than with 1 core cpu.

http://72.5.124.55/docs/hotspot/gc5.0/ergo5.html

regards
Leon

P.S. I know the link looks strange it come ups from google:

http://www.google.de/search?q=hotspot+memory+ergonomicsie=utf-8oe=utf-8aq=trls=org.mozilla:en-US:officialclient=firefox-a

P.P.S I had a similar situation once, as a client had set space sizes
manually (new and old) and the old space was full. The VM was only
full garbage collecting which led to 100% CPU usage and strange thread
dumps. The above option would help you in this case to identify the
full gc runs previously and during the hangup.

On Mon, Aug 31, 2009 at 11:34 PM, Bradley
Wagnerbradley.wag...@hannonhill.com wrote:

The other interesting point is that all of those are hung in places
where

they would naturally be allocating an object or have very recently
allocated

an object, and the JVM *might* in theory be growing the heap. Bradley,
are

your initial and maximum heap sizes identical (I suspect not from your

original message)? If not, what happens if you make them so?

From my original post, I'm setting the following memory parameters:

-Xmx512M -XX:MaxPermSize=128m

so I'm not setting an initial heap size at all. If I remember correctly,
Tomcat's initial heap size default's to 64MB or something.

I'll try to set the initial and max to the same to see if that has any
effect.

Thanks,
Bradley

On Mon, Aug 31, 2009 at 4:59 PM, Peter Crowther
peter.crowt...@melandra.com
wrote:

2009/8/31 Christopher Schultz ch...@christopherschultz.net

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bradley,

On 8/31/2009 4:02 PM, Bradley Wagner wrote:
Sorry, I should have mentioned this before, but in all cases I tried
the
thread dumps 2-3 times at least 30s apart and none of the threads
have
progressed at all.

I agree with Mark: the three threads you showed were different
(different ids), and showing different call stacks. Are you saying
that
each thread dump you took represents multiple thread dumps where that
particular thread didn't progress /at all/ over your multi-dump
sample?

You might even want to run strace to see which library function calls
are actually completing.

Agree.

The other interesting point is that all of those are hung in places
where
they would naturally be allocating an object or have very recently
allocated
an object, and the JVM *might* in theory be growing the heap. Bradley,
are
your initial and maximum heap sizes identical (I suspect not from your
original message)? If not, what happens if you make them so?

- Peter

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Attaching to process ID 26202, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 14.0-b16
Deadlock Detection:

No deadlocks found.

Thread 26216: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Interpreted frame)
- java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
- java.util.TimerThread.mainLoop() @bci=28, line=483 (Interpreted frame)
- java.util.TimerThread.run() @bci=1, line=462 (Interpreted frame)

Thread 26215: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
-
com.hannonhill.cascade.model.search.MemoryQueueSearchJobScheduler$ConsumerThread.run()
@bci=17, line=56 (Interpreted frame)

Thread 26214: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
- com.hannonhill.cascade.view.struts.security.CascadeSessionListener$2.sleep()
@bci=25,

RE: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-02 Thread Caldarale, Charles R

 From: Bradley Wagner [mailto:bradley.wag...@hannonhill.com]
 Subject: Re: 100% usage and hanging on startup of Spring/Hibernate app
 in Tomcat 5.5 on Ubuntu 7

 I've also attached the catalina.out log which shows the GC 
 basically freezing.

Unfortunately, that log was stripped; can you post it somewhere?

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-02 Thread Bradley Wagner

Here's the last 100 lines of it. That line with only 3 letters is actually
what is in the file. It's hanging mid-GC call.
2009-09-02 13:58:05,306 INFO  [TableMetadata] : table found:
hhtest.cxml_recyclerecord
2009-09-02 13:58:05,306 INFO  [TableMetadata] : columns: [id,
parentfolderid, unpublishrequestid, recyclestate, recycledby, recycledon]
2009-09-02 13:58:05,308 INFO  [TableMetadata] : table found:
hhtest.cxml_role_group_link
2009-09-02 13:58:05,308 INFO  [TableMetadata] : columns: [roleid, groupname]
2009-09-02 13:58:05,310 INFO  [TableMetadata] : table found:
hhtest.cxml_role_group_site_link
2009-09-02 13:58:05,310 INFO  [TableMetadata] : columns: [id, roleid,
siteid, groupname]
2009-09-02 13:58:05,312 INFO  [TableMetadata] : table found:
hhtest.cxml_role_user_link
2009-09-02 13:58:05,312 INFO  [TableMetadata] : columns: [username, roleid]
2009-09-02 13:58:05,314 INFO  [TableMetadata] : table found:
hhtest.cxml_role_user_site_link
2009-09-02 13:58:05,314 INFO  [TableMetadata] : columns: [id, username,
roleid, siteid]
2009-09-02 13:58:05,316 INFO  [TableMetadata] : table found:
hhtest.cxml_roles
2009-09-02 13:58:05,317 INFO  [TableMetadata] : columns: [sitesabilities,
toolsabilities, globalabilities, id, rolename, securityareaabilities,
global, homeareaabilities, permissionsid, adminareaabilities]
2009-09-02 13:58:05,320 INFO  [TableMetadata] : table found:
hhtest.cxml_site
2009-09-02 13:58:05,320 INFO  [TableMetadata] : columns: [publishevery,
timetopublish, nextversionid, usesscheduledpublishing, defaultmetadatasetid,
prevversionid, baseassetfactorycontainerid, iscurrentversion,
startingpageid, permissionsid, url, id, cssfileid, metadataid, versiondate,
name, path, publisheverytimeunit, cssclasses, lockid]
2009-09-02 13:58:05,322 INFO  [TableMetadata] : table found:
hhtest.cxml_site_roles
2009-09-02 13:58:05,322 INFO  [TableMetadata] : columns: [roleid, siteid]
2009-09-02 13:58:05,325 INFO  [TableMetadata] : table found:
hhtest.cxml_structureddata
2009-09-02 13:58:05,325 INFO  [TableMetadata] : columns: [blockid,
ownerpageid, fileid, id, name, structureddataid, structureddatatype,
groupstructureddataid, symlinkid, textdata, assettype, pageid, textversion]
2009-09-02 13:58:05,329 INFO  [TableMetadata] : table found:
hhtest.cxml_structureddatadefcont
2009-09-02 13:58:05,329 INFO  [TableMetadata] : columns: [nextversionid,
prevversionid, iscurrentversion, permissionsid, id, isroot, metadataid,
versiondate, name, containerid, path, siteid, lockid]
2009-09-02 13:58:05,332 INFO  [TableMetadata] : table found:
hhtest.cxml_structureddatadefinition
2009-09-02 13:58:05,332 INFO  [TableMetadata] : columns: [nextversionid,
prevversionid, iscurrentversion, permissionsid, id, metadataid, versiondate,
name, containerid, path, siteid, xmlid, lockid]
2009-09-02 13:58:05,335 INFO  [TableMetadata] : table found:
hhtest.cxml_systemproperty
2009-09-02 13:58:05,335 INFO  [TableMetadata] : columns: [id, name, value]
2009-09-02 13:58:05,341 INFO  [TableMetadata] : table found:
hhtest.cxml_target
2009-09-02 13:58:05,341 INFO  [TableMetadata] : columns: [publishpath,
nextversionid, lastdatepublished, shouldincludexmldeclaration,
iscurrentversion, id, isroot, versiondate, containerid, name, path, siteid,
destinationcontainertype, shouldincludetargetpath, basefolderid,
removebasefolderfrompubpath, publishevery, timetopublish,
usesscheduledpublishing, prevversionid, permissionsid, extension,
parenttargetid, cssfileid, metadataid, lastpublishedby, serializationtype,
publisheverytimeunit, cssclasses, lockid]
2009-09-02 13:58:05,345 INFO  [TableMetadata] : table found:
hhtest.cxml_transport
2009-09-02 13:58:05,345 INFO  [TableMetadata] : columns: [port,
databasename, nextversionid, ispassive, foreignsiteid, iscurrentversion,
password, id, username, versiondate, containerid, name, path, siteid,
transporttype, initialdirectory, accountid, ftptransversion, prevversionid,
servername, permissionsid, metadataid, directory, issecure, packetsize,
lockid]
2009-09-02 13:58:05,348 INFO  [TableMetadata] : table found:
hhtest.cxml_transportcontainer
2009-09-02 13:58:05,348 INFO  [TableMetadata] : columns: [nextversionid,
prevversionid, iscurrentversion, permissionsid, id, isroot, metadataid,
versiondate, name, containerid, path, siteid, lockid]
2009-09-02 13:58:05,351 INFO  [TableMetadata] : table found:
hhtest.cxml_user
2009-09-02 13:58:05,351 INFO  [TableMetadata] : columns: [isenabled,
username, hash, email, ldapdn, authmode, defaultsiteid, defaultgroup,
password, fullname]
2009-09-02 13:58:05,376 INFO  [TableMetadata] : table found:
hhtest.cxml_workflow
2009-09-02 13:58:05,376 INFO  [TableMetadata] : columns: [historyid,
definitionxml, expirationwarningsent, id, currentstepid, startdate,
relatedentitytype, name, owner, isinitialized, iscompleted, relatedentityid,
enddate, firststepid]
2009-09-02 13:58:05,379 INFO  [TableMetadata] : table found:
hhtest.cxml_workflowaction
2009-09-02 13:58:05,379 INFO  [TableMetadata] : columns: [id,

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-02 Thread Mark Thomas

Bradley Wagner wrote:
 Any thoughts on which GC parameters I should change.

See what the default collector is for your platform and switch to s
different one.

Mark




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-01 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bradley,

On 8/31/2009 5:27 PM, Bradley Wagner wrote:
 The three thread dumps were from three different occurrences of the same
 problem and three consecutive startup attempts.
 
 With each occurrence, the startup process hangs in the middle of
 initialization. I took 2-3 jstack dumps that were identical each time. After
 that, I killed the process (kill -9 pid) and started it again.
 
 So, while it's not hanging in the same place every time, it's definitely
 hanging every time. And because it's the main initialization thread that is
 hanging, once it hangs, it never gets any further.

Hmm. Do you trust your hardware?

If you've tried several JVM versions and client vs. server (which are
essentially different JVMs) and they are all having problems, then the
JVM itself is probably not to blame (although using JPDA gets you off
the hook? weird: try running in interpreted mode... run with -Xint just
for giggles).

How much physical memory does your machine have? How much swap space?
Try running badblocks on your swap partition. Try running memtest86+ on
your hardware (if it's x86... you didn't say).

Are you running the latest kernel version available? We use
2.6.30-gentoo-r4 on one of our development boxes, 2.6.26-2-openvz-686
(from Debian) on another development box (it's an openvz VM),
2.6.18-6-686 (Debian, again) in production, and 2.4.27-2-686-smp (also
Debian... 2.4? Yikes?!) in demo. All of these seem to have no problem
running our applications for months and months.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqdNiQACgkQ9CaO5/Lv0PDOMgCgsxCDcoOnQA8eKfVDumfER1I7
eQkAn1JrCr0k7mWJyTusbcnU6Tf4Raj2
=35B0
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-01 Thread Peter Crowther

2009/9/1 Christopher Schultz ch...@christopherschultz.net

 Hmm. Do you trust your hardware?
 [...]

How much physical memory does your machine have? How much swap space?
 Try running badblocks on your swap partition. Try running memtest86+ on
 your hardware (if it's x86... you didn't say).


The OP stated this was in a virtual machine.  Some, but not all, of this
advice is appropriate in such an environment :-).

- Peter

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-09-01 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Peter,

On 9/1/2009 11:18 AM, Peter Crowther wrote:
 2009/9/1 Christopher Schultz ch...@christopherschultz.net
 
 Hmm. Do you trust your hardware?
 [...]
 
 How much physical memory does your machine have? How much swap space?
 Try running badblocks on your swap partition. Try running memtest86+ on
 your hardware (if it's x86... you didn't say).

 
 The OP stated this was in a virtual machine.  Some, but not all, of this
 advice is appropriate in such an environment :-).

Whoops! In that case, run vmemtest86+ ha ha ha :)

Seriously, I would check the VM status. There is usually some way to
observe which soft/hard limits the VM has exceeded. For instance, in
Virtuozzo-based VMs, you can check /proc/user_beancounters. Other
techniques must exist for other virtualization technologies (except
maybe host/guest types like VMWare player, workstation, etc., since they
work differently).

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqdPx8ACgkQ9CaO5/Lv0PB4MwCdEUmO88r4qZCXCmim1aKwZReV
HcAAn1P4kXpBIlSt1jjmgXbjayRGyB2b
=vetW
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Ok, I have some information to add here. I was able to get thread dumps with
jstack pid (very handy!) because kill -3 wouldn't work on a hung
process.
I've listed 3 consecutive tests that I run and the thread is blocked at the
time the process is hung.

Test 1

Thread 21348: (state = BLOCKED)
 - java.lang.StringCoding$CharsetSE.encode(char[], int, int) @bci=15,
line=334 (Compiled frame)
 - java.lang.StringCoding.encode(java.lang.String, char[], int, int)
@bci=123, line=378 (Compiled frame)
 - java.lang.String.getBytes(java.lang.String) @bci=25, line=812 (Compiled
frame)
 - java.io.UnixFileSystem.getBooleanAttributes0(java.io.File) @bci=0
(Compiled frame)
 - java.io.UnixFileSystem.getBooleanAttributes(java.io.File) @bci=2,
line=228 (Compiled frame)
 - java.io.File.exists() @bci=20, line=702 (Compiled frame)

This is the main thread and it is hung on startup during the Spring
initialization process.

Test 2

Thread 24193: (state = BLOCKED)
 - java.lang.String.init(char[], int, int) @bci=50, line=208 (Compiled
frame)
 - java.io.DataInputStream.readUTF() @bci=1, line=522 (Compiled frame)
 - org.aspectj.apache.bcel.classfile.ClassParser.readConstantPool() @bci=9,
line=186 (Interpreted frame)
 - org.aspectj.apache.bcel.classfile.ClassParser.parse() @bci=9, line=131
(Interpreted frame)
 -
org.aspectj.apache.bcel.util.NonCachingClassLoaderRepository.loadClass(java.lang.String)
@bci=100, line=226 (Compiled frame)
 -
org.aspectj.apache.bcel.util.NonCachingClassLoaderRepository.loadClass(java.lang.Class)
@bci=5, line=237 (Interpreted frame)
 -
org.aspectj.weaver.reflect.Java15AnnotationFinder.getParameterAnnotationTypes(java.lang.reflect.Member)
@bci=21, line=353 (Compiled frame)

Once again hung on an I/O operation this time in String's constructor

Test 3

Thread 24365: (state = BLOCKED)
 - java.lang.String.init(char[], int, int) @bci=50, line=208 (Compiled
frame)
 - net.sf.cglib.proxy.Enhancer.access$300(net.sf.cglib.proxy.Enhancer,
net.sf.cglib.core.CodeEmitter, int) @bci=3, line=60 (Interpreted frame)
 - net.sf.cglib.proxy.Enhancer$6.emitCallback(net.sf.cglib.core.CodeEmitter,
int) @bci=6, line=913 (Interpreted frame)
 -
net.sf.cglib.proxy.InvocationHandlerGenerator.generate(net.sf.cglib.core.ClassEmitter,
net.sf.cglib.proxy.CallbackGenerator$Context, java.util.List) @bci=85,
line=44 (Compiled frame)
 - net.sf.cglib.proxy.Enhancer.emitMethods(net.sf.cglib.core.ClassEmitter,
java.util.List, java.util.List) @bci=415, line=942 (Interpreted frame)
 - net.sf.cglib.proxy.Enhancer.generateClass(net.sf.cglib.asm.ClassVisitor)
@bci=353, line=498 (Interpreted frame)
 -
net.sf.cglib.core.DefaultGeneratorStrategy.generate(net.sf.cglib.core.ClassGenerator)
@bci=11, line=25 (Interpreted frame)
 - net.sf.cglib.core.AbstractClassGenerator.create(java.lang.Object)
@bci=182, line=216 (Interpreted frame)
 - net.sf.cglib.proxy.Enhancer.createHelper() @bci=105, line=377
(Interpreted frame)
 - net.sf.cglib.proxy.Enhancer.createClass() @bci=6, line=317 (Interpreted
frame)

with the last line in the Catalina log reading:

2009-08-31 14:35:46,261 INFO  [SettingsFactory] : Named query checking :
enabled
2009-08-31 14:35:46,284 INFO  [SessionFactoryImpl] : building session
factory

In all examples, the software appears to be hung on fairly innocuous methods
in the java framework itself. Test 1 and 2 both originate from calls in the
IO package and the third test from the CGlib framework that we used from
Hibernate. In all instances, there is a call to a method/constructor on the
String object in the stack.

I'm about to try Java 6 to see if it makes a difference.

Thanks,
Bradley

On Sun, Aug 30, 2009 at 2:56 AM, Peter Crowther peter.crowt...@melandra.com
 wrote:

 2009/8/28 Bradley Wagner bradley.wag...@hannonhill.com

  I have a Spring/Hibernate app running in Tomcat 5.5.20 that we've tested
 in
  many environments that is currently faltering when running in a Ubuntu 7
  VM.
  Basically on startup, frequently, the startup process will halt when
 trying
  to read Hibernate's HBM files and in various other places in startup
  process. When it stops both CPUs are pegged at 200% usage by the java
  process (from 'top'). At this point, the only recourse is to kill the app
  with kill -9 pid. Occasionally the app will start all the way up.
 Then,
  I can trigger a re-index of the content in the app's database using
 Lucene
  and the app will freeze again.
 
  The only thing I've been able to find in common about these operations is
  that they seem to be heavy I/O.


 One trick would be to take a thread dump when the app locks up.  kill -3
 pid will trigger such a dump.  The output will be in one of Tomcat's
 logfiles (catalina.out by default, I think).  That might allow you to get
 debugging information, as you're getting the data without having to enable
 JPDA.

 Sometimes it's useful to take several thread dumps, a few seconds apart,
 and
 analyse them.  However, given your suspicions of a race condition, I'm not
 sure that's appropriate

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-08-31 Thread Mark Thomas

Bradley Wagner wrote:
 Ok, I have some information to add here. I was able to get thread dumps with
 jstack pid (very handy!) because kill -3 wouldn't work on a hung
 process.
 I've listed 3 consecutive tests that I run and the thread is blocked at the
 time the process is hung.

That isn't the way to analyse this. It is almost impossible to detect a
hung thread from a single thread dump.

For a single hang, take three thread dumps ~10s apart and then examine
those to see which thread(s) has(have) not progressed between dumps.

Mark




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

Mark,
Sorry, I should have mentioned this before, but in all cases I tried the
thread dumps 2-3 times at least 30s apart and none of the threads have
progressed at all.

- Bradley

On Mon, Aug 31, 2009 at 3:52 PM, Mark Thomas ma...@apache.org wrote:

 Bradley Wagner wrote:
  Ok, I have some information to add here. I was able to get thread dumps
 with
  jstack pid (very handy!) because kill -3 wouldn't work on a hung
  process.
  I've listed 3 consecutive tests that I run and the thread is blocked at
 the
  time the process is hung.

 That isn't the way to analyse this. It is almost impossible to detect a
 hung thread from a single thread dump.

 For a single hang, take three thread dumps ~10s apart and then examine
 those to see which thread(s) has(have) not progressed between dumps.

 Mark




 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org




-- 
Hannon Hill - CMS Experience You Can Trust
(678) 904-6900 ext 115
http://www.hannonhill.com

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-08-31 Thread Mark Thomas

Bradley Wagner wrote:
 Mark,
 Sorry, I should have mentioned this before, but in all cases I tried the
 thread dumps 2-3 times at least 30s apart and none of the threads have
 progressed at all.

Hmm. That smells like a JVM issue to me, particularly if it goes away
when you use JPDA.

Things to try:
- use a 1.6 JVM
- try different JVM settings, particularly GC related ones
- switch to/from client from/to server JVM

Mark


 
 - Bradley
 
 On Mon, Aug 31, 2009 at 3:52 PM, Mark Thomas ma...@apache.org wrote:
 
 Bradley Wagner wrote:
 Ok, I have some information to add here. I was able to get thread dumps
 with
 jstack pid (very handy!) because kill -3 wouldn't work on a hung
 process.
 I've listed 3 consecutive tests that I run and the thread is blocked at
 the
 time the process is hung.
 That isn't the way to analyse this. It is almost impossible to detect a
 hung thread from a single thread dump.

 For a single hang, take three thread dumps ~10s apart and then examine
 those to see which thread(s) has(have) not progressed between dumps.

 Mark




 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org


 
 




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-08-31 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bradley,

On 8/31/2009 4:02 PM, Bradley Wagner wrote:
 Sorry, I should have mentioned this before, but in all cases I tried the
 thread dumps 2-3 times at least 30s apart and none of the threads have
 progressed at all.

I agree with Mark: the three threads you showed were different
(different ids), and showing different call stacks. Are you saying that
each thread dump you took represents multiple thread dumps where that
particular thread didn't progress /at all/ over your multi-dump sample?

You might even want to run strace to see which library function calls
are actually completing.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqcMT0ACgkQ9CaO5/Lv0PBokQCeOJzGRtwS5a9tx6N3KRCnP/L/
wp4AoLQFItwFOAGWygNNCMizlArc3n66
=jz8W
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7

2009-08-31 Thread Peter Crowther

2009/8/31 Christopher Schultz ch...@christopherschultz.net

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Bradley,

 On 8/31/2009 4:02 PM, Bradley Wagner wrote:
  Sorry, I should have mentioned this before, but in all cases I tried the
  thread dumps 2-3 times at least 30s apart and none of the threads have
  progressed at all.

 I agree with Mark: the three threads you showed were different
 (different ids), and showing different call stacks. Are you saying that
 each thread dump you took represents multiple thread dumps where that
 particular thread didn't progress /at all/ over your multi-dump sample?

 You might even want to run strace to see which library function calls
 are actually completing.


Agree.

The other interesting point is that all of those are hung in places where
they would naturally be allocating an object or have very recently allocated
an object, and the JVM *might* in theory be growing the heap.  Bradley, are
your initial and maximum heap sizes identical (I suspect not from your
original message)?  If not, what happens if you make them so?

- Peter

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7


 - use a 1.6 JVM


Tried with Sun 1.6 JVM. Same problems. Not the same exact stack traces but
for example:

Thread 24427: (state = BLOCKED)
- java.lang.String.toCharArray() @bci=4, line=2725 (Compiled frame)
- java.beans.Introspector.decapitalize(java.lang.String) @bci=46, line=252
(Compiled frame)
- org.hibernate.property.BasicPropertyAccessor.getterMethod(java.lang.Class,
java.lang.String) @bci=47, line=325 (Compiled frame)
-
org.hibernate.property.BasicPropertyAccessor.getGetterOrNull(java.lang.Class,
java.lang.String) @bci=33, line=297 (Interpreted frame)
-
org.hibernate.property.BasicPropertyAccessor.getGetterOrNull(java.lang.Class,
java.lang.String) @bci=71, line=304 (Interpreted frame)
- org.hibernate.property.BasicPropertyAccessor.setterMethod(java.lang.Class,
java.lang.String) @bci=2, line=251 (Interpreted frame)
-
org.hibernate.property.BasicPropertyAccessor.getSetterOrNull(java.lang.Class,
java.lang.String) @bci=33, line=230 (Interpreted frame)
- org.hibernate.property.BasicPropertyAccessor.createSetter(java.lang.Class,
java.lang.String) @bci=2, line=214 (Interpreted frame)
- org.hibernate.property.BasicPropertyAccessor.getSetter(java.lang.Class,
java.lang.String) @bci=2, line=209 (Interpreted frame)
- org.hibernate.mapping.Property.getSetter(java.lang.Class) @bci=10,
line=277 (Interpreted frame)
-
org.hibernate.tuple.entity.PojoEntityTuplizer.buildPropertySetter(org.hibernate.mapping.Property,


- try different JVM settings, particularly GC related ones


Any thoughts on which GC parameters I should change.


 - switch to/from client from/to server JVM


Have tried with -server flag. Same problems.

I agree with you on JVM problem especially given that all the blocked
threads are blocked in calls to the String object. Problem is I'm not doing
anything exotic with the JVM. We simply did an apt-get to get the latest
Sun JVM for Ubuntu. I've been searching their forums as well for any mention
of incompatibilities with Sun's JVM.

Thanks for your help so far.

- Bradley

On Mon, Aug 31, 2009 at 4:22 PM, Mark Thomas ma...@apache.org wrote:

 Bradley Wagner wrote:
  Mark,
  Sorry, I should have mentioned this before, but in all cases I tried the
  thread dumps 2-3 times at least 30s apart and none of the threads have
  progressed at all.

 Hmm. That smells like a JVM issue to me, particularly if it goes away
 when you use JPDA.

 Things to try:
 - use a 1.6 JVM
 - try different JVM settings, particularly GC related ones
 - switch to/from client from/to server JVM

 Mark


 
  - Bradley
 
  On Mon, Aug 31, 2009 at 3:52 PM, Mark Thomas ma...@apache.org wrote:
 
  Bradley Wagner wrote:
  Ok, I have some information to add here. I was able to get thread
dumps
  with
  jstack pid (very handy!) because kill -3 wouldn't work on a hung
  process.
  I've listed 3 consecutive tests that I run and the thread is blocked
at
  the
  time the process is hung.
  That isn't the way to analyse this. It is almost impossible to detect a
  hung thread from a single thread dump.
 
  For a single hang, take three thread dumps ~10s apart and then examine
  those to see which thread(s) has(have) not progressed between dumps.
 
  Mark
 
 
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
  For additional commands, e-mail: users-h...@tomcat.apache.org
 
 
 
 




 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7


 I agree with Mark: the three threads you showed were different

(different ids), and showing different call stacks. Are you saying that

each thread dump you took represents multiple thread dumps where that

particular thread didn't progress /at all/ over your multi-dump sample?


The three thread dumps were from three different occurrences of the same
problem and three consecutive startup attempts.

With each occurrence, the startup process hangs in the middle of
initialization. I took 2-3 jstack dumps that were identical each time. After
that, I killed the process (kill -9 pid) and started it again.

So, while it's not hanging in the same place every time, it's definitely
hanging every time. And because it's the main initialization thread that is
hanging, once it hangs, it never gets any further.

Sorry for the confusion and thanks for the help!

On Mon, Aug 31, 2009 at 4:23 PM, Christopher Schultz 
ch...@christopherschultz.net wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Bradley,

 On 8/31/2009 4:02 PM, Bradley Wagner wrote:
  Sorry, I should have mentioned this before, but in all cases I tried the
  thread dumps 2-3 times at least 30s apart and none of the threads have
  progressed at all.

 I agree with Mark: the three threads you showed were different
 (different ids), and showing different call stacks. Are you saying that
 each thread dump you took represents multiple thread dumps where that
 particular thread didn't progress /at all/ over your multi-dump sample?

 You might even want to run strace to see which library function calls
 are actually completing.

 - -chris
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.9 (MingW32)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

 iEYEARECAAYFAkqcMT0ACgkQ9CaO5/Lv0PBokQCeOJzGRtwS5a9tx6N3KRCnP/L/
 wp4AoLQFItwFOAGWygNNCMizlArc3n66
 =jz8W
 -END PGP SIGNATURE-

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: 100% usage and hanging on startup of Spring/Hibernate app in Tomcat 5.5 on Ubuntu 7