Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-02 Thread Jeanfrancois Arcand
Remy Maucherat wrote: On 2/28/06, Peter Lin [EMAIL PROTECTED] wrote: honestly, besides Weblogic, most servlet and ejb containers do not provide simple and clear instructions for tracing issues. With websphere, you have to buy an expensive license of WASD and even then debugging an issue

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Tomasz Nowak
Peter Lin [EMAIL PROTECTED] wrote: another potential solution is to try a different JVM like Bea's JRockit. JRockit provides some built in profiling capabilities, so that is another way to get some profile data quickly. I've read about jrockit 5.0 SP1. They say it has a new feature that can

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Tomasz Nowak
Tomasz Nowak [EMAIL PROTECTED] wrote: Hm. Here is my logging.properties: http://www.biochip.pl/logging.properties.txt The way new vhosts are added is pretty stupid for me but lets forget about the semantics now. I've added swallowOutput attibute to serveral context. How should it suppose

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Peter Lin
my advice is dive in and try to debug it, or hire someone with experience debugging webapps. there's no much others can do for you at this point, since it's debugging. good luck. peter On 3/1/06, Tomasz Nowak [EMAIL PROTECTED] wrote: Tomasz Nowak [EMAIL PROTECTED] wrote: Hm. Here is my

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Filip Hanik - Dev Lists
This could be completely unrelated, but let me throw something at you, maybe it will help btw. I prefer to run mod_proxy over mod_jk over http, much simpler, and in *our* load tests has scaled much better during stress. Response already committed This can happen if your servlet/jsp has code

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Tomasz Nowak
Peter Lin [EMAIL PROTECTED] wrote: my advice is dive in and try to debug it, or hire someone with experience debugging webapps. there's no much others can do for you at this point, since it's debugging. good luck. I've just meant configuring the logging system: So how can I: -

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Tomasz Nowak
Filip Hanik - Dev Lists [EMAIL PROTECTED] wrote: This could be completely unrelated, but let me throw something at you, maybe it will help btw. I prefer to run mod_proxy over mod_jk over http, much simpler, and in *our* load tests has scaled much better during stress. Response already

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-03-01 Thread Peter Lin
honestly, I really doubt logging will produce anything useful beyond the VM dump you already posted. My first hand experience, the logs won't help at this point. What needs to happen is to step through the code and see exactly what is happening. if you don't have access to a profiler, the other

Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Probably important: === - 2x Xeon, 3 GB mem - Linux 2.4.26 - Java 1.5.0_06-b05 - Apache Tomcat 5.5.15 - CATALINA_OPTS=\ -server \ -Djava.awt.headless=true \ -Dfile.encoding=UTF-8 \ -XX:MaxPermSize=256m \ -Xms1024m -Xmx1024m - Connector port=8009 protocol=AJP/1.3

RE: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Peter Crowther
From: news [mailto:[EMAIL PROTECTED] On Behalf Of Tomasz Nowak Probably important: === [lots of technical details about kit and container config] Very important == Your application. Have you profiled it for memory leaks that would cause a OOME? Have you tested it

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Mark Eggers
enableLookups=true redirectPort=8443 / Why enableLookups=true ??? Also, Cocoon can be very resource-intensive depending on which blocks you've built. JSP under Cocoon 2.1.7 and JDK 1.5 is also problematic, see for example: http://issues.apache.org/jira/browse/COCOON-1457 I have some

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Mark Eggers [EMAIL PROTECTED] wrote: enableLookups=true redirectPort=8443 / Why enableLookups=true ??? I use request.getRemoteHost in couple of places. Also, Cocoon can be very resource-intensive depending on which blocks you've built. Almost always only: - databases-block -

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Peter Crowther [EMAIL PROTECTED] wrote: Have you tested it for deadlocks that would cause hangs? [...] Profile your application. What's eating memory? Get a thread dump when your application hangs. Is anything deadlocked? What? Where? How?

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Mark Eggers
I'm confused - but wouldn't an enableLookups on the AJP connector return the hostname of the remote Apache server sending the request (and not the remote web client)? I don't know since I'm at work, and I always disable enableLookups since it's a potential performance hit. /mde/ --- Tomasz

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Leon Rosenberg
[...skiped...] I've been crond-restarting Tomcat 4.1 on Java 1.4 every night for more than a year, and now I have to restart Tomcat 5.5.15 on Java 1.5 AND apache httpd also almost every single day. From my -user point of view, Tomcat doesn't change in a good direction. In fact in it is

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Leon Rosenberg
On 2/28/06, Tomasz Nowak [EMAIL PROTECTED] wrote: Peter Crowther [EMAIL PROTECTED] wrote: Have you tested it for deadlocks that would cause hangs? [...] Profile your application. What's eating memory? Get a thread dump when your application hangs. Is anything deadlocked? What? Where?

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Mladen Turk
Mark Eggers wrote: I'm confused - but wouldn't an enableLookups on the AJP connector return the hostname of the remote Apache server sending the request (and not the remote web client)? No. JkOptions +ForwardLocalAddress Regards, Mladen.

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Tim Lucia [EMAIL PROTECTED] wrote: Send Tomcat a QUIT (11) signal on Unix, or control/break (run it interactively) on Windows. This will cause the VM to dump all thread stacks. If you don't like Tomcat, you can always shell out some big bugs for a commercial application server. For some

RE: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tim Lucia
improvements! Tim -Original Message- From: news [mailto:[EMAIL PROTECTED] On Behalf Of Tomasz Nowak Sent: Tuesday, February 28, 2006 4:36 PM To: users@tomcat.apache.org Subject: Re: Sad: Tomcat 5.5.x crashes almost every single day. Tim Lucia [EMAIL PROTECTED] wrote: Send Tomcat a QUIT

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Remy Maucherat
On 2/28/06, Tomasz Nowak [EMAIL PROTECTED] wrote: Your right, the tone of my postings is inproper. I've been using 'free' software for almost 10 years now and I pretty well get the rules. My only excuse is the level of my frustration, based on recent Tomcat use. For now, the only

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Remy Maucherat
On 2/28/06, Peter Lin [EMAIL PROTECTED] wrote: honestly, besides Weblogic, most servlet and ejb containers do not provide simple and clear instructions for tracing issues. With websphere, you have to buy an expensive license of WASD and even then debugging an issue won't be better in my

RE: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tim Lucia
with Jprofiler as well. -Original Message- From: Remy Maucherat [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 28, 2006 5:47 PM To: Tomcat Users List Subject: Re: Sad: Tomcat 5.5.x crashes almost every single day. On 2/28/06, Peter Lin [EMAIL PROTECTED] wrote: honestly, besides Weblogic, most

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Stephen Caine
Tomcat Users: We also had the dreaded, 'out of memory' errors and periodic crashes several times every day. Currently, we are stable. With help from Chuck, we were able to narrow down our problem to that of insufficient permanent generation memory. The point being that figuring out

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Nathan Smith
Hi Tomasz, you mentioned below that you have a MaxPermSize of 256m. I would suggest increasing this value to a much larger amount and even increasing your -Xms and -Xmx values. We had a problem with our application in a live environment where it would regularly get errors like the

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Nathan Smith
luck with Jprofiler as well. -Original Message- From: Remy Maucherat [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 28, 2006 5:47 PM To: Tomcat Users List Subject: Re: Sad: Tomcat 5.5.x crashes almost every single day. On 2/28/06, Peter Lin [EMAIL PROTECTED] wrote: honestly

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Remy Maucherat [EMAIL PROTECTED] wrote: In case you haven't noticed, it is extremely hard to do, because webapps have their own logging mechanism most of the time. You mention the logger element of 4.x, but it didn't actually do anything (it did put the internal logging for the specified

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Nathan Smith [EMAIL PROTECTED] wrote: Hi Tomasz, you mentioned below that you have a MaxPermSize of 256m. I have not seen OOME: PermGen errors on that machine but after having some of them on other machine with aditonal 10 webapps deployed I've increased it from default 64m to 256m to be

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Leon Rosenberg [EMAIL PROTECTED] wrote: We had some serious problems with tomcat, which are solved now, however, we are on 5.0.25 (with session patch) and not on 5.5.x yet. However, when tomcat dies, what your cpu usage is looking like? 100%? I've noticed that when tomcat hangs - load avarage

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Tim Lucia [EMAIL PROTECTED] wrote: #2 - Do any commercial application server vendors offer you assistance on how to find bugs in application code? You're much more likely to find these answers using google, or reading wikis or from community support. There are hundreds of such discussions

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Leon Rosenberg
you should seriously consider upgrading to 2.6.x linux. 2.4.s thread support simply sucks! If your tomcat spawns more than allowed threads it will get an outofmemoryerror and simply die... And I don't mean your 200 tomcat httpworker threads only, but all threads in the jvm, including database

RE: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tim Lucia
I use JProfiler with Tomcat all the time. -Original Message- From: news [mailto:[EMAIL PROTECTED] On Behalf Of Tomasz Nowak Sent: Tuesday, February 28, 2006 6:37 PM To: users@tomcat.apache.org Subject: Re: Sad: Tomcat 5.5.x crashes almost every single day. Remy Maucherat [EMAIL PROTECTED

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Peter Lin
. -Original Message- From: news [mailto:[EMAIL PROTECTED] On Behalf Of Tomasz Nowak Sent: Tuesday, February 28, 2006 6:37 PM To: users@tomcat.apache.org Subject: Re: Sad: Tomcat 5.5.x crashes almost every single day. Remy Maucherat [EMAIL PROTECTED] wrote: In case you haven't noticed

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Peter Lin
On 2/28/06, Tomasz Nowak [EMAIL PROTECTED] wrote: I've not noticed more then 400-500 java threads, but I'll monitor that. I've set different ulimits form differnt users, BUT user that runs java/tomcat has no limits but one: -s 2048 (stacksize). AFAIK that one was recommended by

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Zorro3692
Don't count on it!! In a message dated 2/28/2006 7:26:33 P.M. Eastern Standard Time, [EMAIL PROTECTED] writes: it seems that using a rude tone is a good way to get many answers, your question got 26 answers, mine, with a similar problem, only 1 (and a not helpful one).

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Leon Rosenberg [EMAIL PROTECTED] wrote: kill -SIGQUIT (i think its kill -11) your tomcat-java process Ok, it has happened again ! Simply: Mar 1, 2006 2:28:34 AM org.apache.catalina.connector.CoyoteAdapter service SEVERE: An exception or error occurred in the container during the request

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Peter Lin [EMAIL PROTECTED] wrote: look at the dump, it looks like the permgen ran out of space PSPermGen total 50304K, used 50170K [0x445f, 0x4771, 0x545f) object space 50304K, 99% used [0x445f,0x476ee878,0x4771) try increasing your permGen to 128Mb. Another

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Peter Lin
On 2/28/06, Tomasz Nowak [EMAIL PROTECTED] wrote: Peter Lin [EMAIL PROTECTED] wrote: look at the dump, it looks like the permgen ran out of space PSPermGen total 50304K, used 50170K [0x445f, 0x4771, 0x545f) object space 50304K, 99% used

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Nathan Smith
Hi Tomasz, this is the exact point I was making in my previous email. The error message that was returned was to do with the Java Heap Space. It may be referring to the entire heap that is allocated to the JVM, who knows. The problem can be fixed by increasing your -XX:MaxPermSize value to

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Wade Chandler
--- Tomasz Nowak [EMAIL PROTECTED] wrote: Peter Lin [EMAIL PROTECTED] wrote: look at the dump, it looks like the permgen ran out of space PSPermGen total 50304K, used 50170K [0x445f, 0x4771, 0x545f) object space 50304K, 99% used

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Peter Lin [EMAIL PROTECTED] wrote: It's not at all clear why -XX:MaxPermSize=256m isn't working and the perm size is still at the default. If the perm gen was set correctly, wouldn't the dump show something other than 99%? You're right, it shows PSPermGen total 50304K not the 256m

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Nathan Smith [EMAIL PROTECTED] wrote: The problem can be fixed by increasing your -XX:MaxPermSize value to more than 256. I've increased it to 512m now. I'll see. Download and use a program called visualgc available here http://java.sun.com/performance/jvmstat. You can have it running

Re: Sad: Tomcat 5.5.x crashes almost every single day.

2006-02-28 Thread Tomasz Nowak
Wade Chandler [EMAIL PROTECTED] wrote: Sometimes changing the ratio to not have enough new space and then creating too many objects too fast can make your application eventually throw OOMEs because it can't move the objects into older spaces quick enough depending on what you application is