-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Konstantin,
On 2/27/14, 5:40 PM, Konstantin Kolinko wrote: > 2014-02-28 2:06 GMT+04:00 Isaac Gonzalez > <igonza...@autoreturn.com>: >> Hi Christopher(and Konstantin), attached is a couple of thread >> dumps of when we experienced the issue again today. I also >> noticed we get this message right before the problem occurs: Feb >> 27, 2014 12:47:15 PM >> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run >> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to >> create new native thread) executing >> org.apache.jk.common.ChannelSocket$SocketAcceptor@177ddea, >> terminating thread > > That explains why a connection cannot be accepted. > > I wonder are you hitting an "ulimit" limit, or there is just not > enough free memory to allocate stack area for a new thread (which > size is set by -Xss option to java executable). > > Your thread dumps contain 149 threads each. While it does explain why one (Tomcat) server would become unresponsive, it doesn't really explain why the entire cluster would become unresponsive. Issac, are you using sticky-sessions or anything like that, or does your load-balancing mod_jk configuration choose arbitrarily between a backend server? You initially gave an abridged configuration, so I can't tell. >> After the "split", did both Tomcats appear to lock-up >> simultaneously, or did only one of them have a problem and the >> other one stayed up? > > Isaac: They all appear to lock-up simultaneously, if users try to > access that JK mount point. > > [...] > > Isaac: I am not load-balancing the tomcat servers...I only have > one...I do "load balance" the apache front end servers via dns > round-robin.... Oh, you only have a single back-end server? Well, then that why they "all" go down at once, so you seem to have found your problem: the server itself is going down because you don't have enough resources to keep it up. - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTEOWeAAoJEBzwKT+lPKRYUxYQAItxQZTOWjubWA32fDwGhcGy Y4cuBDU7OZiw/I4oQTCEY0Uk62eKf6g0dXQL4VqM8j4jz2VLhP10uPDA3c/+CBhU kpVJg2ifLD0YJmUqnxuTHYIdwwvPrPa/PNCAWYJCcDXE1DVEz8HLAoYlQY5oJ5Cf P+wtYTgWqJyzddtB2sjB+YQcVj+83aWkfKuipednSdm0utCPP5PQzPiF7agoP9qt vDB0preG0GFQXTShYqMRKeEt3hu+BdLXugp7kJA5KDMEcSbWyPzefxWl0CtKhcJB d/ntEtoYbR0gWGO4Qajio6NVmw9TWzBf4spbg8scBz8ijE314VNsw6mdT9F55TZZ 43iYSnDAK1dNfs7guqAAk7z5Gf+fChy28zFmOm0lSzs1/o5HHvJFqKse94hzjJW6 R4uCUVktbvoJPfot6zoG3ofsYm+PVcibPOj4Xh0m7nBPKvqTZ5BVyeLBRR/E+KRy O4jJ0DshRnhy3qL9l5uO6h7miIb+GMwjpc3A6lbbITHVDKspaq8kll+m6sAn1ppV Z8PnNysSMTGHY6azjMisZlp4b/i9r+Nc+HabtbjRrt1StfuWrHTeIwQ46n7XcMrr biATXUpo94kM2eGJecP0jtyBrgYwkaz22NtbW3i2l47XQKa/dhhP6IzlgzK8Fmki eKiBf5+iNyhc9dB3adQ2 =Uchz -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
