-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Isaac,
On 3/11/14, 6:56 PM, Isaac Gonzalez wrote: > > > -----Original Message----- From: Christopher Schultz > [mailto:ch...@christopherschultz.net] Sent: Friday, March 07, 2014 > 8:18 AM To: Tomcat Users List Subject: Re: tomcat 6 refuses mod_jk > connections after server runs for a couple of days > > -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 > > > > On 3/6/14, 7:39 AM, Daniel Mikusa wrote: >> On Mar 5, 2014, at 4:51 PM, Isaac Gonzalez >> <igonza...@autoreturn.com> wrote: >> >>> >>> >>> -----Original Message----- From: Daniel Mikusa >>> [mailto:dmik...@gopivotal.com] Sent: Tuesday, March 04, 2014 >>> 12:42 PM To: Tomcat Users List Subject: Re: tomcat 6 refuses >>> mod_jk connections after server runs for a couple of days >>> >>> On Mar 4, 2014, at 1:55 PM, Isaac Gonzalez >>> <igonza...@autoreturn.com> wrote: >>> >>>> Dan, >>>> >>>> ________________________________________ From: Daniel Mikusa >>>> [dmik...@gopivotal.com] Sent: Tuesday, March 04, 2014 6:20 >>>> AM To: Tomcat Users List Subject: Re: tomcat 6 refuses mod_jk >>>> connections after server runs for a couple of days >>>> >>>> On Mar 4, 2014, at 6:32 AM, Rainer Jung >>>> <rainer.j...@kippdata.de> wrote: >>>> >>>>> On 27.02.2014 23:06, Isaac Gonzalez wrote: >>>>>> Hi Christopher(and Konstantin), attached is a couple of >>>>>> thread dumps of when we experienced the issue again >>>>>> today. I also noticed we get this message right before >>>>>> the problem occurs: Feb 27, 2014 12:47:15 PM >>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable >>>>>> >>>>>> run SEVERE: Caught exception (java.lang.OutOfMemoryError: >>>>>> unable to create new native thread) executing >>>>>> org.apache.jk.common.ChannelSocket$SocketAcceptor@177ddea, >>>>>> >>>>>> terminating thread >>>>> >>>>> Is it a 32Bit system? You have 2GB of heap plus Perm plus >>>>> native memory needed by the process plus thread stacks. Not >>>>> unlikely, that you ran out of memory address space for a 32 >>>>> bit process. >>>>> >>>>> The only fixes would then be: >>>>> >>>>> - switch to a 64 bit system >>>>> >>>>> - reduce heap if the app can work with less >>>>> >>>>> - improve performance or eliminate bottlenecks so that the >>>>> app works with less threads >>>>> >>>>> - limit you connector thread pool size. That will still >>>>> mean that if requests begin to queue because of performance >>>>> problems, the web server can't create additional >>>>> connections, but you won't get in an irregular situation as >>>>> you experience now. In that case you would need to >>>>> configure a low idle timeout for the connections on the JK >>>>> and TC side. >>>> >>>> It may also be possible to lower the thread stack size with >>>> the -Xss option. >>>> >>>> Ok so we are 64 bit Linux with 1024k in the 64-bit >>>> VM....would lowering it to 64k be a bit too low? What sort of >>>> repercussions would we run into? Very helpful information by >>>> the way. >>> >>> It depends on your apps, so you'll need to test and see. If >>> you go too low, you'll get StackOverflow exceptions. If you >>> see those, just gradually increase until they go away. >>> >>> Dan >>> >>> >>>> >>>> -Isaac >>>> >>>> >>>> http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#thread >>>> >>>> s_ >>>> >>>> > oom >>>> >>>> Might buy you some room for a few additional threads. >>>> >>>> Dan >>>> >>>>> >>>>> Regards, >>>>> >>>>> Rainer >>>>> >>>>> >>>>> ------------------------------------------------------------------- >>>>> >>>>> - -- >>>>> >>>>> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>>> For additional commands, e-mail: >>>>> users-h...@tomcat.apache.org >>>>> >>>> >>>> >>>> -------------------------------------------------------------------- >>>> >>>> - - >>>> >>>> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: >>>> users-h...@tomcat.apache.org >>>> -------------------------------------------------------------------- >>>> >>>> - - >>>> >>>> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: >>>> users-h...@tomcat.apache.org >>>> >>> >>> >>> --------------------------------------------------------------------- >>> >>> > >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >>> Ok so the problem just happened again just now. Dan, Can you >>> elaborate on how to configure limiting the connector thread >>> pool size. I am also going to lower the thread stack size as >>> you recommended. It seems like this problem creeps up when we >>> have a hiccup in connectivity at our data center. Perhaps I >>> also need to lower the idle timeout some more between tomcat >>> and mod_jk. They are also between a firewall by the way, so I >>> can configure a timeout between the two there as well. We >>> aren't experiencing too many idle disconnects there. >> >> See maxConnections / maxThreads on the Connector tag. >> >> http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Standard_Impl >> >> ementation >> >> or Executor if you’re using an executor. >> >> http://tomcat.apache.org/tomcat-7.0-doc/config/executor.html > >> ... and you definitely *should* be using a manually-configured >> Executor. > >> - -chris > > Chris, why should I be using a connector since we are only having > users use the single 8009 AJP connection on each tomcat instance? I > am the only one that uses the 8080 connector for troubleshooting > and monitoring purposes. Is it mainly to help recycle unused > threads? Yes, you can better-manage all the thread-based resources using an <Executor> rather than having one (or two) created for you automatically by the connector. - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTIKA/AAoJEBzwKT+lPKRYSHgQALvACFo5ayaSkX/d8EuXwBUl wMd9L0XBUOpc+gnM+nERqa29/uTmZisWCifo0VlI7p6rbmFQOAZSl7eqFHpT1eZF DZgeahpLCRIABTmXwKfzExXue7OEu+Ry4JPhmPM/6aO0vZK+pFQGyl/v6V8b+Hfo kGsaYlNFxTiuYqyOWLKoWMbUNjhwAVlz/hqZkO+QpuVf1DtrI9Srnc2OBcC6pNSf xiP/aybrMYRH2rZWkU1qafAqo3lVBus3T32HBGwPJ1MSvHDoMNluBMMrzeSxECFl Pv0tgdAzStLCE+UPwyeSMkZlotC/fIz+l2e0V0fCxYBZZuWIWMOkyaw5qUGnrI/f Ur7PQM7El77yHKn+0MrTC7kXlRISdO7GQRDn0lTUqLDpLjTGF/qdS4gJLYR+fqHS n5rLXnuOzEThCpFKYV8wBm9ePtGqk65jl59828yQ9x3QcchYU9byzIfoGI8KQf/O SG8OdVV1G0z1axnnukhnqUl65teA6ST9h1qrrli/CP8NRO5PNQ4ftHwVhWkxVRPH wfis6YSCnqNPBG8JwsHYWaRpNqeDnHRXaGdKOGd/GEhZrxNwd3UNzA81NDQI1yoE hZfvaOi0xUFOQeX+PVhhPvFeGD8HqNlh02GTPKkfBZSeG6mENM/cEtFbG69mfmoG IskslmrAlsqtm3igHHoX =EWbM -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
