Rainer thanx for the input, comments below. > > On 16.04.2009 01:49, Anthony J. Biacco wrote: > > A month or so ago I posted that I was having problems with mod_jk > > (1.2.27) getting a pong response back from tomcat (6.0.18) in > responses > > to a ping. Apache is 2.2.11 with worker mpm. > > > > I have a little more information now and am hoping with help I can > solve > > the problem so I can keep the ping timeout low and get rid of the > error. > > Error does not show up when ping timeout is >= 25000. > > It sounds like you can easily reproduce on a test system? > If yes, despite what I'll write below, increasing the log level to > trace > presents even more info for us developers, so in case we do another > test, please adjust the log level. But debug is fine for now. >
Yes, this is a test system, so the requests I'm sending are the only ones going into it. > > > worker.template.socket_timeout=10 > > I'm not very much in favor of the socket_timeout, but well, if you > think > you need it. Just for the sake of completeness, please check, whether > having no socket_timeout changes anything about your problem. > Confirmed same errors with socket_timeout commented out > > (*) We failed to read the Cpoong response. Note that we are still in > the > same second, so the wait time did not expire. We must have had some > other failure. > > My expectation: Tomcat closed the connection due to connectionTimeout. > mod_jk didn't check the connection_pool_timeout for this connection, > because our pool is already small enough (13 conns). > > Try setting connection_pool_minsize to "0" and check, whether the > observation goes away. Even with minsize "0", it can happen rarely, > because Tomcat closes the connection at the moment the > connectionTimeout > is reached, whereas mod_jk needs to run the internal maintenance (once > a > minute) to check the idle connection timeouts. > With connection_pool_minsize=0 I can confirm the different event from mod_jk after the tomcat connectionTimeout. This is for a request finished at 16:42:12. [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] wc_maintain::jk_worker.c (339): Maintaining worker app-01 [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] ajp_reset_endpoint::jk_ajp_common.c (743): (app-01) resetting endpoint with sd = 52 (socket shutdown) [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] jk_shutdown_socket::jk_connect.c (681): About to shutdown socket 52 [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] jk_shutdown_socket::jk_connect.c (732): Shutdown socket 52 and read 0 lingering bytes [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] ajp_maintain::jk_ajp_common.c (3074): cleaning pool slot=0 elapsed 116 in 0 [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] ajp_maintain::jk_ajp_common.c (3081): reached pool min size 0 from 25 cache slots [Thu Apr 16 16:44:08 2009] [26891:1114327360] [debug] ajp_maintain::jk_ajp_common.c (3121): recycled 1 sockets in 0 seconds from 25 pool slots > > *But*: I don't fully understand, why the problem goes away by switching > to a ping timeout of 25000 milliseconds. Maybe the 10 second > socket_timeout hides the thing in this case [wild guess]. There is also > a relation between the ping_timeout and which connections do get > checked > during maintenance (the connection_ping_interval). So a longer > ping_timeout reduces also the frequency of ping checks, so it should > also reduce the frequency of the info messages. Not sure whether the > combination of all results in not getting them any longer. > If I increase the ping timeout to 20000 the maintaining worker process still executes 2 minutes (almost to the second) after the tomcat request finished. No error. The 3rd minute's process, same. It isn't until 4 minutes after the tomcat request finished that I get the cpong error. If I set connection_ping_interval to something like 10 (with a 10000 ping_timeout), the cpong error shows up 1 minute after the tomcat request finished instead of 2. > Go with connection_pool_minsize=0 and ignore very occasional info level > messages. > Will this affect my performance since it won't keep any connections in the pool and have to recreate one on every request? As long as the ping without pong isn't going to break anything (like you said, it's info, not error (just sounds like a not-good thing)), i'm fine with it. > > I send a request to tomcat via apache/mod_jk. Request's mount is > mapped > > to worker, not loadbalancer. > > After successful request, Tomcat manager status shows the socket in > > stage 'R' and keeped alive socket count as 1. > > After 60 seconds, Tomcat manager status shows keeped alive socket > count > > as 0, socket still in stage 'R' > > Above error happens 2 minutes after a request is finished, I'm > assuming > > because the keep alive socket is not there anymore. > > You could have a look at "netstat -an | grep 8009" on the Apache server > (or whatever your AJP port is) and see, how the status of the > connections changes, e.g. from ESTABLISHED to CLOSE_WAIT (remote side > has closed the connection) and TIME_WAIT (local side has also closed > the > connection). After a tomcat request I have: tcp 0 0 10.10.10.18:13597 10.10.10.16:8009 ESTABLISHED After the tomcat connectiontimeout I have: tcp 1 0 10.10.10.18:13597 10.10.10.16:8009 CLOSE_WAIT After the cpong error I have nothing/no connection Because of the connectionTimeout in tomcat and the cpong failure, it seems like I'm defeating the purpose of a connection pool anyway, no? Since the tomcat socket close is minutes later causing the mod_jk socket close. > > We saw that. Since Cping/Cpong worked, we do not even have an info > message, only debug messages. > > > At this point tomcat manager status shows the socket in stage 'P' and > > keeped alive socket count is 0. > > Above mod_jk log repeats every 2 minutes. Tomcat manager status > doesn't > > change. Is it me or is this a better situation to be in? I'm not sure about the 'P' stage though, if it's suppose to be that way, or if it should be back at 'R' at this point. Much thanx, -Tony --------------------------- Manager, IT Operations Format Dynamics, Inc. 303-573-1800x27 abia...@formatdynamics.com http://www.formatdynamics.com --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
