Re: mod_proxy_ajp TIME_WAIT
That would be excellent ! Is the only change - as far as mod_proxy_ajp is concerned the one below or is that a work around for 2.2.6 ? Thanks D On Tue, 2007-11-20 at 14:39 -0500, Jim Jagielski wrote: I'm hoping to get it out the top of December :) On Nov 20, 2007, at 3:57 AM, David Cassidy wrote: Hi Jim !!! This is fantastic news ! When is 2.2.7 going to be released ? :) Many many thanks David On Mon, 2007-11-19 at 15:27 -0500, Jim Jagielski wrote: 2.2.6 has a nasty bug were AJP connections are being closed when they shouldn't. 2.2.7 will fix that. In the meantime, trying building httpd with USE_ALTERNATE_IS_CONNECTED defined as 0 (proxy_util.c). On Nov 19, 2007, at 9:07 AM, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
The below is the workaround for 2.2.6... 2.2.7 contains the actual fix that negates the need for the workaround :) On Nov 21, 2007, at 4:34 AM, David Cassidy wrote: That would be excellent ! Is the only change - as far as mod_proxy_ajp is concerned the one below or is that a work around for 2.2.6 ? Thanks D On Tue, 2007-11-20 at 14:39 -0500, Jim Jagielski wrote: I'm hoping to get it out the top of December :) On Nov 20, 2007, at 3:57 AM, David Cassidy wrote: Hi Jim !!! This is fantastic news ! When is 2.2.7 going to be released ? :) Many many thanks David On Mon, 2007-11-19 at 15:27 -0500, Jim Jagielski wrote: 2.2.6 has a nasty bug were AJP connections are being closed when they shouldn't. 2.2.7 will fix that. In the meantime, trying building httpd with USE_ALTERNATE_IS_CONNECTED defined as 0 (proxy_util.c). On Nov 19, 2007, at 9:07 AM, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David -- -- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --- -- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
Hi Jim !!! This is fantastic news ! When is 2.2.7 going to be released ? :) Many many thanks David On Mon, 2007-11-19 at 15:27 -0500, Jim Jagielski wrote: 2.2.6 has a nasty bug were AJP connections are being closed when they shouldn't. 2.2.7 will fix that. In the meantime, trying building httpd with USE_ALTERNATE_IS_CONNECTED defined as 0 (proxy_util.c). On Nov 19, 2007, at 9:07 AM, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
I'm hoping to get it out the top of December :) On Nov 20, 2007, at 3:57 AM, David Cassidy wrote: Hi Jim !!! This is fantastic news ! When is 2.2.7 going to be released ? :) Many many thanks David On Mon, 2007-11-19 at 15:27 -0500, Jim Jagielski wrote: 2.2.6 has a nasty bug were AJP connections are being closed when they shouldn't. 2.2.7 will fix that. In the meantime, trying building httpd with USE_ALTERNATE_IS_CONNECTED defined as 0 (proxy_util.c). On Nov 19, 2007, at 9:07 AM, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
Is this worker or prefork MPM? On Nov 15, 2007, at 4:03 AM, David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
Hi ! This is using worker rather than prefork - apache 2.2.6 as comes with fedora 7. I've changed /usr/sbin/httpd to be /usr/sbin/httpd.worker. If I make a 1000 requests with ab with keep alive to apache - eg ab -k -n 1000 url then I get alot of connections from apache to tomcat that are in TIME_WAIT - eg tcp0 0 127.0.0.1:46284 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46374 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46306 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46396 127.0.0.1:8009 TIME_WAIT the counts look like 1 LISTEN 2 ESTABLISHED 999 TIME_WAIT ie 999 connections in TIME_WAIT Is there anyway to tell Apache to keep the connections alive for a more connections ? Apache config is : ProxyPass / balancer://myclusterclear/ stickysession=JSESSIONID| jsessionid Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy Tomcat is Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ If I increase the concurrency to 10 from 1 and re-run the same 1,000 tests I get 1 LISTEN 20 ESTABLISHED 990 TIME_WAIT Slightly better but i'd have liked to see there be more ESTABLISHED connections. Running with 30 concurrency i get 1 LISTEN 60 ESTABLISHED 970 TIME_WAIT With 50 concurrency I get 1 LISTEN 28 CLOSE_WAIT 28 FIN_WAIT2 66 ESTABLISHED 939 TIME_WAIT In each case the netstat is performed immediately after the test has finished and before each test run the netstat only has the 1 listen socket for 8009. In each case the netstat is from apache to tomcat So any ideas why tomcat would close the connections ? Many thanks David On Mon, 2007-11-19 at 15:07 +0100, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
David Cassidy wrote: Hi ! This is using worker rather than prefork - apache 2.2.6 as comes with fedora 7. I've changed /usr/sbin/httpd to be /usr/sbin/httpd.worker. If I make a 1000 requests with ab with keep alive to apache - eg ab -k -n 1000 url then I get alot of connections from apache to tomcat that are in TIME_WAIT - eg tcp0 0 127.0.0.1:46284 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46374 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46306 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46396 127.0.0.1:8009 TIME_WAIT the counts look like 1 LISTEN 2 ESTABLISHED 999 TIME_WAIT ie 999 connections in TIME_WAIT Is there anyway to tell Apache to keep the connections alive for a more connections ? First of all, for me this looks like really Apache httpd is closing the connections (you ask later, if Tomcat can be told to keep the connection open, but this doesn't help, because httpd closes it). Does setting a ttl help (see http://httpd.apache.org/docs/2.2/mod/mod_proxy.html)? Regards, Rainer Apache config is : ProxyPass / balancer://myclusterclear/ stickysession=JSESSIONID| jsessionid Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy Tomcat is Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ If I increase the concurrency to 10 from 1 and re-run the same 1,000 tests I get 1 LISTEN 20 ESTABLISHED 990 TIME_WAIT Slightly better but i'd have liked to see there be more ESTABLISHED connections. Running with 30 concurrency i get 1 LISTEN 60 ESTABLISHED 970 TIME_WAIT With 50 concurrency I get 1 LISTEN 28 CLOSE_WAIT 28 FIN_WAIT2 66 ESTABLISHED 939 TIME_WAIT In each case the netstat is performed immediately after the test has finished and before each test run the netstat only has the 1 listen socket for 8009. In each case the netstat is from apache to tomcat So any ideas why tomcat would close the connections ? Many thanks David On Mon, 2007-11-19 at 15:07 +0100, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_ajp TIME_WAIT
Hi Rainer, I've set the ttl to 120 re-run the last test with 30 concurrent connections 1 LISTEN 25 CLOSE_WAIT 26 FIN_WAIT2 104 ESTABLISHED 924 TIME_WAIT Not made too much difference. But as the test is only taking 20 secs max none of the connections should have reached the ttl unless ttl is not seconds. Do you think I should be asking this on the httpd dev mailing list as its an apache prob and not tomcat ? Thanks David On Mon, 2007-11-19 at 15:47 +0100, Rainer Jung wrote: David Cassidy wrote: Hi ! This is using worker rather than prefork - apache 2.2.6 as comes with fedora 7. I've changed /usr/sbin/httpd to be /usr/sbin/httpd.worker. If I make a 1000 requests with ab with keep alive to apache - eg ab -k -n 1000 url then I get alot of connections from apache to tomcat that are in TIME_WAIT - eg tcp0 0 127.0.0.1:46284 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46374 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46306 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46396 127.0.0.1:8009 TIME_WAIT the counts look like 1 LISTEN 2 ESTABLISHED 999 TIME_WAIT ie 999 connections in TIME_WAIT Is there anyway to tell Apache to keep the connections alive for a more connections ? First of all, for me this looks like really Apache httpd is closing the connections (you ask later, if Tomcat can be told to keep the connection open, but this doesn't help, because httpd closes it). Does setting a ttl help (see http://httpd.apache.org/docs/2.2/mod/mod_proxy.html)? Regards, Rainer Apache config is : ProxyPass / balancer://myclusterclear/ stickysession=JSESSIONID| jsessionid Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy Tomcat is Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ If I increase the concurrency to 10 from 1 and re-run the same 1,000 tests I get 1 LISTEN 20 ESTABLISHED 990 TIME_WAIT Slightly better but i'd have liked to see there be more ESTABLISHED connections. Running with 30 concurrency i get 1 LISTEN 60 ESTABLISHED 970 TIME_WAIT With 50 concurrency I get 1 LISTEN 28 CLOSE_WAIT 28 FIN_WAIT2 66 ESTABLISHED 939 TIME_WAIT In each case the netstat is performed immediately after the test has finished and before each test run the netstat only has the 1 listen socket for 8009. In each case the netstat is from apache to tomcat So any ideas why tomcat would close the connections ? Many thanks David On Mon, 2007-11-19 at 15:07 +0100, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between
Re: mod_proxy_ajp TIME_WAIT
David Cassidy wrote: Hi Rainer, I've set the ttl to 120 re-run the last test with 30 concurrent connections 1 LISTEN 25 CLOSE_WAIT 26 FIN_WAIT2 104 ESTABLISHED 924 TIME_WAIT Not made too much difference. But as the test is only taking 20 secs max none of the connections should have reached the ttl unless ttl is not seconds. Do you think I should be asking this on the httpd dev mailing list as its an apache prob and not tomcat ? Yes, maybe starting with the httpd user list, before going to dev. Thanks David On Mon, 2007-11-19 at 15:47 +0100, Rainer Jung wrote: David Cassidy wrote: Hi ! This is using worker rather than prefork - apache 2.2.6 as comes with fedora 7. I've changed /usr/sbin/httpd to be /usr/sbin/httpd.worker. If I make a 1000 requests with ab with keep alive to apache - eg ab -k -n 1000 url then I get alot of connections from apache to tomcat that are in TIME_WAIT - eg tcp0 0 127.0.0.1:46284 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46374 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46306 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46396 127.0.0.1:8009 TIME_WAIT the counts look like 1 LISTEN 2 ESTABLISHED 999 TIME_WAIT ie 999 connections in TIME_WAIT Is there anyway to tell Apache to keep the connections alive for a more connections ? First of all, for me this looks like really Apache httpd is closing the connections (you ask later, if Tomcat can be told to keep the connection open, but this doesn't help, because httpd closes it). Does setting a ttl help (see http://httpd.apache.org/docs/2.2/mod/mod_proxy.html)? Regards, Rainer Apache config is : ProxyPass / balancer://myclusterclear/ stickysession=JSESSIONID| jsessionid Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy Tomcat is Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ If I increase the concurrency to 10 from 1 and re-run the same 1,000 tests I get 1 LISTEN 20 ESTABLISHED 990 TIME_WAIT Slightly better but i'd have liked to see there be more ESTABLISHED connections. Running with 30 concurrency i get 1 LISTEN 60 ESTABLISHED 970 TIME_WAIT With 50 concurrency I get 1 LISTEN 28 CLOSE_WAIT 28 FIN_WAIT2 66 ESTABLISHED 939 TIME_WAIT In each case the netstat is performed immediately after the test has finished and before each test run the netstat only has the 1 listen socket for 8009. In each case the netstat is from apache to tomcat So any ideas why tomcat would close the connections ? Many thanks David On Mon, 2007-11-19 at 15:07 +0100, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks
Re: mod_proxy_ajp TIME_WAIT
OK I'll give that a go ! Thanks Rainer for your help D On Mon, 2007-11-19 at 16:09 +0100, Rainer Jung wrote: David Cassidy wrote: Hi Rainer, I've set the ttl to 120 re-run the last test with 30 concurrent connections 1 LISTEN 25 CLOSE_WAIT 26 FIN_WAIT2 104 ESTABLISHED 924 TIME_WAIT Not made too much difference. But as the test is only taking 20 secs max none of the connections should have reached the ttl unless ttl is not seconds. Do you think I should be asking this on the httpd dev mailing list as its an apache prob and not tomcat ? Yes, maybe starting with the httpd user list, before going to dev. Thanks David On Mon, 2007-11-19 at 15:47 +0100, Rainer Jung wrote: David Cassidy wrote: Hi ! This is using worker rather than prefork - apache 2.2.6 as comes with fedora 7. I've changed /usr/sbin/httpd to be /usr/sbin/httpd.worker. If I make a 1000 requests with ab with keep alive to apache - eg ab -k -n 1000 url then I get alot of connections from apache to tomcat that are in TIME_WAIT - eg tcp0 0 127.0.0.1:46284 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46374 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46306 127.0.0.1:8009 TIME_WAIT tcp0 0 127.0.0.1:46396 127.0.0.1:8009 TIME_WAIT the counts look like 1 LISTEN 2 ESTABLISHED 999 TIME_WAIT ie 999 connections in TIME_WAIT Is there anyway to tell Apache to keep the connections alive for a more connections ? First of all, for me this looks like really Apache httpd is closing the connections (you ask later, if Tomcat can be told to keep the connection open, but this doesn't help, because httpd closes it). Does setting a ttl help (see http://httpd.apache.org/docs/2.2/mod/mod_proxy.html)? Regards, Rainer Apache config is : ProxyPass / balancer://myclusterclear/ stickysession=JSESSIONID| jsessionid Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy Tomcat is Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ If I increase the concurrency to 10 from 1 and re-run the same 1,000 tests I get 1 LISTEN 20 ESTABLISHED 990 TIME_WAIT Slightly better but i'd have liked to see there be more ESTABLISHED connections. Running with 30 concurrency i get 1 LISTEN 60 ESTABLISHED 970 TIME_WAIT With 50 concurrency I get 1 LISTEN 28 CLOSE_WAIT 28 FIN_WAIT2 66 ESTABLISHED 939 TIME_WAIT In each case the netstat is performed immediately after the test has finished and before each test run the netstat only has the 1 listen socket for 8009. In each case the netstat is from apache to tomcat So any ideas why tomcat would close the connections ? Many thanks David On Mon, 2007-11-19 at 15:07 +0100, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false
Re: mod_proxy_ajp TIME_WAIT
2.2.6 has a nasty bug were AJP connections are being closed when they shouldn't. 2.2.7 will fix that. In the meantime, trying building httpd with USE_ALTERNATE_IS_CONNECTED defined as 0 (proxy_util.c). On Nov 19, 2007, at 9:07 AM, Rainer Jung wrote: Hi David, TIME_WAIT is a normal TCP state after a connection was successfully closed. Only one side of the connection goes into TIME_WAIT, namely the side that sent the first FIN. So since you've got httpd and Tomcat on the same server, you first need to find out, which side of the conection is in TIME_WAIT. In netstat, usually the left hand IP:PORT is the local side, and the right IP:PORT the remote side. In case the left pair of the TIME_WAIT line includes the port 8009, this would mean, that Tomcat closed the connection first, in case 8009 is on the right side, it would mean, that Apache httpd closed the connection first. Maybe you could show us some of the TIME_WAIT netstat lines. Both could be OK, so we could ask ourselves, if we expect such behaviour. In general AJP connections should be used persistently and only closed, if they have been idle for to long. Is the number of TIME_WAIT connections much larger, than the concurrency (-c) used with ab? Regards, Rainer David Cassidy wrote: Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
mod_proxy_ajp TIME_WAIT
Guys, I'm using mod_proxy in apache 2.2.6 with the ajp connector in tomcat. apache config - Proxy balancer://myclusterclear BalancerMember ajp://localhost:8009 route=server1 min=0 smax=1000 max=1000 keepalive=On /Proxy ProxyPass // balancer://myclusterclear/ stickysession=JSESSIONID|jsessionid Tomcat config (Using the native apr libs) Executor name=tomcatThreadPool namePrefix=catalina-exec- maxThreads=150 minSpareThreads=4/ Connector executor=tomcatThreadPool port=8009 secure=false protocol=AJP/1.3 enableLookups=false proxyPort=80 redirectPort=443 maxKeepAliveRequests=2000 tcpNoDelay=true keepAliveTimeout=1 connectionTimeout=60/ After running a few hits with ab to give it some load there are a very large number of connections between apache and tomcat in a TIME_WAIT status. Is this a common happening ? Is there something that can be configured to prevent this from appearing ? Thanks David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
