Re: Reuse mod ajp proxy connections
On Wed, Oct 19, 2011 at 10:27 AM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/18/2011 10:50 PM, Dimitar Georgievski wrote: SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 This setting should control the mod_http_proxy connections. I forgot to mention this Apache server does that. My reading suggests that it controls mod_proxy, which should be the same for all mod_proxy_*. The documentation for mod_proxy_ajp is woefully slim. Sure, you can learn all about the AJP protocol, just not how to configure the proxy in httpd.conf :( We have started series of tests to determine the effect of the following changes; 1. Upgrade Apache to 2.2.21 (the latest) 2. Upgrade Tomcat to 5.5.34 (the latest in the 5.x series) 3. Remove *SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1* It will take a while to complete these tests whch has to do with our test methodologies, which could be summarized as make one change and repeat the baseline test at least twice. I'll definitely take out these settings, because the http proxy connections should also be persistent. Okay. Did it work? Seems like a simple test to make. Regarding the upgrade of the Tomcat server we might not be able to do that. The application hosted by Tomcat has dependency on the Tomcat 5.5.x version which limits our options. I'll need to determine yet which version would be compliant with our application. If you have dependencies on Tomcat code, it's best to replace those with standards-compliant code. Maybe we could help with that kind of thing. As for 5.5 - 6.0 - 7.0 upgrades, I have recent done that exact thing and have had no problems whatsoever. Tomcat *should* be backward-compatible with previous releases, but there are some gotchas to be found when the TC code has become more spec-compliant or more cranky about certain bad configurations that it used to allow. As with any upgrade, always read the release notes / changelog carefully. The application that is deployed on Tomcat 5.x is called Sakai ( http://sakaiproject.org/). From what I know it has strict dependencies on the Tomcat version. We started reviewing the requirements for the migration to Tomcat 6.x but this will take a while, probably months to complete. I am also considering to install Apache Portable Runtime to improve connectors performance and determine its effects on the persistence of the proxy connections. What do you think? If you expect high concurrency, try using the NIO connector(s): they should scale better and have the advantage of being all Java, so you have less risk of crashing your JVM if something goes wrong (though the APR AJP connector /is/ pretty rock-solid). This is also in our test plans but this change has to wait on the result of the servers upgrade tests. Moving to NIO or APR seems a sensible thing to do. The Apache server does more than being a reverse proxy for this Tomcat server and couldn't be removed from the server configuration of this application. Thanks Dimitar - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6e3kYACgkQ9CaO5/Lv0PDtEgCeJTRF7GujmZefSjofidEDJOf2 wxUAnj/Sjt4fFXbs3cdCsnqZCQaz5SMl =LlVh -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/20/2011 11:37 AM, Dimitar Georgievski wrote: We have started series of tests to determine the effect of the following changes; 1. Upgrade Apache to 2.2.21 (the latest) 2. Upgrade Tomcat to 5.5.34 (the latest in the 5.x series) 3. Remove *SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1* It will take a while to complete these tests whch has to do with our test methodologies, which could be summarized as make one change and repeat the baseline test at least twice. I would just go ahead with #3 in a test environment. You should be able to see in about 5 minutes what's going on. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6ghvMACgkQ9CaO5/Lv0PCBAQCeOXpC96cV0ecvknZv9p74NLve hNIAn2IR1E2C5u+yyfozhyd29lJraWvg =CdzV -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
Dimitar Georgievski wrote: ... Mod proxy settings in Apache ProxyStatus On ProxyRequests Off ProxyTimeout 1800 ProxyPassMatch ^/(.*) ajp://localhost:8009/$1 SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 What happens if you just comment out the previous 2 lines ? Unless I totally misunderstand the httpd documentation and HTTP, the above 2 lines tell the Apache mod_proxy module to issue all requests to Tomcat using HTTP 1.0, which does not support persistent connections. No wonder then that each request causes a separate connection setup and tear-down. Next question : ProxyPassMatch ^/(.*) ajp://localhost:8009/$1 If you are forwarding *everything* to Tomcat anyway, then why do you have an Apache httpd in front ? why not just set up Tomcat to listen on port 80 and avoid the complication ? - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/18/2011 10:50 PM, Dimitar Georgievski wrote: SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 This setting should control the mod_http_proxy connections. I forgot to mention this Apache server does that. My reading suggests that it controls mod_proxy, which should be the same for all mod_proxy_*. The documentation for mod_proxy_ajp is woefully slim. Sure, you can learn all about the AJP protocol, just not how to configure the proxy in httpd.conf :( I'll definitely take out these settings, because the http proxy connections should also be persistent. Okay. Did it work? Seems like a simple test to make. Regarding the upgrade of the Tomcat server we might not be able to do that. The application hosted by Tomcat has dependency on the Tomcat 5.5.x version which limits our options. I'll need to determine yet which version would be compliant with our application. If you have dependencies on Tomcat code, it's best to replace those with standards-compliant code. Maybe we could help with that kind of thing. As for 5.5 - 6.0 - 7.0 upgrades, I have recent done that exact thing and have had no problems whatsoever. Tomcat *should* be backward-compatible with previous releases, but there are some gotchas to be found when the TC code has become more spec-compliant or more cranky about certain bad configurations that it used to allow. As with any upgrade, always read the release notes / changelog carefully. I am also considering to install Apache Portable Runtime to improve connectors performance and determine its effects on the persistence of the proxy connections. What do you think? If you expect high concurrency, try using the NIO connector(s): they should scale better and have the advantage of being all Java, so you have less risk of crashing your JVM if something goes wrong (though the APR AJP connector /is/ pretty rock-solid). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6e3kYACgkQ9CaO5/Lv0PDtEgCeJTRF7GujmZefSjofidEDJOf2 wxUAnj/Sjt4fFXbs3cdCsnqZCQaz5SMl =LlVh -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 André, On 10/19/2011 3:21 AM, André Warnier wrote: Next question : ProxyPassMatch ^/(.*) ajp://localhost:8009/$1 If you are forwarding *everything* to Tomcat anyway, then why do you have an Apache httpd in front ? why not just set up Tomcat to listen on port 80 and avoid the complication ? This could just be in a single VirtualHost, etc. But I agree: don't use https unless you really need it. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6e3oUACgkQ9CaO5/Lv0PCwsQCfYBsP3sVlYIs31DwmCDRoOMMZ YF8AmwQ34Y1yRtNf1VVKxLx+pARdl/nj =BOis -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/18/2011 2:15 PM, Dimitar Georgievski wrote: We have a Tomcat 5.5.25 server connected to Apache 2.2.9 over mod_ajp_proxy. Monitoring of the servers shows that AJP proxy connections are not reused but closed and reopened. Frequent closing of TCP connections leaves many connections in TIME_WAIT state and this is something we would prefer to avoid if possible. AJP is intended to maintain persistent connections. Something must be going wrong or your configuration must be forcing such behavior. My understanding of the *reuse* flag in AJP13_END_RESPONSE structure is that this flag is hard coded and cannot be controlled by configuration settings. I couldn't find any in Tomcat AJP connector configuration that could do that. Since Tomcat is sending the response, is there any way to control the reuse of AJP TCP connection through Tomcat configuration? I hope I don't have to resort to editing and recompilation of the mod_ajp_proxy module :-) Can you post your AJP Connector settings from conf/server.xml as well as your mod_proxy_ajp configuration as well? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6d1iEACgkQ9CaO5/Lv0PCnGgCcCmrD69wV7EAdj3qMbuMC3pZz jmQAn3X6C9buUoCS7SwCDHIR6HzJqGVY =HWpj -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
On 18/10/2011 20:40, Christopher Schultz wrote: Dimitar, On 10/18/2011 2:15 PM, Dimitar Georgievski wrote: We have a Tomcat 5.5.25 server connected to Apache 2.2.9 over mod_ajp_proxy. Monitoring of the servers shows that AJP proxy connections are not reused but closed and reopened. Frequent closing of TCP connections leaves many connections in TIME_WAIT state and this is something we would prefer to avoid if possible. You could update both Tomcat and HTTPD to more recent versions. p AJP is intended to maintain persistent connections. Something must be going wrong or your configuration must be forcing such behavior. My understanding of the *reuse* flag in AJP13_END_RESPONSE structure is that this flag is hard coded and cannot be controlled by configuration settings. I couldn't find any in Tomcat AJP connector configuration that could do that. Since Tomcat is sending the response, is there any way to control the reuse of AJP TCP connection through Tomcat configuration? I hope I don't have to resort to editing and recompilation of the mod_ajp_proxy module :-) Can you post your AJP Connector settings from conf/server.xml as well as your mod_proxy_ajp configuration as well? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org signature.asc Description: OpenPGP digital signature
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Pid, On 10/18/2011 6:17 PM, Pid wrote: On 18/10/2011 20:40, Christopher Schultz wrote: Dimitar, On 10/18/2011 2:15 PM, Dimitar Georgievski wrote: We have a Tomcat 5.5.25 server connected to Apache 2.2.9 over mod_ajp_proxy. Monitoring of the servers shows that AJP proxy connections are not reused but closed and reopened. Frequent closing of TCP connections leaves many connections in TIME_WAIT state and this is something we would prefer to avoid if possible. You could update both Tomcat and HTTPD to more recent versions. Nice catch: I didn't notice the old version of Apache httpd in particular. mod_proxy_ajp is relatively new to httpd, and so AJP performance and behavior can vary wildly from point-release to point-release. Dimitar, please upgrade if you can. Or, consider using mod_jk, whose code is much more mature. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6eGUcACgkQ9CaO5/Lv0PA3XQCgk87sg0De8m+udhXkVXyZLqpL DyQAn2eDYjQBq/q49idMBb+T8nck/qxI =6etX -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
Hi Chris,
Tomcat AJP Connector settings:
Connector port=${ajp.port} protocol=AJP/1.3 redirectPort=${ssl.port}
maxThreads=750 backlog=100
enableLookups=false
emptySessionPath=true connectionTimeout=80 URIEncoding=UTF-8 /
Apache MPM worker settings;
StartServers 3
ServerLimit 30
MaxClients 750
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
Mod proxy settings in Apache
ProxyStatus On
ProxyRequests Off
ProxyTimeout 1800
ProxyPassMatch ^/(.*) ajp://localhost:8009/$1
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
by monitoring Apache performance through stats provided by mod_status I've
noticed that the AJP TCP connection is always closed after Tomcat serves a
static resource like CSS, JavaScript, image or a JAR file
Thanks,
Dimitar
On Tue, Oct 18, 2011 at 3:40 PM, Christopher Schultz
ch...@christopherschultz.net wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Dimitar,
On 10/18/2011 2:15 PM, Dimitar Georgievski wrote:
We have a Tomcat 5.5.25 server connected to Apache 2.2.9 over
mod_ajp_proxy. Monitoring of the servers shows that AJP proxy
connections are not reused but closed and reopened. Frequent
closing of TCP connections leaves many connections in TIME_WAIT
state and this is something we would prefer to avoid if possible.
AJP is intended to maintain persistent connections. Something must be
going wrong or your configuration must be forcing such behavior.
My understanding of the *reuse* flag in AJP13_END_RESPONSE
structure is that this flag is hard coded and cannot be controlled
by configuration settings. I couldn't find any in Tomcat AJP
connector configuration that could do that.
Since Tomcat is sending the response, is there any way to control
the reuse of AJP TCP connection through Tomcat configuration? I
hope I don't have to resort to editing and recompilation of the
mod_ajp_proxy module :-)
Can you post your AJP Connector settings from conf/server.xml as
well as your mod_proxy_ajp configuration as well?
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6d1iEACgkQ9CaO5/Lv0PCnGgCcCmrD69wV7EAdj3qMbuMC3pZz
jmQAn3X6C9buUoCS7SwCDHIR6HzJqGVY
=HWpj
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/18/2011 8:30 PM, Dimitar Georgievski wrote: SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 I'm no expert, but seeing keepalives disabled for a proxy connection would sure make me think that maybe that setting is disabling, ya know, the keepalive-style connection that AJP is supposed to maintain. by monitoring Apache performance through stats provided by mod_status I've noticed that the AJP TCP connection is always closed after Tomcat serves a static resource like CSS, JavaScript, image or a JAR file . How about after a request to a dynamic resource? by monitoring Apache performance through stats provided by mod_status I've noticed that the AJP TCP connection is always closed after Tomcat serves a static resource like CSS, JavaScript, image or a JAR file So you're sure you're seeing the AJP connection close and not just the HTTP connection though it? I'd try NOT disabling keepalives and see if that helps. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6eJWsACgkQ9CaO5/Lv0PB+vQCfTTS+zrEO2pnrQkQMRxjGiON1 NWIAoJPuBwz6yP0ilHCIG1ofUlXVQvpT =dlYi -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Reuse mod ajp proxy connections
SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 This setting should control the mod_http_proxy connections. I forgot to mention this Apache server does that. I'll definitely take out these settings, because the http proxy connections should also be persistent. Regarding the upgrade of the Tomcat server we might not be able to do that. The application hosted by Tomcat has dependency on the Tomcat 5.5.x version which limits our options. I'll need to determine yet which version would be compliant with our application. I am also considering to install Apache Portable Runtime to improve connectors performance and determine its effects on the persistence of the proxy connections. What do you think? Thanks, Dimitar On Tue, Oct 18, 2011 at 9:18 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dimitar, On 10/18/2011 8:30 PM, Dimitar Georgievski wrote: SetEnv force-proxy-request-1.0 1 SetEnv proxy-nokeepalive 1 I'm no expert, but seeing keepalives disabled for a proxy connection would sure make me think that maybe that setting is disabling, ya know, the keepalive-style connection that AJP is supposed to maintain. by monitoring Apache performance through stats provided by mod_status I've noticed that the AJP TCP connection is always closed after Tomcat serves a static resource like CSS, JavaScript, image or a JAR file . How about after a request to a dynamic resource? by monitoring Apache performance through stats provided by mod_status I've noticed that the AJP TCP connection is always closed after Tomcat serves a static resource like CSS, JavaScript, image or a JAR file So you're sure you're seeing the AJP connection close and not just the HTTP connection though it? I'd try NOT disabling keepalives and see if that helps. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6eJWsACgkQ9CaO5/Lv0PB+vQCfTTS+zrEO2pnrQkQMRxjGiON1 NWIAoJPuBwz6yP0ilHCIG1ofUlXVQvpT =dlYi -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
