I'm resubmitting this since we didn't get any response from the original
posting. In the meantime, we've tried duplicating the problem in a
non-production environment, but have been unsuccessful in reproducing the problem.
The problem, again, is that users with established sessions on one of the Tomcat
instances are being redirected to another Tomcat instance because the mod_jk
module believes the original Tomcat instance is in an error state:
(from the mod_jk.log on the Apache side):
[Wed Jan 31 13:58:34 2007] [10614:16384] [error]
ajp_service::jk_ajp_common.c (1879): (mahiWorker) Connecting to tomcat
failed. Tomcat is probably not started or is listening on the wrong port
On the Tomcat side, we see the SocketTimeout Exception. However, other jk
connections to the Tomcat instance seem to be OK, and other Apache/mod_jk
frontends don't detect the error state and continue to send existing and new
requests to the Tomcat instance without any problems.
In researching this, I've seen some indication that the errors we are seeing may
be 'noise' and not necessarily indicative of a problem. If that's the case, is
there any other logging or errors we should be looking for?
As stated previously, there is a PIX firewall between the Apache/mod_jk
frontends and the Tomcat instances. I know in the past there have been issues
w/ firewalls, but we are using the socket_keepalive option.
Any help or guidance will be appreciated.
Thanks,
Chris
--
--- Begin Message ---
We currently have the following setup: Two Tomcat servers and two Apache web
servers using the JK connector to load balance the Tomcat servers. There is a
Cisco PIX firewall between the Apache servers and the Tomcat servers, but we are
using the keepalive option (see config below).
Apache 2.0.46
Tomcat 5.5.9
mod_jk 1.2.19
OS:
Apache servers: Red Hat Enterprise Linux ES 3 Update 8 (2.4.21-32.0.1.EL)
Tomcat servers: Red Hat Enterprise Linux ES 3 Update 8 (2.4.21-32.0.1.EL)
The problem is that users are being kicked out before their sessions have
expired and before they log out. A user will be navigating any random page on
the site when they are suddenly kicked out from one Tomcat instance and "load
balanced" over to the other instance. Since we aren't using session clustering
the user will be thrown to the login page of the other Tomcat instance because
they don't have a replicated session on that instance.
When this happens we see the following log messages:
catalina.out (on the Tomcat server that user was kicked out of):
SEVERE: Error decoding request
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:601)
at
org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:538)
at
org.apache.jk.common.JkInputStream.receive(JkInputStream.java:247)
at
org.apache.jk.common.HandlerRequest.decodeRequest(HandlerRequest.java:525)
at
org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:363)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:748)
at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:678)
at
org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:871)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Jan 31, 2007 2:07:15 PM org.apache.jk.common.ChannelSocket processConnection
mod_jk.log:
[Wed Jan 31 13:58:34 2007] [10614:16384] [error]
ajp_service::jk_ajp_common.c (1879): (mahiWorker) Connecting to tomcat
failed. Tomcat is probably not started or is listening on the wrong port
Here are the JK Connector settings in the Tomcat instances:
<Connector
port="8009"
enableLookups="false"
redirectPort="8443"
protocol="AJP/1.3"
connectionTimeout="180000"
maxThreads="400"
maxSpareThreads="8"
minSpareThreads="1" />
And here is the worker.properties file on Apache (Note that this problem is only
happening with Load Balancer lb2, with mahiWorker, mulletWorker. The other load
balancer is dealing with older version of Tomcat and different application and
JDK versions).
# List the workers by name
worker.list=lb1,lb2,status
# -----------------------
# Loadbalance Worker for ES
# -----------------------
worker.lb1.type=lb
worker.lb1.sticky_session=1
worker.lb1.balance_workers=mahiWorker,mulletWorker
worker.lb1.method=Request
# -----------------------
# Loadbalance Worker for TPSI
# -----------------------
worker.lb2.type=lb
worker.lb2.sticky_session=1
worker.lb2.balance_workers=makoWorker,marlinWorker
worker.lb2.method=Request
# -----------------------
# Status Worker
# -----------------------
worker.status.type=status
worker.status.css=/jk.css
# -----------------------
# mako Tomcat Server
# -----------------------
worker.makoWorker.port=8009
worker.makoWorker.host=192.168.221.8
worker.makoWorker.type=ajp13
worker.makoWorker.lbfactor=1
#worker.makoWorker.cachesize=25
#worker.makoWorker.cache_timeout=600
worker.makoWorker.socket_keepalive=1
worker.makoWorker.socket_timeout=30
worker.makoWorker.connection_pool_timeout=600
worker.makoWorker.connect_timeout=60000
# -----------------------
# marlin Tomcat Server
# -----------------------
worker.marlinWorker.port=8009
worker.marlinWorker.host=192.168.221.9
worker.marlinWorker.type=ajp13
worker.marlinWorker.lbfactor=1
#worker.marlinWorker.cachesize=25
#worker.marlinWorker.cache_timeout=600
worker.marlinWorker.socket_keepalive=1
worker.marlinWorker.socket_timeout=30
worker.marlinWorker.connection_pool_timeout=600
worker.marlinWorker.connect_timeout=60000
# -----------------------
# mahi Tomcat Server
# -----------------------
worker.mahiWorker.port=8009
worker.mahiWorker.host=192.168.221.10
worker.mahiWorker.type=ajp13
worker.mahiWorker.lbfactor=1
#worker.mahiWorker.cachesize=25
#worker.mahiWorker.cache_timeout=600
worker.mahiWorker.socket_keepalive=1
worker.mahiWorker.socket_timeout=30
worker.mahiWorker.connection_pool_timeout=600
worker.mahiWorker.connect_timeout=180000
# -----------------------
# mullet Tomcat Server
# -----------------------
worker.mulletWorker.port=8009
worker.mulletWorker.host=192.168.221.11
worker.mulletWorker.type=ajp13
worker.mulletWorker.lbfactor=1
#worker.mulletWorker.cachesize=25
#worker.mulletWorker.cache_timeout=600
worker.mulletWorker.socket_keepalive=1
worker.mulletWorker.socket_timeout=30
worker.mulletWorker.connection_pool_timeout=600
worker.mulletWorker.connect_timeout=180000
--
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--- End Message ---
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
