Hey,
Really hoping somebody can help. I am attempting a cluster with session
replication. Tomcat 7, apache with mod_jk. The cluster and load balancing
seems to be ok but I cannot get session replication working. I am using
static membership as I am testing this on EC2. The load balancer and 2
workers are on individual servers.
I have two tomcat instances, tomcatone and tomcattwo. When starting up
tomcattwo I saw this ONCE on tomcattwo and never again :
*INFO: Manager [localhost#/ClusterApp], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://tomcatone:4110,tomcatone,4110,
alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14
15 1 }, payload={}, command={}, domain={100 101 108 116 97 45 115 116 97
...(12)}, ]. This operation will timeout if no session state has been
received within 60 seconds.*
*Sep 20, 2012 3:19:45 PM org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions*
*SEVERE: Manager [localhost#/ClusterApp]: No session state send at 9/20/12
3:18 PM received, timing out after 60,109 ms.*
The failure was due to firewall I think. Now the firewall is open and I
haven't seen this again. Weird. Right now it doesn't look like the nodes
are trying to communicate at all.
Many thanks in advance!
Server.xml's and logs :
*tomcatone server.xml*
*
*
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<!-- Security listener. Documentation at /docs/config/listeners.html
<Listener className="org.apache.catalina.security.SecurityListener" />
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that share
a single "Container" Note: A "Service" is not itself a "Container",
so you may not define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one or
more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking &
non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL HTTP/1.1 Connector on port 8443
This connector uses the JSSE configuration, when using APR, the
connector should be using the OpenSSL style configuration
described in the APR documentation -->
<!--
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that
processes
every request. The Engine implementation for Tomcat stand alone
analyzes the HTTP headers included with the request, and passes
them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker1">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8" channelStartOptions="3">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="localhost" port="4110" autoBind="100"
selectorTimeout="5000" maxThreads="6"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<!--Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4110"
host="localhost"
domain="delta-static"
uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"
/-->
<Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4210"
host="tomcattwo"
domain="delta-static"
uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,2}"
/>
</Interceptor>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
<!-- Use the LockOutRealm to prevent attempts to guess user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web
applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common"
-->
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
*tomcat2 server.xml*
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<!-- Security listener. Documentation at /docs/config/listeners.html
<Listener className="org.apache.catalina.security.SecurityListener" />
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that share
a single "Container" Note: A "Service" is not itself a "Container",
so you may not define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one or
more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking &
non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL HTTP/1.1 Connector on port 8443
This connector uses the JSSE configuration, when using APR, the
connector should be using the OpenSSL style configuration
described in the APR documentation -->
<!--
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that
processes
every request. The Engine implementation for Tomcat stand alone
analyzes the HTTP headers included with the request, and passes
them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost" jvmRoute="worker2">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8" channelStartOptions="3">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="localhost" port="4210" autoBind="100"
selectorTimeout="5000" maxThreads="6"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<!--Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4110"
host="localhost"
domain="delta-static"
uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"
/-->
<Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4110"
host="tomcatone"
domain="delta-static"
uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,1}"
/>
</Interceptor>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
<!-- Use the LockOutRealm to prevent attempts to guess user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web
applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common"
-->
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
*Cluster Start Node 1*
*
*
INFO: Cluster is about to start
Sep 20, 2012 5:13:38 PM org.apache.catalina.tribes.transport.ReceiverBase
bind
INFO: Receiver Server Socket bound to:localhost/127.0.0.1:4110
Sep 20, 2012 5:13:39 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
/usr/local/apache-tomcat-7.0.30/webapps/ClusterApp.war
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Register manager localhost#/ClusterApp to cluster element Engine with
name Catalina
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Starting clustering manager at localhost#/ClusterApp
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
INFO: Manager [localhost#/ClusterApp]: skipping state transfer. No members
active in cluster group.
Sep 20, 2012 5:14:49 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/examples
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/docs
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/host-manager
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/ROOT
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/manager
Sep 20, 2012 5:14:51 PM org.apache.catalina.ha.session.JvmRouteBinderValve
startInternal
INFO: JvmRouteBinderValve started
Sep 20, 2012 5:14:51 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Sep 20, 2012 5:14:51 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Sep 20, 2012 5:14:52 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 73236 ms
*
*
*Cluster Start Node 2*
*
*
INFO: Cluster is about to start
Sep 20, 2012 5:16:45 PM org.apache.catalina.tribes.transport.ReceiverBase
bind
INFO: Receiver Server Socket bound to:localhost/127.0.0.1:4210
Sep 20, 2012 5:16:45 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
/usr/local/apache-tomcat-7.0.30/webapps/ClusterApp.war
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Register manager localhost#/ClusterApp to cluster element Engine with
name Catalina
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Starting clustering manager at localhost#/ClusterApp
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
*INFO: Manager [localhost#/ClusterApp]: skipping state transfer. No members
active in cluster group.*
Sep 20, 2012 5:17:53 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/examples
Sep 20, 2012 5:17:54 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/docs
Sep 20, 2012 5:17:54 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/host-manager
Sep 20, 2012 5:17:55 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/ROOT
Sep 20, 2012 5:17:55 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/manager
Sep 20, 2012 5:17:55 PM org.apache.catalina.ha.session.JvmRouteBinderValve
startInternal
INFO: JvmRouteBinderValve started
Sep 20, 2012 5:17:55 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Sep 20, 2012 5:17:55 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Sep 20, 2012 5:17:56 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 70803 ms
--
John Rellis
