RE: Tomcat 9 Session replication
Many thanks Mark!
-Original Message-
From: Mark Thomas
Sent: 01 February 2022 09:25
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
On 31/01/2022 14:54, Alan F wrote:
> Many thanks Chris,
>
> Don't laugh I was looking at those values after Keiichi kindly mentioned this
> too (thankyou!) and was thinking hmm where is 15 no mention! Ok makes sense
> now.
>
> Im trying to find out why we chose static, I think it was a guess at trying
> to stop the multicast interference from other hosts. I think we just had it
> all set wrong to start with.
>
> Looking at many online examples like here
> https://tomcat.apache.org/tomcat-9.0-doc/cluster-howto.html is what i looked
> would this suffice?
>
> If you could recommend the params I need just for two hosts to talk without
> interference ie specifying group or unique id etc.
This is a working static cluster example from my 4-node test cluster.
https://people.apache.org/~markt/dev/server-static-cluster-example.xml
Things you'll need to change:
- the jvmRoute on the Engine element
- must be unique for each node
- must match the reverse proxy config for sticky sessions to work
- you'll only need two members
- LocalMember needs to be correct on each node
- The address attribute of the receiver needs to be correct on each node
Mark
>
> -Original Message-
> From: Christopher Schultz
> Sent: 31 January 2022 14:46
> To: users@tomcat.apache.org
> Subject: Re: Tomcat 9 Session replication
>
> All,
>
> On 1/31/22 08:04, Keiichi Fujino wrote:
>> If you use StaticMembershipService, you must set
>> Cluster#channelStartOptions to 15 (default).
>
> To spell that out (since the docs aren't very explicit), the value of "15" is
> the combination of the following flags:
>
> SND_RX_SEQ (1) - Starts the data receiver.
> SND_TX_SEQ (2) - Starts the data transmitter ("sender").
> MBR_RX_SEQ (4) - Starts the membership receiver ("listener").
> MBR_TX_SEQ (8) - Starts the membership transmitter ("broadcaster").
> |= 15 (0xf)
>
> I'm curious why, if one is using static membership, are the membership
> transmitter and receiver flags required? It seems to be that the membership
> should remain static and therefore no membership comms shuould be required.
> Are those important to ensure that the cluster members (through static) are
> actually present during operation?
>
> -chris
>
>> 2022年1月31日(月) 16:47 Alan F :
>>
>>> OK with your advice I tried what I thought would work from example
>>> and doesn't at all. The old example below works but this doesn’t
>>> even detect members.
>>>
>>> Below is example to which Im using on both nodes which are remote to
>>> eachother.
>>>
>>>
>>> >> channelSendOptions="8" channelStartOptions = "3">
>>> >> expireSessionsOnShutdown="false"
>>> notifyListenersOnReplication="true"/>
>>>
>>> >> className="org.apache.catalina.tribes.group.GroupChannel">
>>>
>>> >> className="org.apache.catalina.tribes.membership.StaticMembershipService">
>>> >> className="org.apache.catalina.tribes.membership.StaticMember"
>>> port="4110"
>>> host="local-tomcat"
>>> domain="tomcat-pc2"
>>>
>>> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}" />
>>> >> className="org.apache.catalina.tribes.membership.StaticMember"
>>> port="4110"
>>> host="remote-tomcat"
>>> domain="tomcat-pc2"
>>>
>>> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}" />
>>>
>>>
>>> >> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>>>address="local-tomcat"
>>>port="4110"
>>>autoBind="9"
>>>selectorTimeout="2000"
>>>maxThreads="6"/>
>>>
>>> >> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
>>>
Re: Tomcat 9 Session replication
On 31/01/2022 14:54, Alan F wrote:
Many thanks Chris,
Don't laugh I was looking at those values after Keiichi kindly mentioned this
too (thankyou!) and was thinking hmm where is 15 no mention! Ok makes sense
now.
Im trying to find out why we chose static, I think it was a guess at trying to
stop the multicast interference from other hosts. I think we just had it all
set wrong to start with.
Looking at many online examples like here
https://tomcat.apache.org/tomcat-9.0-doc/cluster-howto.html is what i looked
would this suffice?
If you could recommend the params I need just for two hosts to talk without
interference ie specifying group or unique id etc.
This is a working static cluster example from my 4-node test cluster.
https://people.apache.org/~markt/dev/server-static-cluster-example.xml
Things you'll need to change:
- the jvmRoute on the Engine element
- must be unique for each node
- must match the reverse proxy config for sticky sessions to work
- you'll only need two members
- LocalMember needs to be correct on each node
- The address attribute of the receiver needs to be correct on each node
Mark
-Original Message-
From: Christopher Schultz
Sent: 31 January 2022 14:46
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
All,
On 1/31/22 08:04, Keiichi Fujino wrote:
If you use StaticMembershipService, you must set
Cluster#channelStartOptions to 15 (default).
To spell that out (since the docs aren't very explicit), the value of "15" is
the combination of the following flags:
SND_RX_SEQ (1) - Starts the data receiver.
SND_TX_SEQ (2) - Starts the data transmitter ("sender").
MBR_RX_SEQ (4) - Starts the membership receiver ("listener").
MBR_TX_SEQ (8) - Starts the membership transmitter ("broadcaster").
|= 15 (0xf)
I'm curious why, if one is using static membership, are the membership
transmitter and receiver flags required? It seems to be that the membership
should remain static and therefore no membership comms shuould be required. Are
those important to ensure that the cluster members (through static) are
actually present during operation?
-chris
2022年1月31日(月) 16:47 Alan F :
OK with your advice I tried what I thought would work from example
and doesn't at all. The old example below works but this doesn’t even
detect members.
Below is example to which Im using on both nodes which are remote to
eachother.
-Original Message-
From: Mark Thomas
Sent: 28 January 2022 18:15
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
On 28/01/2022 17:05, Alan F wrote:
We are currently getting traffic from all cluster members in other
environments using .staticmember opposed to multicast can I confirm
why this is see below.
What do we need to set here for a clustered pair to make them unique
and talk to eachother only without seeing traffic from other members
in Catalina.out
This is how they are currently configured as you can see only
difference
between nodes is the receiver and member ips are reversed. Im
concerned uniqueID is the same, do we also need to specify domain?
uniqueID should be unique at least within the subnet, ideally globally.
Yes, you should use a separate domain for each cluster.
Looking at your config:
You are missing the local member definition
https://tomcat.apache.org/tomcat-10.0-doc/config/cluster-interceptor.
html#Static_Membership
The deployer should be defined under the cluster, not under an interceptor.
Mark
WE have hostname A and B if you could check below:
HOSTNAME A config
channelSendOptions="8">
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
className="org.apache.catalina.tribes.transport.ReplicationTransmitte
r">
className="org.apache.catalina.tribes.transport.nio.PooledParallelSen
der" />
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
maxThreads="6"
port="4100"
selectorTimeout="5000"
/>
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"
staticOnly="true"/>
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
/>
className="org.apache.catalina.tribes.group.interceptors.StaticMember
shipIntercep
Re: Tomcat 9 Session replication
Alan,
On 1/31/22 09:54, Alan F wrote:
Many thanks Chris,
Don't laugh I was looking at those values after Keiichi kindly mentioned this
too (thankyou!) and was thinking hmm where is 15 no mention! Ok makes sense
now.
Im trying to find out why we chose static, I think it was a guess at trying to
stop the multicast interference from other hosts. I think we just had it all
set wrong to start with.
Looking at many online examples like here
https://tomcat.apache.org/tomcat-9.0-doc/cluster-howto.html is what i looked
would this suffice?
If you could recommend the params I need just for two hosts to talk without
interference ie specifying group or unique id etc.
Keiichi knows what he's talking about: set that channelStartOptions
value to "15" and try again.
With static membership, there is no multicast so there shouldn't be any
interference with other non-members. Just make sure you have your ports
unique for each cluster and your membership ids unique for each node in
each cluster.
-chris
-Original Message-
From: Christopher Schultz
Sent: 31 January 2022 14:46
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
All,
On 1/31/22 08:04, Keiichi Fujino wrote:
If you use StaticMembershipService, you must set
Cluster#channelStartOptions to 15 (default).
To spell that out (since the docs aren't very explicit), the value of "15" is
the combination of the following flags:
SND_RX_SEQ (1) - Starts the data receiver.
SND_TX_SEQ (2) - Starts the data transmitter ("sender").
MBR_RX_SEQ (4) - Starts the membership receiver ("listener").
MBR_TX_SEQ (8) - Starts the membership transmitter ("broadcaster").
|= 15 (0xf)
I'm curious why, if one is using static membership, are the membership
transmitter and receiver flags required? It seems to be that the membership
should remain static and therefore no membership comms shuould be required. Are
those important to ensure that the cluster members (through static) are
actually present during operation?
-chris
2022年1月31日(月) 16:47 Alan F :
OK with your advice I tried what I thought would work from example
and doesn't at all. The old example below works but this doesn’t even
detect members.
Below is example to which Im using on both nodes which are remote to
eachother.
-Original Message-
From: Mark Thomas
Sent: 28 January 2022 18:15
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
On 28/01/2022 17:05, Alan F wrote:
We are currently getting traffic from all cluster members in other
environments using .staticmember opposed to multicast can I confirm
why this is see below.
What do we need to set here for a clustered pair to make them unique
and talk to eachother only without seeing traffic from other members
in Catalina.out
This is how they are currently configured as you can see only
difference
between nodes is the receiver and member ips are reversed. Im
concerned uniqueID is the same, do we also need to specify domain?
uniqueID should be unique at least within the subnet, ideally globally.
Yes, you should use a separate domain for each cluster.
Looking at your config:
You are missing the local member definition
https://tomcat.apache.org/tomcat-10.0-doc/config/cluster-interceptor.
html#Static_Membership
The deployer should be defined under the cluster, not under an interceptor.
Mark
WE have hostname A and B if you could check below:
HOSTNAME A config
channelSendOptions="8">
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
className="org.apache.catalina.tribes.transport.ReplicationTransmitte
r">
className="org.apache.catalina.tribes.transport.nio.PooledParallelSen
der" />
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
maxThreads="6"
port="4100"
selectorTimeout="5000"
/>
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"
staticOnly="true"/>
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
/>
className="org.apache.catalina.tribes.group.interceptors.StaticMember
shipInterceptor">
className="org.apache.catalina.tribes.membership.StaticMember"
port="4100"
RE: Tomcat 9 Session replication
Many thanks Chris,
Don't laugh I was looking at those values after Keiichi kindly mentioned this
too (thankyou!) and was thinking hmm where is 15 no mention! Ok makes sense
now.
Im trying to find out why we chose static, I think it was a guess at trying to
stop the multicast interference from other hosts. I think we just had it all
set wrong to start with.
Looking at many online examples like here
https://tomcat.apache.org/tomcat-9.0-doc/cluster-howto.html is what i looked
would this suffice?
If you could recommend the params I need just for two hosts to talk without
interference ie specifying group or unique id etc.
-Original Message-
From: Christopher Schultz
Sent: 31 January 2022 14:46
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
All,
On 1/31/22 08:04, Keiichi Fujino wrote:
> If you use StaticMembershipService, you must set
> Cluster#channelStartOptions to 15 (default).
To spell that out (since the docs aren't very explicit), the value of "15" is
the combination of the following flags:
SND_RX_SEQ (1) - Starts the data receiver.
SND_TX_SEQ (2) - Starts the data transmitter ("sender").
MBR_RX_SEQ (4) - Starts the membership receiver ("listener").
MBR_TX_SEQ (8) - Starts the membership transmitter ("broadcaster").
|= 15 (0xf)
I'm curious why, if one is using static membership, are the membership
transmitter and receiver flags required? It seems to be that the membership
should remain static and therefore no membership comms shuould be required. Are
those important to ensure that the cluster members (through static) are
actually present during operation?
-chris
> 2022年1月31日(月) 16:47 Alan F :
>
>> OK with your advice I tried what I thought would work from example
>> and doesn't at all. The old example below works but this doesn’t even
>> detect members.
>>
>> Below is example to which Im using on both nodes which are remote to
>> eachother.
>>
>>
>> > channelSendOptions="8" channelStartOptions = "3">
>>> expireSessionsOnShutdown="false"
>> notifyListenersOnReplication="true"/>
>>
>>> className="org.apache.catalina.tribes.group.GroupChannel">
>>
>> > className="org.apache.catalina.tribes.membership.StaticMembershipService">
>> > className="org.apache.catalina.tribes.membership.StaticMember"
>>port="4110"
>>host="local-tomcat"
>>domain="tomcat-pc2"
>>
>> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}" />
>> > className="org.apache.catalina.tribes.membership.StaticMember"
>>port="4110"
>>host="remote-tomcat"
>>domain="tomcat-pc2"
>>
>> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}" />
>>
>>
>> > className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>> address="local-tomcat"
>> port="4110"
>> autoBind="9"
>> selectorTimeout="2000"
>> maxThreads="6"/>
>>
>> > className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
>> > className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
>>
>>
>> > className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"
>> staticOnly="true"/>
>> > className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
>> />
>> > className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
>> > className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
>> > className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
>>
>>
>>
>>
>>> filter=""/>
>>> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
>>
>>> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>> tempDir="/op
Re: Tomcat 9 Session replication
Hi
If you use StaticMembershipService, you must set
Cluster#channelStartOptions to 15 (default).
2022年1月31日(月) 16:47 Alan F :
> OK with your advice I tried what I thought would work from example and
> doesn't at all. The old example below works but this doesn’t even detect
> members.
>
> Below is example to which Im using on both nodes which are remote to
> eachother.
>
>
> channelSendOptions="8" channelStartOptions = "3">
> expireSessionsOnShutdown="false"
>notifyListenersOnReplication="true"/>
>
>
>
> className="org.apache.catalina.tribes.membership.StaticMembershipService">
> className="org.apache.catalina.tribes.membership.StaticMember"
> port="4110"
> host="local-tomcat"
> domain="tomcat-pc2"
>
> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}" />
> className="org.apache.catalina.tribes.membership.StaticMember"
> port="4110"
> host="remote-tomcat"
> domain="tomcat-pc2"
>
> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}" />
>
>
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
> address="local-tomcat"
> port="4110"
> autoBind="9"
> selectorTimeout="2000"
> maxThreads="6"/>
>
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
>
>
> className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"
> staticOnly="true"/>
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
> />
> className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
> className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
> className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
>
>
>
>
>filter=""/>
>className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
>
>className="org.apache.catalina.ha.deploy.FarmWarDeployer"
> tempDir="/opt/tomcat/war-temp/"
> deployDir="/opt/tomcat/war-deploy/"
> watchDir="/opt/tomcat/war-listen/"
> watchEnabled="true"/>
>className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>
>
>
> -Original Message-
> From: Mark Thomas
> Sent: 28 January 2022 18:15
> To: users@tomcat.apache.org
> Subject: Re: Tomcat 9 Session replication
>
> On 28/01/2022 17:05, Alan F wrote:
> > We are currently getting traffic from all cluster members in other
> environments using .staticmember opposed to multicast can I confirm why
> this is see below.
> >
> > What do we need to set here for a clustered pair to make them unique
> > and talk to eachother only without seeing traffic from other members
> > in Catalina.out
> >
> > This is how they are currently configured as you can see only difference
> between nodes is the receiver and member ips are reversed. Im concerned
> uniqueID is the same, do we also need to specify domain?
>
> uniqueID should be unique at least within the subnet, ideally globally.
>
> Yes, you should use a separate domain for each cluster.
>
> Looking at your config:
>
> You are missing the local member definition
> https://tomcat.apache.org/tomcat-10.0-doc/config/cluster-interceptor.html#Static_Membership
>
> The deployer should be defined under the cluster, not under an interceptor.
>
> Mark
>
> >
> > WE have hostname A and B if you could check below:
> >
> > HOSTNAME A config
> >
> >
> >
> > channelSendOptions="8">
> > className="org.apache.catalina.ha.session.DeltaManager"
> > expireSessionsOnShutdown="false"
> > notifyListenersOnReplication="true&quo
RE: Tomcat 9 Session replication
OK with your advice I tried what I thought would work from example and doesn't
at all. The old example below works but this doesn’t even detect members.
Below is example to which Im using on both nodes which are remote to eachother.
-Original Message-
From: Mark Thomas
Sent: 28 January 2022 18:15
To: users@tomcat.apache.org
Subject: Re: Tomcat 9 Session replication
On 28/01/2022 17:05, Alan F wrote:
> We are currently getting traffic from all cluster members in other
> environments using .staticmember opposed to multicast can I confirm why this
> is see below.
>
> What do we need to set here for a clustered pair to make them unique
> and talk to eachother only without seeing traffic from other members
> in Catalina.out
>
> This is how they are currently configured as you can see only difference
> between nodes is the receiver and member ips are reversed. Im concerned
> uniqueID is the same, do we also need to specify domain?
uniqueID should be unique at least within the subnet, ideally globally.
Yes, you should use a separate domain for each cluster.
Looking at your config:
You are missing the local member definition
https://tomcat.apache.org/tomcat-10.0-doc/config/cluster-interceptor.html#Static_Membership
The deployer should be defined under the cluster, not under an interceptor.
Mark
>
> WE have hostname A and B if you could check below:
>
> HOSTNAME A config
>
>
>
> channelSendOptions="8">
> expireSessionsOnShutdown="false"
> notifyListenersOnReplication="true"/>
>
>
>
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
>
>address="hostnameA"
> autoBind="0"
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
> maxThreads="6"
> port="4100"
> selectorTimeout="5000"
> />
>
> className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"
> staticOnly="true"/>
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
> />
> className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
>
> className="org.apache.catalina.tribes.membership.StaticMember"
> port="4100"
> host="HostnameB"
> uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1}"
> />
> tempDir="/opt/tomcat/war-temp/"
> deployDir="/opt/tomcat/war-deploy/"
> watchDir="/opt/tomcat/war-listen/"
> watchEnabled="true"/>
>
>
> className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"
> />
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 9 Session replication
On 28/01/2022 17:05, Alan F wrote: We are currently getting traffic from all cluster members in other environments using .staticmember opposed to multicast can I confirm why this is see below. What do we need to set here for a clustered pair to make them unique and talk to eachother only without seeing traffic from other members in Catalina.out This is how they are currently configured as you can see only difference between nodes is the receiver and member ips are reversed. Im concerned uniqueID is the same, do we also need to specify domain? uniqueID should be unique at least within the subnet, ideally globally. Yes, you should use a separate domain for each cluster. Looking at your config: You are missing the local member definition https://tomcat.apache.org/tomcat-10.0-doc/config/cluster-interceptor.html#Static_Membership The deployer should be defined under the cluster, not under an interceptor. Mark WE have hostname A and B if you could check below: HOSTNAME A config - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
