Here's the two stack traces, in order of occurrence, for the DeltaSession
creation. The second seems to the result of a cluster message received
which seems odd on the machine where the session is being created:
<init>:109, DeltaSession (org.apache.catalina.ha.session)
createEmptySession:471, DeltaManager (org.apache.catalina.ha.session)
createSession:708, ManagerBase (org.apache.catalina.session)
createSession:422, DeltaManager (org.apache.catalina.ha.session)
createSession:410, DeltaManager (org.apache.catalina.ha.session)
doGetSession:3043, Request (org.apache.catalina.connector)
getSession:2441, Request (org.apache.catalina.connector)
getSession:908, RequestFacade (org.apache.catalina.connector)
getSession:920, RequestFacade (org.apache.catalina.connector)
getSession:253, HttpServletRequestWrapper (javax.servlet.http)
getSession:253, HttpServletRequestWrapper (javax.servlet.http)
getSession:253, HttpServletRequestWrapper (javax.servlet.http)
onAuthentication:66, RegisterSessionAuthenticationStrategy
(org.springframework.security.web.authentication.session)
doFilter:218, AbstractAuthenticationProcessingFilter
(org.springframework.security.web.authentication)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:200, AbstractAuthenticationProcessingFilter
(org.springframework.security.web.authentication)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:116, LogoutFilter
(org.springframework.security.web.authentication.logout)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilterInternal:74, HeaderWriterFilter
(org.springframework.security.web.header)
doFilter:119, OncePerRequestFilter (org.springframework.web.filter)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:74, CSRFSameOriginHeaderCheckFilter (ourcompany.common.web)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:36, SessionTrackingFilter (ourcompany.common.web)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:105, SecurityContextPersistenceFilter
(org.springframework.security.web.context)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilterInternal:56, WebAsyncManagerIntegrationFilter
(org.springframework.security.web.context.request.async)
doFilter:119, OncePerRequestFilter (org.springframework.web.filter)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:157, ChannelProcessingFilter
(org.springframework.security.web.access.channel)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilter:87, MetadataGeneratorFilter
(org.springframework.security.saml.metadata)
doFilter:334, FilterChainProxy$VirtualFilterChain
(org.springframework.security.web)
doFilterInternal:215, FilterChainProxy (org.springframework.security.web)
doFilter:178, FilterChainProxy (org.springframework.security.web)
invokeDelegate:358, DelegatingFilterProxy (org.springframework.web.filter)
doFilter:271, DelegatingFilterProxy (org.springframework.web.filter)
internalDoFilter:193, ApplicationFilterChain (org.apache.catalina.core)
doFilter:166, ApplicationFilterChain (org.apache.catalina.core)
invoke:202, StandardWrapperValve (org.apache.catalina.core)
__invoke:96, StandardContextValve (org.apache.catalina.core)
invoke:41002, StandardContextValve (org.apache.catalina.core)
invoke:541, AuthenticatorBase (org.apache.catalina.authenticator)
invoke:139, StandardHostValve (org.apache.catalina.core)
invoke:182, JvmRouteBinderValve (org.apache.catalina.ha.session)
invoke:330, ReplicationValve (org.apache.catalina.ha.tcp)
invoke:92, ErrorReportValve (org.apache.catalina.valves)
invoke:74, StandardEngineValve (org.apache.catalina.core)
invoke:747, RemoteIpValve (org.apache.catalina.valves)
service:343, CoyoteAdapter (org.apache.catalina.connector)
service:373, Http11Processor (org.apache.coyote.http11)
process:65, AbstractProcessorLight (org.apache.coyote)
process:868, AbstractProtocol$ConnectionHandler (org.apache.coyote)
doRun:1589, NioEndpoint$SocketProcessor (org.apache.tomcat.util.net)
run:49, SocketProcessorBase (org.apache.tomcat.util.net)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:61, TaskThread$WrappingRunnable (org.apache.tomcat.util.threads)
run:748, Thread (java.lang)
...and the second session creation:
<init>:109, DeltaSession (org.apache.catalina.ha.session)
createEmptySession:471, DeltaManager (org.apache.catalina.ha.session)
handleSESSION_CREATED:1314, DeltaManager (org.apache.catalina.ha.session)
messageReceived:1192, DeltaManager (org.apache.catalina.ha.session)
messageDataReceived:949, DeltaManager (org.apache.catalina.ha.session)
messageReceived:77, ClusterSessionListener (org.apache.catalina.ha.session)
messageReceived:788, SimpleTcpCluster (org.apache.catalina.ha.tcp)
messageReceived:771, SimpleTcpCluster (org.apache.catalina.ha.tcp)
messageReceived:335, GroupChannel (org.apache.catalina.tribes.group)
messageReceived:91, ChannelInterceptorBase (org.apache.catalina.tribes.group)
messageReceived:97, StaticMembershipInterceptor
(org.apache.catalina.tribes.group.interceptors)
messageReceived:91, ChannelInterceptorBase (org.apache.catalina.tribes.group)
messageReceived:175, TcpPingInterceptor
(org.apache.catalina.tribes.group.interceptors)
messageReceived:91, ChannelInterceptorBase (org.apache.catalina.tribes.group)
messageReceived:117, TcpFailureDetector
(org.apache.catalina.tribes.group.interceptors)
messageReceived:91, ChannelInterceptorBase (org.apache.catalina.tribes.group)
messageReceived:91, ChannelInterceptorBase (org.apache.catalina.tribes.group)
messageReceived:274, ChannelCoordinator (org.apache.catalina.tribes.group)
messageDataReceived:261, ReceiverBase (org.apache.catalina.tribes.transport)
drainChannel:216, NioReplicationTask (org.apache.catalina.tribes.transport.nio)
run:101, NioReplicationTask (org.apache.catalina.tribes.transport.nio)
runWorker:1149, ThreadPoolExecutor (java.util.concurrent)
run:624, ThreadPoolExecutor$Worker (java.util.concurrent)
run:748, Thread (java.lang)
On Fri, Oct 9, 2020 at 11:40 AM Tim N <tnti...@gmail.com> wrote:
> > Can you show us how you configured this cluster please?
>
> Sure.
>
> Tomcat tomcat = new Tomcat();
> tomcat.setBaseDir(baseDir);
> tomcat.getServer().setAddress("127.0.0.1");
> tomcat.getServer().setPort(shutDownPort);
>
> tomcat.getServer().setShutdown("SHUTDOWN");
> tomcat.getServer().addLifecycleListener(new AprLifecycleListener());
> tomcat.getServer().addLifecycleListener(new
> JreMemoryLeakPreventionListener());
> tomcat.getServer().addLifecycleListener(new
> GlobalResourcesLifecycleListener());
> tomcat.getServer().addLifecycleListener(new
> ThreadLocalLeakPreventionListener());
>
> tomcat.setPort(httpPort);
> tomcat.getConnector().getPort();//The default connector will only be created
> if getConnector is called
>
> LaunchServerHelper.addRemoteIpValve(tomcat);
>
> //...
>
> StandardHost hostManage = getStandardHost(tomcat, "manage." + sgDomain,
> "sg-manage", serverProperties);
> tomcat.getEngine().addChild(hostManage);
>
> tomcat.start();
> tomcat.getServer().await();
> tomcat.stop();
>
>
> private static StandardHost getStandardHost(Tomcat tomcat, String name,
> String module, Properties serverProperties) {
> StandardHost host = new StandardHost();
>
> modifyHost(tomcat, name, module, host, serverProperties);
> return host;
> }
>
>
> private static void modifyHost(Tomcat tomcat, String name, String module,
> StandardHost host, Properties serverProperties) {
> String webappDirLocation = module + "/src/main/webapp/";
> File webAppDir = new File(webappDirLocation);
>
> host.setName(name);
> String[] aliases = serverProperties.getProperty("tomcat-" + module +
> "-aliases", "").split(",");
> for (String alias : aliases) {
> if (alias.length() > 0) {
> logger.log(Level.INFO, "Adding alias {0} for {1}", new Object[] {alias,
> module});
> host.addAlias(alias);
> }
> }
> host.setParent(tomcat.getEngine());
> host.setAppBase(webAppDir.getAbsolutePath());
>
> StandardContext ctx = (StandardContext) tomcat.addWebapp(host, "/",
> webAppDir.getAbsolutePath());
> ((StandardJarScanner) ctx.getJarScanner()).setScanManifest(false);
> ctx.setCookieProcessor(new LegacyCookieProcessor());
>
> // Declare an alternative location for your "WEB-INF/classes" dir
> // Servlet 3.0 annotation will work
> File additionWebInfClasses = new File(module + "/target/classes");
> WebResourceRoot resources = new StandardRoot(ctx);
> resources.addPreResources(new DirResourceSet(resources, "/WEB-INF/classes",
> additionWebInfClasses.getAbsolutePath(), "/"));
> ctx.setResources(resources);
>
> LaunchServerHelper.addSimpleTcpCluster(host, 4000, serverProperties);
> }
>
>
> public static void addSimpleTcpCluster(StandardHost host, int port,
> Properties serverProperties) {
> if (!serverProperties.containsKey("tomcat-clusterMemberCount")) {
> return;
> }
>
> SimpleTcpCluster cluster = new SimpleTcpCluster();
> cluster.setClusterName("SG-cluster");
> cluster.setChannelStartOptions(3);
> cluster.setChannelSendOptions(8);
>
> {
> DeltaManager clusterManager = new DeltaManager();
> clusterManager.setName("SG-cluster-manager");
> clusterManager.setExpireSessionsOnShutdown(false);
> clusterManager.setNotifyListenersOnReplication(true);
> cluster.registerManager(clusterManager);
> }
>
> {
> GroupChannel channel = new GroupChannel();
> channel.setName("SG-channel");
> {
> NioReceiver channelReceiver = new NioReceiver();
>
> channelReceiver.setAddress(serverProperties.getProperty("tomcat-clusterAddress"));
> channelReceiver.setPort(port);
> channelReceiver.setAutoBind(100);
> channelReceiver.setSelectorTimeout(5000);
> channelReceiver.setMaxThreads(6);
> channel.setChannelReceiver(channelReceiver);
> }
> cluster.setChannel(channel);
>
> {
> StaticMembershipInterceptor interceptor = new
> StaticMembershipInterceptor();
> int clusterMemberCount =
> Integer.parseInt(serverProperties.getProperty("tomcat-clusterMemberCount"));
> {
> StaticMember localMember = new StaticMember();
> localMember.setPort(port);
> localMember.setSecurePort(-1);
>
> localMember.setHost(serverProperties.getProperty("tomcat-clusterAddress"));
> localMember.setDomain("publish-cluster");
>
> localMember.setUniqueId(serverProperties.getProperty("tomcat-clusterMemberUniqueId"));
> interceptor.setLocalMember(localMember);
> interceptor.addStaticMember(localMember);
> }
> for (int i = 1; i <= clusterMemberCount; i++) {
> StaticMember member = new StaticMember();
> member.setLocal(false);
> member.setPort(port);
> member.setSecurePort(-1);
>
> member.setHost(serverProperties.getProperty("tomcat-clusterMemberAddress" +
> i));
> member.setDomain("publish-cluster");
>
> member.setUniqueId(serverProperties.getProperty("tomcat-clusterMemberUniqueId"
> + i));
> interceptor.addStaticMember(member);
> }
> channel.addInterceptor(interceptor);
> }
> channel.addInterceptor(new TcpPingInterceptor());
> channel.addInterceptor(new TcpFailureDetector());
> channel.addInterceptor(new MessageDispatchInterceptor());
> }
>
>
> ReplicationTransmitter replicationTransmitter = new
> ReplicationTransmitter();
> PooledParallelSender sender = new PooledParallelSender();
> sender.setTimeout(60000);
> sender.setMaxRetryAttempts(5);
> replicationTransmitter.setTransport(sender);
>
> ReplicationValve replicationValve = new ReplicationValve();
> replicationValve.setFilter("");
> cluster.addValve(replicationValve);
> cluster.addValve(new JvmRouteBinderValve());
>
> FarmWarDeployer farmDeployer = new FarmWarDeployer();
> farmDeployer.setTempDir("/tmp/war-temp/");
> farmDeployer.setDeployDir("/tmp/war-deploy/");
> farmDeployer.setWatchDir("/tmp/war-listen/");
> farmDeployer.setWatchEnabled(false);
> cluster.setClusterDeployer(farmDeployer);
>
> //cluster.addClusterListener(new
> JvmRouteSessionIDBinderListener());//Removed in Tomcat 8.
> https://tomcat.apache.org/migration-8.html#Clustering
> cluster.addClusterListener(new ClusterSessionListener());
>
> host.setCluster(cluster);
> }
>
>
> tomcat-clusterAddress=192.168.0.2
> tomcat-clusterMemberUniqueId={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}
> tomcat-clusterMemberCount=1
> tomcat-clusterMemberAddress1=192.168.0.3
> tomcat-clusterMemberUniqueId1={0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}
>
>
> This was initially converted from a server.xml configuration a year ago on
> Tomcat 7 with minor modifications as we've upgraded to Tomcat 8 then 9.
>
> Cheers,
>
> Tim
>
>
> On Thu, Oct 8, 2020 at 10:17 PM Mark Thomas <ma...@apache.org> wrote:
>
>> On 08/10/2020 10:04, Tim N wrote:
>> > Hi,
>> >
>> > I'm in the early stages of analysing this problem:
>> >
>> > - Tomcat Embedded 9.0.37 with clustering enabled
>> > - SpringBoot application, 2.1.16
>> > - Login with no existing JSESSIONID fails
>> >
>> > I can see the following code is executed twice within one request, the
>> > login attempt:
>> >
>> >
>> *~/.m2/repository/org/apache/tomcat/tomcat-catalina/9.0.37/tomcat-catalina-9.0.37-sources.jar!/org/apache/catalina/session/ManagerBase.java:677*
>> >
>> > public void add(Session session) {
>> > sessions.put(session.getIdInternal(), session);
>> >
>> > The first time it's executed, the session with the Spring context is
>> added.
>> > The second time it's executed, a second session with the same ID, but
>> > without the Spring context, or any other session attribute I add for
>> that
>> > matter, overwrites the existing session, and login fails. If I debug and
>> > prevent this by renaming the second session ID, login works because the
>> > original session is preserved.
>> >
>> > The stack-trace for the first call is shown below:
>> >
>> > add:678, ManagerBase (org.apache.catalina.session)
>> > setId:358, StandardSession (org.apache.catalina.session)
>> > setId:327, DeltaSession (org.apache.catalina.ha.session)
>> > setId:345, DeltaSession (org.apache.catalina.ha.session)
>> > createSession:719, ManagerBase (org.apache.catalina.session)
>> > createSession:422, DeltaManager (org.apache.catalina.ha.session)
>> > createSession:410, DeltaManager (org.apache.catalina.ha.session)
>> > doGetSession:3043, Request (org.apache.catalina.connector)
>> > getSession:2441, Request (org.apache.catalina.connector)
>>
>> <snip/>
>>
>> This is the app triggering the creation of the session.
>>
>> > The stack-trace for the second call is shown below:
>> >
>> > add:678, ManagerBase (org.apache.catalina.session)
>> > setId:358, StandardSession (org.apache.catalina.session)
>> > setId:327, DeltaSession (org.apache.catalina.ha.session)
>> > handleSESSION_CREATED:1322, DeltaManager
>> (org.apache.catalina.ha.session)
>> > messageReceived:1192, DeltaManager (org.apache.catalina.ha.session)
>>
>> <snip/>
>>
>> This is the DeltaManager receiving notification that a new session has
>> been created.
>>
>> > Any help would be appreciated. I can replicate this every time and spend
>> > some time investigating this.
>>
>> The new session created message should be send to (and then processed
>> on) every node *except* the node on which the session was originally
>> created.
>>
>> Can you show us how you configured this cluster please?
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
