Hi, I have an exception when one on my Tomcat nodes restarts. The session are not synced and user complain about being logged out. I was running 6.0.32 and had the problem. Now I'm running a custom build of 6.0.33 from yesterday. I added a little code which prints ' SEVERE: name=user'. The key on which the sync fails while reading the value.
I looked into the java code in ObjectInputStream and it looks like some inconsistency in the ObjectStream. I have 4 nodes on Debian 5, sun-java6-jdk 6-22-0lenny1, seen the problem on Tomcat 6.0.29, 6.0.32 and a custom build 6.0.33. I can't reproduce this (yet) on my test environment, so debugging it is difficult without upset customers. <Engine name="Catalina" defaultHost="xxx.xxx.xxx"> <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" /> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" port="47727" /> </Channel> </Cluster> <Host name="xxx.xxx.xxx" appBase="/data/webapps/crm-LIVE/deployed" unpackWARs="true" autoDeploy="false" xmlValidation="false" xmlNamespaceAware="true"> </Host> </Engine> Can I provide more information? Where should I look. NB: In some previous mails about another problem I thought that might be the reason for my logout problem, that is why I tried 6.0.33-dev. Ronald. Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaManager getAllClusterSessions WARNING: Manager [crm.realworks.nl#], requesting session state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 10, 110}:4000,{10, 0, 10, 110},4000, alive=45370885,id={-34 112 102 -93 -87 -88 77 18 -113 -30 62 8 62 -65 -112 -13 }, payload={}, command={}, domain={}, ]. This operation will timeout if no session state has been received within 60 seconds. Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaManager waitForSendAllSessions INFO: Manager [crm.realworks.nl#]; session state send at 6/23/11 5:49 PM received in 304 ms. Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaManager getAllClusterSessions WARNING: Manager [crm.realworks.nl#]: Drop message SESSION-ACCESSED inside GET_ALL_SESSIONS sync phase start date 6/23/11 5:49 PM message date 6/23/11 5:49 PM Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaSession readObject SEVERE: name=user. Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaManager deserializeSessions SEVERE: IOException while loading persisted sessions: java.io.OptionalDataException java.io.OptionalDataException at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at java.util.HashMap.readObject(HashMap.java:1029) at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at org.apache.catalina.ha.session.DeltaSession.readObject(DeltaSession.java:655) at org.apache.catalina.ha.session.DeltaSession.readObjectData(DeltaSession.java:481) at org.apache.catalina.ha.session.DeltaManager.deserializeSessions(DeltaManager.java:745) at org.apache.catalina.ha.session.DeltaManager.handleALL_SESSION_DATA(DeltaManager.java:1583) at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1440) at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:1173) at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:92) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:901) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:882) at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:269) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailureDetector.java:110) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:241) at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:225) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:188) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:91) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Jun 23, 2011 5:49:47 PM org.apache.catalina.ha.session.DeltaManager messageReceived SEVERE: Manager [crm.realworks.nl#]: Unable to receive message through TCP channel java.io.OptionalDataException at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at java.util.HashMap.readObject(HashMap.java:1029) at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) at org.apache.catalina.ha.session.DeltaSession.readObject(DeltaSession.java:655) at org.apache.catalina.ha.session.DeltaSession.readObjectData(DeltaSession.java:481) at org.apache.catalina.ha.session.DeltaManager.deserializeSessions(DeltaManager.java:745) at org.apache.catalina.ha.session.DeltaManager.handleALL_SESSION_DATA(DeltaManager.java:1583) at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1440) at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:1173) at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:92) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:901) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:882) at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:269) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailureDetector.java:110) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79) at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:241) at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:225) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:188) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:91) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)