Re: Clustering problems
Read the doc and configure greater waitForAck timeouts and more pooled worker. Sender ... ackTimeout=5 maxPoolSocketLimit=40 .. I only used fastasyncqueue mode in my production system! Sender className=org.apache.catalina.cluster.tcp.ReplicationTransmitter replicationMode=fastasyncqueue doTransmitterProcessingStats=true doProcessingStats=true doWaitAckStats=true queueTimeWait=true queueDoStats=true queueCheckLock=true waitForAck=false keepAliveTimeout=8 keepAliveMaxRequestCount=-1/ Peter Randy Paries schrieb: Hello I have two tomcat servers (flanders and krusty) Their server.xmls are below they have two nic cards. The incoming http requests come in via the 192.168 ips and go out the real ips 66.208 the 192.168 nics are on a gigbit network When my servers get really busy i messages in the log file (see below) Once this happens my httpd starts backing up and it gets real ugly fast. I have to stop both instances and restart them Does my config look ok? any suggestions? Thanks Randy From log file Aug 25, 2005 3:22:36 PM org.apache.catalina.cluster.tcp.SimpleTcpCluster memberDisappeared INFO: Received member disappeared:org.apache.catalina.cluster.mcast.McastMember[tcp://192.168.0.20 4:4001,192.168.0.204,4001, a live=47087979] Aug 25, 2005 3:22:37 PM org.apache.catalina.cluster.tcp.SimpleTcpCluster memberAdded INFO: Replication member added:org.apache.catalina.cluster.mcast.McastMember[tcp://192.168.0.204:4001 ,192.168.0.204,4001, aliv e=47092679] Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting socket, and trying ag ain. Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.ReplicationTransmitter sendMessageData WARNING: Unable to send replicated message, is server down? java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at org.apache.catalina.cluster.tcp.SocketSender.waitForAck(SocketSender.java:13 4) at org.apache.catalina.cluster.tcp.SocketSender.sendMessage(SocketSender.java:1 25) at org.apache.catalina.cluster.tcp.PooledSocketSender.sendMessage(PooledSocketS ender.java:119) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessageData(Repli cationTransmitter.java:117) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessage(Replicati onTransmitter.java:149) at org.apache.catalina.cluster.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java: 409) at org.apache.catalina.cluster.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java: 417) at org.apache.catalina.cluster.tcp.ReplicationValve.invoke(ReplicationValve.jav a:202) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContex t.java:102) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :109) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContex t.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:296) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:372) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:694) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:626) at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:807) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav a:644) at java.lang.Thread.run(Thread.java:534) Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting socket, and trying ag ain. Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting
Clustering problems
Hello I have two tomcat servers (flanders and krusty) Their server.xmls are below they have two nic cards. The incoming http requests come in via the 192.168 ips and go out the real ips 66.208 the 192.168 nics are on a gigbit network When my servers get really busy i messages in the log file (see below) Once this happens my httpd starts backing up and it gets real ugly fast. I have to stop both instances and restart them Does my config look ok? any suggestions? Thanks Randy From log file Aug 25, 2005 3:22:36 PM org.apache.catalina.cluster.tcp.SimpleTcpCluster memberDisappeared INFO: Received member disappeared:org.apache.catalina.cluster.mcast.McastMember[tcp://192.168.0.20 4:4001,192.168.0.204,4001, a live=47087979] Aug 25, 2005 3:22:37 PM org.apache.catalina.cluster.tcp.SimpleTcpCluster memberAdded INFO: Replication member added:org.apache.catalina.cluster.mcast.McastMember[tcp://192.168.0.204:4001 ,192.168.0.204,4001, aliv e=47092679] Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting socket, and trying ag ain. Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.ReplicationTransmitter sendMessageData WARNING: Unable to send replicated message, is server down? java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at org.apache.catalina.cluster.tcp.SocketSender.waitForAck(SocketSender.java:13 4) at org.apache.catalina.cluster.tcp.SocketSender.sendMessage(SocketSender.java:1 25) at org.apache.catalina.cluster.tcp.PooledSocketSender.sendMessage(PooledSocketS ender.java:119) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessageData(Repli cationTransmitter.java:117) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessage(Replicati onTransmitter.java:149) at org.apache.catalina.cluster.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java: 409) at org.apache.catalina.cluster.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java: 417) at org.apache.catalina.cluster.tcp.ReplicationValve.invoke(ReplicationValve.jav a:202) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContex t.java:102) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :109) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContex t.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:296) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:372) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:694) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:626) at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:807) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav a:644) at java.lang.Thread.run(Thread.java:534) Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting socket, and trying ag ain. Aug 25, 2005 3:22:51 PM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/192.168.0.204:4001] in 15000 ms. Disconnecting socket, and trying ag ain. Flanders - 192.168.0.203 Cluster className=org.apache.catalina.cluster.tcp.SimpleTcpCluster managerClassName=org.apache.catalina.cluster.session.DeltaManager expireSessionsOnShutdown=false useDirtyFlag=true Membership className=org.apache.catalina.cluster.mcast.McastService mcastAddr=228.0.0.9 mcastPort=45564 mcastFrequency=500 mcastDropTime=3000/ Receiver className=org.apache.catalina.cluster.tcp.ReplicationListener tcpListenAddress=192.168.0.203 tcpListenPort=4001 tcpSelectorTimeout=100 tcpThreadCount=6/
Clustering problems during session replication at startup
I am having some problems with clustering (version 5.0.27) and session replication at startup across multiple servers. I can replicate in normal conditions, but introducting servers to the cluster after a period of time causes Unable to send replicated message. This error only seems to occur when you have more than 50 sessions stored (session size = 50k). I've read some documents regarding the NTPL thread library which i have tried to no avail, has anyone else had a similar problem, or have any advise on the issue ? Here is a list of some important settings, i've tried to keep most of them default. Redhat ES3 using JDK 1.4.2_05 with LD_ASSUME_KERNEL=2.4.0 ClusterclassName=org.apache.catalina.cluster.tcp.SimpleTcpCluster expireSessionsOnShutdown=false useDirtyFlag=true printSessionInfo=true tcpKeepAliveTime=0 synchronousReplication=true Membership mcastFrequency=500 mcastDropTime=1/ Receiver tcpSelectorTimeout=100 tcpThreadCount=2/ Errors (ip addresses masked) Aug 24, 2004 11:25:00 AM org.apache.catalina.cluster.session.DeltaManager start INFO: Starting clustering manager...:/webapp Aug 24, 2004 11:25:15 AM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/*.*.*.*:4001] in 15000 ms. Disconnecting socket, and trying again. Aug 24, 2004 11:25:30 AM org.apache.catalina.cluster.tcp.SocketSender waitForAck WARNING: Wasn't able to read acknowledgement from server[/*.*.*.*:4001] in 15000 ms. Disconnecting socket, and trying again. Aug 24, 2004 11:25:30 AM org.apache.catalina.cluster.tcp.ReplicationTransmitter sendMessageData WARNING: Unable to send replicated message, is server down? java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at org.apache.catalina.cluster.tcp.SocketSender.waitForAck(SocketSender.java:13 4) at org.apache.catalina.cluster.tcp.SocketSender.sendMessage(SocketSender.java:1 25) at org.apache.catalina.cluster.tcp.PooledSocketSender.sendMessage(PooledSocketS ender.java:119) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessageData(Repli cationTransmitter.java:117) at org.apache.catalina.cluster.tcp.ReplicationTransmitter.sendMessage(Replicati onTransmitter.java:136) at org.apache.catalina.cluster.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java: 457) at org.apache.catalina.cluster.session.DeltaManager.start(DeltaManager.java:648 ) at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:499) at org.apache.catalina.startup.ContextConfig.managerConfig(ContextConfig.java:3 08) at org.apache.catalina.startup.ContextConfig.start(ContextConfig.java:635) at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java: 216) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSuppor t.java:119) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4290) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:8 23) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:807) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:595) at org.apache.catalina.core.StandardHostDeployer.addChild(StandardHostDeployer. java:903) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 ) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:25) at java.lang.reflect.Method.invoke(Method.java:324) at org.apache.commons.beanutils.MethodUtils.invokeMethod(MethodUtils.java:252) at org.apache.commons.digester.SetNextRule.end(SetNextRule.java:256) at org.apache.commons.digester.Rule.end(Rule.java:276) at org.apache.commons.digester.Digester.endElement(Digester.java:1058) at org.apache.catalina.util.CatalinaDigester.endElement(CatalinaDigester.java:7 6) at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source) at org.apache.xerces.impl.dtd.XMLDTDValidator.emptyElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unkno wn Source) at org.apache.xerces.impl.XMLDocumentScannerImpl$ContentDispatcher.scanRootElem entHook(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatc her.dispatch(Unknown Source) at
