I see this exception and the servers not doing anything. java.io.IOException: Channel eof at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:630) ERROR - 124554051584(higestZxid) > 21477836646(next log) for type -11 WARN - Sending snapshot last zxid of peer is 0xe00000000 zxid of leader is 0x1e00000000 WARN - Sending snapshot last zxid of peer is 0x1800000000 zxid of leader is 0x1e00000000 WARN - Sending snapshot last zxid of peer is 0x5002dc766 zxid of leader is 0x1e00000000 WARN - Sending snapshot last zxid of peer is 0x1c00000000 zxid of leader is 0x1e00000000 ERROR - Unexpected exception causing shutdown while sock still open java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:78) at java.io.DataOutputStream.writeInt(DataOutputStream.java:180) at org.apache.jute.BinaryOutputArchive.writeInt(BinaryOutputArchive.java:55) at org.apache.zookeeper.data.StatPersisted.serialize(StatPersisted.java:116) at org.apache.zookeeper.server.DataNode.serialize(DataNode.java:167) at org.apache.jute.BinaryOutputArchive.writeRecord(BinaryOutputArchive.java:123) at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:967) at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:982) at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:982) at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:982) at org.apache.zookeeper.server.DataTree.serialize(DataTree.java:1031) at org.apache.zookeeper.server.util.SerializeUtils.serializeSnapshot(SerializeUtils.java:104) at org.apache.zookeeper.server.ZKDatabase.serializeSnapshot(ZKDatabase.java:426) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:331) WARN - ******* GOODBYE /10.138.34.212:33272 ********
Avinash