In theory, the 3.4 client can connect / work with the 3.6 server without any problem. I am not sure how Curator complicates this issue, but based on the logs I don't see any ZooKeeper issue just yet.
The logs you pasted here shows that the Storm/Curator node closed the ZooKeeper client session, maybe due to some timeout. The ZooKeeper Client will terminate the session if the 2/3 time of SessionTimeout elapsed, then it tries to reconnect (I guess to an other ZooKeeper node). The "Revalidating client" appears because of the reconnection. Maybe the ZooKeeper server is overloaded / slow to respond? Or maybe the negotiated session timeout is too low? Can you see the session timeout for these sessions (e.g. for 0x200866346830367) in the Storm logs? Or can you see anything else suspicious in the ZooKeeper client logs (aka. Storm/Curator logs) around this time? On Fri, May 15, 2020 at 1:22 AM ashish soni <[email protected]> wrote: > An old issue for reference: > https://issues.apache.org/jira/browse/ZOOKEEPER-1582 > > On Thu, May 14, 2020 at 4:20 PM ashish soni <[email protected]> > wrote: > > > Hi, > > > > I am running a containerized zookeeper 3.6.1 version and it is getting > > filled with below WARN logs, > > > > [myid:2] - WARN [NIOWorkerThread-2:NIOServerCnxn@364] - Unexpected > > exception EndOfStreamException: Unable to read additional data from > client, > > it probably closed the socket: address = /10.162.53.62:48988, session = > > 0x200866346830367 at > > > org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:163) > > at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:326) > > at > > > org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522) > > at > > > org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154) > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > [myid:2] - INFO [NIOWorkerThread-1:Learner@137] - Revalidating client: > > 0x3008663464f03ac [myid:2] - INFO [NIOWorkerThread-1:Learner@137] - > > Revalidating client: 0x100866346320382 INFO > [NIOWorkerThread-1:Learner@137] > > - Revalidating client: 0x1008663463202d9 INFO > [NIOWorkerThread-1:Learner@137] > > - Revalidating client: 0x1008663463203b2 INFO > [NIOWorkerThread-1:Learner@137] > > - Revalidating client: 0x2008663468302d5 INFO > [NIOWorkerThread-1:Learner@137] > > - Revalidating client: 0x4008663438f03ca INFO > [NIOWorkerThread-1:Learner@137] > > - Revalidating client: 0x1008663463203bf and so on > > > > I have multiple instances of Apache Storm 1.22 running as containerized > > service which is running on 10.162.53.62 (IP address in the above WARN > > logs). The storm is using a 3.4.14 zookeeper version and the curator is > at > > 4.0.1 (which is compatible with 3.5+ versions). Here is the pool file > from > > the storm 1.1.x branch > > https://github.com/apache/storm/blob/1.1.x-branch/pom.xml for reference. > > Even the latest storm is using the same configs as far as the zookeeper > is > > concerned. > > > > So, is something wrong with the compatibility of the storm's zookeeper > > version and mine? Or is anything missing from my zookeeper configs point > of > > view? Any details on what might be happening would really be helpful. > > > > Here is my zoo.cfg, > > > > tickTime=2000 > > # The number of ticks that the initial > > # synchronization phase can take > > initLimit=20 > > # The number of ticks that can pass between > > # sending a request and getting an acknowledgement > > syncLimit=15 > > # HA (multi-node) setup > > standaloneEnabled=false > > dynamicConfigFile=/opt/zookeeper/conf/zoo.cfg.dynamic > > > > # location of in-memory database snapshots > > dataDir=/data > > # location of the transaction-log > > dataLogDir=/data/xnlogs > > > > # enable regular purging of old data > > autopurge.purgeInterval=24 > > autopurge.snapRetainCount=5 > > > > >
