1. Turn off your firewall or do #sudo iptables --list-rules 2. netstat -nlutp to see if ZK is listening on that port Issue is there is something blocking access to the port.
+1.314.452.2896 (Tel/SMS) On Thu, Aug 16, 2018 at 8:15 PM, Raghav <raghavas...@gmail.com> wrote: > 1. Ids/Nodes are correct and their is no problem, > 2. ZooKeeper started on all three nodes within a span of 5 seconds. > 3. Our cluster is not on AWS or on any cloud for that matter. > > Is there any zookeeper settings that could possibly help in such scenarios > ? > > Thanks > > R > > On Thu, Aug 16, 2018 at 6:12 PM, Dan Simoes <d...@tcell.io> wrote: > > > Ensure ids/nodes are correct in zoo.cfg and zookeeper is running on each. > > Also any changes to ports being open? If it’s aws, beck security groups. > > Node 1 cannot talk to the other two nodes. > > > > > On Aug 16, 2018, at 6:02 PM, Raghav <raghavas...@gmail.com> wrote: > > > > > > Hi > > > > > > Our 3 node Zookeeper ensemble got powered down, and upon powering up > the > > > zookeeper could get quorum and kept throwing these errors. As a result > > our > > > Kafka cluster was unusable. What is the best way to revive ZK cluster > in > > > such situations ? Please suggest. > > > > > > > > > 2018-08-17_00:59:18.87009 2018-08-17 00:59:18,869 [myid:1] - WARN > > > [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@584] - > Cannot > > > open channel to 2 at election address /1.1.1.143:3888 > > > 2018-08-17_00:59:18.87011 java.net.ConnectException: Connection refused > > > 2018-08-17_00:59:18.87011 at > > > java.net.PlainSocketImpl.socketConnect(Native Method) > > > 2018-08-17_00:59:18.87011 at > > > java.net.AbstractPlainSocketImpl.doConnect( > AbstractPlainSocketImpl.java: > > 345) > > > 2018-08-17_00:59:18.87012 at > > > java.net.AbstractPlainSocketImpl.connectToAddress( > > AbstractPlainSocketImpl.java:206) > > > 2018-08-17_00:59:18.87012 at > > > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java: > > 188) > > > 2018-08-17_00:59:18.87013 at > > > java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > 2018-08-17_00:59:18.87013 at java.net.Socket.connect( > > Socket.java:589) > > > 2018-08-17_00:59:18.87013 at > > > org.apache.zookeeper.server.quorum.QuorumCnxManager. > > connectOne(QuorumCnxManager.java:558) > > > 2018-08-17_00:59:18.87014 at > > > org.apache.zookeeper.server.quorum.QuorumCnxManager. > > connectAll(QuorumCnxManager.java:610) > > > 2018-08-17_00:59:18.87014 at > > > org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader( > > FastLeaderElection.java:838) > > > 2018-08-17_00:59:18.87014 at > > > org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957) > > > 2018-08-17_00:59:18.87034 2018-08-17 00:59:18,870 [myid:1] - INFO > > > [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer$QuorumServer@184] > - > > > Resolved hostname: 1.1.1.143 to address: /1.1.1.143 > > > 2018-08-17_00:59:18.87095 2018-08-17 00:59:18,870 [myid:1] - WARN > > > [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@584] - > Cannot > > > open channel to 3 at election address /1.1.1.144:3888 > > > 2018-08-17_00:59:18.87097 java.net.ConnectException: Connection refused > > > 2018-08-17_00:59:18.87097 at > > > java.net.PlainSocketImpl.socketConnect(Native Method) > > > 2018-08-17_00:59:18.87097 at > > > java.net.AbstractPlainSocketImpl.doConnect( > AbstractPlainSocketImpl.java: > > 345) > > > 2018-08-17_00:59:18.87098 at > > > java.net.AbstractPlainSocketImpl.connectToAddress( > > AbstractPlainSocketImpl.java:206) > > > 2018-08-17_00:59:18.87098 at > > > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java: > > 188) > > > 2018-08-17_00:59:18.87098 at > > > java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > > > 2018-08-17_00:59:18.87098 at java.net.Socket.connect( > > Socket.java:589) > > > 2018-08-17_00:59:18.87099 at > > > org.apache.zookeeper.server.quorum.QuorumCnxManager. > > connectOne(QuorumCnxManager.java:558) > > > 2018-08-17_00:59:18.87099 at > > > org.apache.zookeeper.server.quorum.QuorumCnxManager. > > connectAll(QuorumCnxManager.java:610) > > > 2018-08-17_00:59:18.87099 at > > > org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader( > > FastLeaderElection.java:838) > > > 2018-08-17_00:59:18.87099 at > > > org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957) > > > > > > Thanks. > > > > > > R > > > > > > -- > Raghav >
