[
https://issues.apache.org/jira/browse/ZOOKEEPER-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072876#comment-17072876
]
Philipp edited comment on ZOOKEEPER-3776 at 4/1/20, 3:40 PM:
-------------------------------------------------------------
Hello.
With 3.4.8 I configured this way (which did work):
clientPort=2181
clientPortAddress=0.0.0.0
server.1=ip1:2888:3888
server.2=ip2:2888:3888
server.3=ip3:2888:3888
With 3.6.0 I deleted clientPort and clientPortAddress and use
dynamicConfigFile, in which I put:
server.1=ip1:2888:3888:participant;0.0.0.0:2181
server.2=ip2:2888:3888:participant;0.0.0.0:2181
server.3=ip3:2888:3888:participant;0.0.0.0:2181
Changing it to e.g. {{server.1=ip1:2888:3888:participant;ip1:2181}} doesn't
help. It looks not as a fully same issue to me...
These machines are in AWS EC2; Zk running plain by systemd, no containers, no
networking abstractions.
Ubuntu 18.04.2, uname -a: {{Linux somehost 4.15.0-1032-aws #34-Ubuntu SMP Thu
Jan 17 15:18:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux}}
Oracle server jre 1.8.0 u112
Zk configs are identical:
dataDir=/somepath
dataLogDir=/somepath
tickTime=3000
maxClientCnxns=0
autopurge.purgeInterval=1
autopurge.snapRetainCount=10
zookeeper.extendedTypesEnabled=true
snapshot.trust.empty=false
initLimit=40
syncLimit=40
standaloneEnabled=false
reconfigEnabled=true
dynamicConfigFile=/somepath/zookeeper.dynamic
skipACL=yes
tcpKeepAlive=true
electionPortBindRetry=10
admin.enableServer=true
admin.serverAddress=127.0.0.1
admin.serverPort=8080
/somepath/zookeeper.dynamic (identical too):
server.1=ip1:2888:3888:participant;ip1:2181
server.2=ip2:2888:3888:participant;ip2:2181
server.3=ip3:2888:3888:participant;ip3:2181
If it's not enough, I will try to gather debug logs some time later.
was (Author: fsfeel):
Hello.
With 3.4.8 I configured this way (which did work):
clientPort=2181
clientPortAddress=0.0.0.0
server.1=ip1:2888:3888
server.2=ip2:2888:3888
server.3=ip3:2888:3888
With 3.6.0 I deleted clientPort and clientPortAddress and use
dynamicConfigFile, in which I put:
server.1=ip1:2888:3888:participant;0.0.0.0:2181
server.2=ip2:2888:3888:participant;0.0.0.0:2181
server.3=ip3:2888:3888:participant;0.0.0.0:2181
Changing it to e.g. {{server.1=ip1:2888:3888:participant;ip1:2181}} doesn't
help. It looks not as a fully same issue to me...
These machines are in AWS EC2; Zk running plain by systemd, no containers, no
networking abstractions.
Ubuntu 18.04.2, uname -a: {{Linux somehost 4.15.0-1032-aws #34-Ubuntu SMP Thu
Jan 17 15:18:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux}}
Oracle server jre 1.8.0 u112
Zk configs are identical:
dataDir=/somepath
dataLogDir=/somepath
tickTime=3000
maxClientCnxns=0
autopurge.purgeInterval=1
autopurge.snapRetainCount=10
zookeeper.extendedTypesEnabled=true
snapshot.trust.empty=false
initLimit=40
syncLimit=40
standaloneEnabled=false
reconfigEnabled=true
dynamicConfigFile=/somepath/zookeeper.dynamic
skipACL=yes
tcpKeepAlive=true
electionPortBindRetry=10
admin.enableServer=true
admin.serverAddress=127.0.0.1
admin.serverPort=8080
/somepath/zookeeper.dynamic (identical too):
server.1=ip1:2888:3888:participant;ip1:2181
server.2=ip2:2888:3888:participant;ip2:2181
server.3=ip3:2888:3888:participant;ip3:2181
If it's not enough, I will try to gather debug logs some time later.
> Cluster stuck not forming up quorum
> ------------------------------------
>
> Key: ZOOKEEPER-3776
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3776
> Project: ZooKeeper
> Issue Type: Bug
> Reporter: Philipp
> Assignee: Mate Szalay-Beko
> Priority: Major
>
> We had working Zookeeper 3.4.8 that came with Kafka. 3 machines cluster. Now
> I'm trying to upgrade it to 3.6.0. I didn't touch the data directory, just
> set up 3.6.0 Zookeeper in the new directory and updated config. Stopped all
> Zookeepers. Now I've started eache Zk in turn with some delay. And they are
> not forming up quorum, stuck seemingly forever, sometimes trying to elect and
> sometimes throwing nonsensical errors.
> For example, on node 1 (sorry don't know how to format this properly in jira):
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,133 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,133 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,134 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,335 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@857] -
> Peer state changed: following
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1453] -
> FOLLOWING
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1246]
> - minSessionTimeout set to 6000
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1255]
> - maxSessionTimeout set to 60000
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ResponseCache@45] -
> Response cache size is initialized with value 400.
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@111]
> - zookeeper.pathStats.slotCapacity = 60
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@112]
> - zookeeper.pathStats.slotDuration = 15
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@113]
> - zookeeper.pathStats.maxDepth = 6
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,536 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@114]
> - zookeeper.pathStats.initialDelay = 5
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@115]
> - zookeeper.pathStats.delay = 5
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@116]
> - zookeeper.pathStats.enabled = false
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1470]
> - The max bytes for all large requests are set to 104857600
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1484]
> - The large request threshold is set to -1
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@329]
> - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout
> 60000 clientPortListenBacklog -1 datadir
> /kafka/kafka_zookeeper_data/logdir/version-2 snapdir
> /kafka/kafka_zookeeper_data/version-2
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):Follower@75] -
> FOLLOWING - LEADER ELECTION TOOK - 92790 MS
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,537 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@863] -
> Peer state changed: following - discovery
> Mar 31 20:05:21 java[24055]: 2020-03-31 20:05:21,736 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:FOLLOWING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,538 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:FOLLOWING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - WARN
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):Follower@129] -
> Exception when following the leader
> Mar 31 20:05:22 java[24055]: java.lang.IllegalArgumentException
> Mar 31 20:05:22 java[24055]: at
> java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1307)
> Mar 31 20:05:22 java[24055]: at
> java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1195)
> Mar 31 20:05:22 java[24055]: at
> java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89)
> Mar 31 20:05:22 java[24055]: at
> org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:275)
> Mar 31 20:05:22 java[24055]: at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:87)
> Mar 31 20:05:22 java[24055]: at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1455)
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):Follower@292] -
> shutdown Follower
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@863] -
> Peer state changed: looking
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - WARN
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1501] -
> PeerState set to LOOKING
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1371] -
> LOOKING
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@931]
> - New election. My id = 1, proposed zxid=0x10002b653
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,539 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,542 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,544 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LEADING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,744 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,744 [myid:1] - INFO
> [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@966]
> - Notification time out: 400
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,745 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LEADING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[24055]: 2020-03-31 20:05:22,746 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0{code}
> ----------
> On node 2 in the same time:
> Mar 31 20:04:39 java[23556]: 2020-03-31 20:04:39,764 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,130 [myid:2] - WARN
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1471] -
> Unexpected exception
> Mar 31 20:05:21 java[23556]: java.lang.InterruptedException: Timeout while
> waiting for epoch from quorum
> Mar 31 20:05:21 java[23556]: at
> org.apache.zookeeper.server.quorum.Leader.getEpochToPropose(Leader.java:1425)
> Mar 31 20:05:21 java[23556]: at
> org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:599)
> Mar 31 20:05:21 java[23556]: at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1468)
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):Leader@792] -
> Shutting down
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):Leader@798] -
> Shutdown called. For the reason Forcing shutdown
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@863] -
> Peer state changed: looking
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - WARN
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1501] -
> PeerState set to LOOKING
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1371] -
> LOOKING
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - WARN
> [LearnerCnxAcceptorHandler-/172.30.2.252:2888:Leader$LearnerCnxAcceptor$LearnerCnxAcceptorHandler@523]
> - Exception while shutting down acceptor.
> Mar 31 20:05:21 java[23556]: java.net.SocketException: Socket closed
> Mar 31 20:05:21 java[23556]: at
> java.net.PlainSocketImpl.socketAccept(Native Method)
> Mar 31 20:05:21 java[23556]: at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
> Mar 31 20:05:21 java[23556]: at
> java.net.ServerSocket.implAccept(ServerSocket.java:545)
> Mar 31 20:05:21 java[23556]: at
> java.net.ServerSocket.accept(ServerSocket.java:513)
> Mar 31 20:05:21 java[23556]: at
> org.apache.zookeeper.server.quorum.Leader$LearnerCnxAcceptor$LearnerCnxAcceptorHandler.acceptConnections(Leader.java:510)
> Mar 31 20:05:21 java[23556]: at
> org.apache.zookeeper.server.quorum.Leader$LearnerCnxAcceptor$LearnerCnxAcceptorHandler.run(Leader.java:493)
> Mar 31 20:05:21 java[23556]: at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> Mar 31 20:05:21 java[23556]: at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Mar 31 20:05:21 java[23556]: at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> Mar 31 20:05:21 java[23556]: at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> Mar 31 20:05:21 java[23556]: at java.lang.Thread.run(Thread.java:745)
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,131 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@931]
> - New election. My id = 2, proposed zxid=0x10002b653
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,133 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,134 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,134 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,335 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@966]
> - Notification time out: 400
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,335 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,336 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,736 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@966]
> - Notification time out: 800
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,736 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,737 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:FOLLOWING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[23556]: 2020-03-31 20:05:21,737 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,538 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@966]
> - Notification time out: 1600
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,538 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LOOKING; n.sid:1, n.state:FOLLOWING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@857] -
> Peer state changed: leading
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@1465] -
> LEADING
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1246]
> - minSessionTimeout set to 6000
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1255]
> - maxSessionTimeout set to 60000
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ResponseCache@45] -
> Response cache size is initialized with value 400.
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@111]
> - zookeeper.pathStats.slotCapacity = 60
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@112]
> - zookeeper.pathStats.slotDuration = 15
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,539 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@113]
> - zookeeper.pathStats.maxDepth = 6
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,540 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,540 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@114]
> - zookeeper.pathStats.initialDelay = 5
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@115]
> - zookeeper.pathStats.delay = 5
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):RequestPathMetricsCollector@116]
> - zookeeper.pathStats.enabled = false
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1470]
> - The max bytes for all large requests are set to 104857600
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@1484]
> - The large request threshold is set to -1
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@329]
> - Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout
> 60000 clientPortListenBacklog -1 datadir
> /kafka/kafka_zookeeper_data/logdir/version-2 snapdir
> /kafka/kafka_zookeeper_data/version-2
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):Leader@581] -
> LEADING - LEADER ELECTION TOOK - 1410 MS
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):QuorumPeer@863] -
> Peer state changed: leading - discovery
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,541 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):FileTxnSnapLog@470]
> - Snapshotting: 0x10002b653 to
> /kafka/kafka_zookeeper_data/version-2/snapshot.10002b653
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,544 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,544 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LEADING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,544 [myid:2] - INFO
> [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:3, n.state:LEADING, n.leader:3,
> n.round:0x4a, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[23556]: 2020-03-31 20:05:22,568 [myid:2] - INFO
> [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):ZooKeeperServer@519]
> - Snapshot taken in 26 ms
> ----------------------
> node 3:
> Mar 31 20:05:21 java[20867]: 2020-03-31 20:05:21,133 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[20867]: 2020-03-31 20:05:21,134 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[20867]: 2020-03-31 20:05:21,335 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:21 java[20867]: 2020-03-31 20:05:21,736 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[20867]: 2020-03-31 20:05:22,540 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[20867]: 2020-03-31 20:05:22,544 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:2, n.state:LOOKING, n.leader:2,
> n.round:0x4b, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:22 java[20867]: 2020-03-31 20:05:22,745 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
> Mar 31 20:05:23 java[20867]: 2020-03-31 20:05:23,147 [myid:3] - INFO
> [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@376] -
> Notification: my state:LEADING; n.sid:1, n.state:LOOKING, n.leader:1,
> n.round:0x4c, n.peerEpoch:0x1, n.zxid:0x10002b653, message format
> version:0x2, n.config version:0x0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
