Hello, I have a 5 node zookeeper (+ kafka) cluster. I am trying to find out if the inter node connection behaviour is normal. I should have 1 zookeeper leader and 4 followers, and I expect that all 4 followers would open connections exclusively to the leader. So that "echo stat | nc localhost 2181" on the leader should show me 6 connections: 4 from the followers and 2 from itself (one via proper ip, one from localhost)
Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /host02:48610[1](queued=0,recved=1455831,sent=1455833) /host03:36244[1](queued=0,recved=1459769,sent=1459793) /host05:53680[1](queued=0,recved=484,sent=484) /host01:48978[1](queued=0,recved=226,sent=226) /host04:44810[1](queued=0,recved=52,sent=52) /127.0.0.1:41434[0](queued=0,recved=1,sent=0) But instead I have ended up with a few connections on each node, including the followers (as seen in below output). Some followers are even connecting to themselves? The cluster looks stable, with only 1 leader and 4 followers, but I just do not understand this connection behaviour and would like to understand if this is normal behaviour, mis configuration, a bug or something else. user@host01:~$<mailto:jason.grammenos@ca02vleventstream01:~$> echo stat | nc localhost 2181 Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /host01:50768[1](queued=0,recved=6156,sent=6156) /127.0.0.1:35508[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/3 Received: 9855 Sent: 9854 Connections: 2 Outstanding: 0 Zxid: 0xb00000068 Mode: follower Node count: 226 user@host02:~$<mailto:jason.grammenos@ca02vleventstream02:~$> echo stat | nc localhost 2181 Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /127.0.0.1:49624[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/0 Received: 3699 Sent: 3698 Connections: 1 Outstanding: 0 Zxid: 0xb00000068 Mode: follower Node count: 226 usaer@host03:~$<mailto:jason.grammenos@ca02vleventstream03:~$> echo stat | nc localhost 2181 Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /host03:50674[1](queued=0,recved=6136,sent=6136) /127.0.0.1:54554[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/22 Received: 9795 Sent: 9794 Connections: 2 Outstanding: 0 Zxid: 0xb00000068 Mode: follower Node count: 226 user@host04:~$<mailto:jason.grammenos@ca04vleventstream04:~$> echo stat | nc localhost 2181 Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /host02:42874[1](queued=0,recved=143,sent=143) /host04:38222[1](queued=0,recved=89,sent=89) /127.0.0.1:58796[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/27 Received: 287 Sent: 286 Connections: 3 Outstanding: 0 Zxid: 0xb00000056 Mode: follower Node count: 226 user@host05:~$<mailto:jason.grammenos@ca04vleventstream05:~$> echo stat | nc localhost 2181 Zookeeper version: 3.5.8-f439ca583e70862c3068a1f2a7d4d068eec33315, built on 05/04/2020 15:53 GMT Clients: /127.0.0.1:38680[0](queued=0,recved=1,sent=0) /host01:41198[1](queued=0,recved=6339,sent=6340) Latency min/avg/max: 0/0/8 Received: 10041 Sent: 10041 Connections: 2 Outstanding: 0 Zxid: 0xb00000068 Mode: leader Node count: 226 Proposal sizes last/min/max: 220/36/392 Thank you Jason [cid:image001.png@01D82885.92BDE6F0] Jason Grammenos | Operations and Infrastructure Analyst Pronouns: he/him P: 613.232.7797 x1131 Toll-free: 866.545.3745 x1131 jason.gramme...@agilitypr.com agilitypr.com<https://agilitypr.com/> Learn new PR tips from our free resources<https://www.agilitypr.com/resources/>.
