Is anybody running 3.4 branch ZooKeeper on Linux with iptables? We are running 3.4.6 and have run into conntrack silently expiring the leader election connections after they are idle for 5 days. (/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established) We then see leader election on some machines sometimes gets stuck for 15 minutes or so, until the TCP socket times-out.
This JIRA seems to fix this but only in 3.5 branch https://issues.apache.org/jira/browse/ZOOKEEPER-1748 Does 3.4.8 make a difference to this issue? If not then this scenario does not seem rare - perhaps it is something to add to the wiki? (Will be happy to do it)
