On Monday, I stood up a cluster with the same configuration as was done
successfully in 1.0.0 a three-server NiFi cluster. Before I started the
cluster, I cleaned out all zookeeper state and data from the old cluster,
but kept the same flow intact, connected to Kafka to pull data from a topic.
This was a performance environment, and there was heavy load on that kafka
topic, so it was immediately busy.
My strong belief is that, due to the volume of data that the flow needed to
process during the election process, the election of a coordinator never
occurred, and, to this day, each node remains disconnected from the others,
although they are running independently.
Could this be a defect in NiFi or Zookeeper? What would you suggest that I
do to resolve this issue?
All servers in the cluster are configured in the following manner:
nifi.properties:
nifi.state.management.embedded.zookeeper.start=true
nifi.cluster.is.node=true
nifi.cluster.node.address=server1
nifi.zookeeper.connect.string=server1:2181,server2:2181,server3:2181
zookeeper.properties:
server.1=server1:2888:3888
server.2=server2:2888:3888
server.3=server3:2888:3888
state-management.xml:
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect
String">server1:2181,server2:2181,server3:2181</property>
<property name="Root Node">/nifi</property>
<property name="Session Timeout">10 seconds</property>
<property name="Access Control">Open</property>
</cluster-provider>
Let me know if you need additional information, please.
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Upgrade-from-1-0-0-to-1-1-1-cluster-config-under-heavy-load-nodes-do-not-connect-tp14523.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
