[
https://issues.apache.org/jira/browse/NIFI-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-6589:
-----------------------------
Fix Version/s: 1.10.0
Status: Patch Available (was: Open)
> Leader Election should cache results obtained from ZooKeeper
> ------------------------------------------------------------
>
> Key: NIFI-6589
> URL: https://issues.apache.org/jira/browse/NIFI-6589
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Minor
> Fix For: 1.10.0
>
>
> In order to determine which node in a cluster is the Cluster Coordinator, a
> node must make a request to ZooKeeper. That means that if we have N nodes in
> a cluster, then we must ask ZooKeeper for each request at least (N+1) times
> (and no more than N+2) who is the Cluster Coordinator. This is done because
> when the request comes in, the node must determine whether or not it is the
> Cluster Coordinator. If so, it must replicate the request to each node. If
> not, it must forward the request to the Cluster Coordinator, which will then
> do so. When the request is replicated, it will again check if it is the
> cluster coordinator. If we instead cache the result of querying ZooKeeper for
> a short period of time, say 1 minute, we can dramatically decrease the number
> of times that we hit ZooKeeper. If the Coordinator / Primary Node changes in
> the mean time, it should still be notified of the change asynchronously.
> The polling is done currently because we've seen situations where the
> asynchronous notification did not happen. But if we update the code so that
> we cache the results, this means that we will also update the code for
> caching results of which node is Primary Node. This is a benefit as well,
> because currently we don't poll for this and as a result, if we do happen to
> miss the notification, we could theoretically have 2 nodes running processors
> should only run on Primary Node.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
