[
https://issues.apache.org/jira/browse/NIFI-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022753#comment-16022753
]
ASF GitHub Bot commented on NIFI-3681:
--------------------------------------
Github user josephxsxn commented on the issue:
https://github.com/apache/nifi/pull/1800
1) I'll look into taking the inner classes out but will need to look at how
the permissions scopes are working between them first.
2) Will look into applying the KeeperException recommendation to clean the
code up some.
3) Will fix versions
4) I agree, I got tired of fighting maven's arch type generator. Will
repackage things to facilitate a generic election API like the jira kinda talks
about.
5) Plan is to make the GetEventHubProcessor use this as it has some major
issues in clustered deployments with duplication of data. Unlike Kafka which
uses a consumer group to only allow 1 reader per partition. In EventHub your
allowed 5 readers per partition, so we need a method to discover alive workers,
elect the leader, have the leader assign partition work into the shared cache,
and then run.
> Controller Service for Processor Leader Elections
> --------------------------------------------------
>
> Key: NIFI-3681
> URL: https://issues.apache.org/jira/browse/NIFI-3681
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Joseph Niemiec
> Assignee: Joseph Niemiec
>
> Today I find a need for NiFi Cluster to allow a specific set of processors to
> perform a 'LeaderElection' of sorts to allow for a single processor to
> update the process shared cluster state with assignments (both initial and
> recovery.) The CS would be responsible for joining a Zookeeper group, the
> logic itself, performing new elections should an leader die, etc...
> At its core I envision a simple API provided by the CS
> * String - whoIsLeader
> * List[String] - aliveElectors
> * Long- LastElectionEpoch ? - Not sure about this, but would it be good to
> detect if an election occurred and the leader did not change but the election
> occured. Maybe a UUID-4?
> While thinking ZK is best as Clustered NiFi already requires it would there
> be value in implementing a standalone RAFT or PAXOS that runs at the cluster
> level?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
