[
https://issues.apache.org/jira/browse/NIP-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063048#comment-18063048
]
Mark Payne commented on NIP-22:
-------------------------------
I would actually also want to allow periodic redistribution of the load, such
that if you have a 3 node cluster and spin up a new node it gets roughly 25% of
the Single Node work as well.
> "Single Node Only" Execution Node
> ---------------------------------
>
> Key: NIP-22
> URL: https://issues.apache.org/jira/browse/NIP-22
> Project: NiFi Improvement Proposal
> Issue Type: New Feature
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
>
> h3. Motivation
> NiFi supports running a Processor on "All Nodes" or "Primary Node Only." This
> works well when you have a few processors that don't require a large amount
> of overhead. However, some flows have many different processors that have to
> be run as Primary Node Only, and it can cause a single node to be heavily
> overloaded.
> There are times that we want to have a Primary Node because we want multiple
> flows running on the same node. But the vast majority of the time we simply
> care that the processor runs on 1 node only - but we don't care which node.
> To this end, we should introduce the ability to schedule a Processor to run
> on a "Single Node Only" but not care which node. If we have 100 Processors
> configured for Single Node Only, each node in the cluster should run some
> number of those Processors but not all. It is worth noting that we already
> have the necessary prerequisites in place for Leader Election, as we already
> use these for Primary Node and Cluster Coordinator - this simply allows for
> more leases to be used.
> h3. NiFi API Changes
> Introduce a new ExecutionNode value of `SINGLE_NODE_ONLY`.
> Deprecate the `@PrimaryNodeOnly` in favor of a new `@SingleNodeOnly`
> Introduce a new `@OnSingleNodeElectionChange` annotation that can be added to
> processors that is analogous to the @OnPrimaryNodeChange.
> h3. REST API Changes
> There should be no significant changes to the REST API, except to allow for
> the new Execution Node value.
> h3. UI Changes
> We will need a new icon similar to the 'P' badge that shows on Processors
> when the Processor is run on Primary Node Only. This also appears in the
> Summary table. We also need the ability to choose the new value for the
> "Execution Node".
> h3. Open Questions
> Do we truly have a need for Primary Node still? Or should this replace it
> entirely?
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
