Thanks Bryan, This really helps.
-----Original Message----- From: Bryan Bende <bbe...@gmail.com> Sent: September 12, 2018 10:23 AM To: users@nifi.apache.org Subject: Re: Running Nifi in cluster mode Hello, There is a graph of processors often referred to as the "flow" and each node in the cluster runs a copy of the flow. So all nodes are running the same components, with the exception of source processors that happen to be scheduled as primary node only. The data must be divided across the cluster to make use of the cluster appropriately, and this depends on the source of your data [1]. Since all nodes are generally doing the same thing, it probably makes the most sense for them to similar in terms of hardware. They don't have to be, but NiFi is not making any decisions based on the hardware. -Bryan [1] https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html On Wed, Sep 12, 2018 at 9:58 AM, Jean-Sebastien Vachon <jsvac...@brizodata.com> wrote: > Hi all, > > > > Can someone tell me how Nifi manages/dispatches jobs to nodes in a cluster? > Right now, I have a cluster of only three identical machines running > on AWS but I would like to be able to extend my cluster by adding spot > instances of different types and capacity. Will Nifi be aware that > some machines do not have the same capacity ? or should I try to keep > the capacity (CPU, RAM) the same across the cluster? > > > > Also, is Nifi looking at some metrics to determine where a given > processor should be executed? Does it have any load balancing > algorithm to spread the load as evenly as possible? > > > > Thanks
