You're saying it wouldn't kick in because of low rates of files? The interesting bit for us is that we don't process a lot of files but the files themselves are quite large. And in our case some nodes keep getting behind and it seems that load balancer is not kicking in to rebalance the cluster. Is there any way we could force the balancer to be a lot more aggressive even on smaller queues?
On Thu, Jan 28, 2021 at 8:23 PM Joe Witt <joe.w...@gmail.com> wrote: > Maksym, > > Very difficult to look at these brief/limited details and offer meaningful > responses. In the example you show about the data volumes are so small > that I dont even know that load balancing would kick in. But yes generally > speaking the combination of load balancing and back pressure controls can > yield extremely well balanced flows even when you have > heterogeneous infrastructure. > > Thanks > > On Thu, Jan 28, 2021 at 4:45 AM Maksym Skrynnikov < > skrynnikov.mak...@verizonmedia.com> wrote: > >> I am running NiFi 1.12.1 without NiFi Registry and have a connection that >> is configured to *Round Robin *flow files. After some time I see some >> nodes performing worse than the other and the queue is piling up on 1-2 >> nodes. >> [image: niifi-queue.jpg] >> >> The question I have is how rebalancing actually work? Would it ideally >> try to rebalance what's already on the node? Or nodes that are >> underperforming would always end up with the largest queue? I expected it >> to rebalance the load so each node would get its portion of the queue. >> >> Thank you >> >
