Hello, thanks for your answer i have another question about the single node load balancing strategy. I have this strategy on the first queue of all my groups and i noticed that only one of the nodes has flowfiles in its queues. So i have many identical groups and the first processor that generates the data in each group runs in the primary node. I've added the 'single node' load balancing strategy to the queue after the first processor. I did that because i want all the flowfiles in the same group to be in the same node, but i don't want all the flowfiles among all the groups to be on the same node. My assumption was that all the flowfiles of lets say group A would go to the same node, but the flowfiles of group B will not go necessarily to the same node as those of A. Can someone clarify if this strategy is 'per queue' or for the totality of the flowfiles? The volume of the flowfiles is not so big so maybe it is by chance that all queues were sent to the same node (I have 3 nodes total). I can't use round robin load balancing because the flowfiles in a single group must reside on the same node. My next choice would be to partition by (an arbitrary) attribute.
On Fri, Dec 20, 2024 at 5:12 PM Mark Payne <marka...@hotmail.com> wrote: > Bill, > > The node assignment can change only when the cluster topology changes. > I.e., when a new node joins the cluster or a node is removed from the > cluster. > Note that this does not include when a node becomes disconnected - only > when it’s completely removed. At that point, if a different node is elected > as the > “Winner” all data will be pushed from the queue to the new node. > > Thanks > -Mark > > > On Dec 20, 2024, at 9:39 AM, Bill Tsek <billtse...@gmail.com> wrote: > > > > I want to clarify how single node load balancing works. > > > > My assumption is the following: > > > > When load balancing occurs, all flowfiles are directed to the same node, > but to which one they are directed to, can change over time. > > > > So if the queue contains 10 flowfiles right now, they will all be > directed to the same node, lets say 'node x'. > > > > After processing, the queue becomes empty and 100 flowfiles arrive again. > > Now all those flowfiles will be directed to the same node, but that node > might not be 'node x'. > > > > If a flowfile arrives to a non-empty queue, the new flowfile will be > directed to the same node as the other flowfiles currently in the queue. > > > > So the node assignment can change only after a queue is emptied. > > > > Is my assumption correct? > > > > Thank you. > >
