Hi Mohit, NiFi RPG batches multiple FlowFiles into the same Site-to-Site transaction, and the default batch settings are configured for higher throughput. If you prefer more granular distribution, you can lower the batch configurations from "Manage Remote Ports" context menu of a RemoteProcessGroup. The batch size configuration from UI is added since NiFi 1.2.0, and the JIRA can be a reference. https://issues.apache.org/jira/browse/NIFI-1202
Thanks, Koji On Tue, May 8, 2018 at 2:24 PM, Mohit <[email protected]> wrote: > Hi, > > > > I need some clarity on how flowfile is distributed among different nodes in > a Nifi cluster. > > > > I have a flow where I’m using GenerateTableFetch to fetch the data from > database. Source table has around 40 million records. I tried with different > partition size which led to create different number of flowfiles. > > When there are less number of flowfiles(~40), RPG sends it to only one > node(in a 4 node cluster) but when there are large number of > flowfiles(~400), it distribute the flowfile among all the nodes. > > Are there some rules or best practices to fine tune the flow, so that the > flowfiles are evenly distributed across the nodes even if there are less > number of flowfiles. > > > > Regards, > > Mohit
