Matthew Clarke created NIFI-2987:
------------------------------------
Summary: RPG does not do load-balancing well when getting
FlowFiles from output ports
Key: NIFI-2987
URL: https://issues.apache.org/jira/browse/NIFI-2987
Project: Apache NiFi
Issue Type: New Feature
Components: Core Framework
Affects Versions: 1.0.0
Reporter: Matthew Clarke
When a RPG connects to a destination system's output port, it retrieves every
FlowFile queued at the time of the connection. If the source system with the
RPG is a NiFi cluster, only one node in the cluster receives all the FlowFiles
from that output port. If there is a steady stream of FlowFiles to the output
port, there is still no true balanced delivery of data.
We need to be able to limit the number of FlowFiles per connection when the
source of the FlowFiles is an output port.
When the destination system with the output port is a cluster and the source
system is a cluster with a RPG, the first node to connect will pull all data
from the node with the highest queue. Next node will pull from next highest
queued destination node and so on. There is also no guarantee that avery node
in the source cluster will get any data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
