[
https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542985#comment-15542985
]
ASF subversion and git services commented on NIFI-2825:
-------------------------------------------------------
Commit 17a36c6fd525dba837e13a71a52ddfd224782fae in nifi's branch
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=17a36c6 ]
NIFI-2825: Fix S2S getPeers flow file count
- Added ClusterWorkload message to retrieve workload information from a
cluster coordinator
- Use cluster workload to return queued flow file count to site-to-site
client so that it can calculate distribution of data transfer
This closes #1084.
Signed-off-by: Bryan Bende <[email protected]>
> Remote Process Group should use num of queued flow files in remote nodes to
> distribute load
> -------------------------------------------------------------------------------------------
>
> Key: NIFI-2825
> URL: https://issues.apache.org/jira/browse/NIFI-2825
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.0.0
> Environment: Clustered
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Labels: Cluster
> Fix For: 1.1.0
>
>
> A RPG (Site-to-Site client) should take into account of the number of queued
> flow files in each remote cluster nodes, to calculate the load distribution
> balance among target nodes. So that it can send more data to nodes those have
> less queued files, or receive more data from nodes those have more queued
> files.
> However, after the clustering mechanism has changed since 1.0, this load
> distribution logic is not handled appropriately. Site-to-Site remote
> component always returns flow file count as 0. So RPG distribute load evenly,
> even if a remote cluster has unbalanced load among nodes.
> We need to fix the remote node API to expose queued flow file count correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
