[
https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536241#comment-15536241
]
ASF GitHub Bot commented on NIFI-2825:
--------------------------------------
GitHub user ijokarumawak opened a pull request:
https://github.com/apache/nifi/pull/1084
NIFI-2825: Fix S2S getPeers flow file count
- Added ClusterWorkload message to retrieve workload information from a
cluster coordinator
- Use cluster workload to return queued flow file count to site-to-site
client so that it can calculate distribution of data transfer
Added related unit tests, as well as test with running NiFi secure/unsecure
cluster, and standalone.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ijokarumawak/nifi nifi-2825
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1084.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1084
----
commit 2fde9b9d478f3a421bc883ea0d23a21bfa79d5b2
Author: Koji Kawamura <[email protected]>
Date: 2016-09-28T02:07:22Z
NIFI-2825: Fix S2S getPeers flow file count
- Added ClusterWorkload message to retrieve workload information from a
cluster coordinator
- Use cluster workload to return queued flow file count to site-to-site
client so that it can calculate distribution of data transfer
----
> Remote Process Group should use num of queued flow files in remote nodes to
> distribute load
> -------------------------------------------------------------------------------------------
>
> Key: NIFI-2825
> URL: https://issues.apache.org/jira/browse/NIFI-2825
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.0.0
> Environment: Clustered
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Fix For: 1.1.0
>
>
> A RPG (Site-to-Site client) should take into account of the number of queued
> flow files in each remote cluster nodes, to calculate the load distribution
> balance among target nodes. So that it can send more data to nodes those have
> less queued files, or receive more data from nodes those have more queued
> files.
> However, after the clustering mechanism has changed since 1.0, this load
> distribution logic is not handled appropriately. Site-to-Site remote
> component always returns flow file count as 0. So RPG distribute load evenly,
> even if a remote cluster has unbalanced load among nodes.
> We need to fix the remote node API to expose queued flow file count correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
