[
https://issues.apache.org/jira/browse/NIFI-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727829#comment-15727829
]
ASF GitHub Bot commented on NIFI-1202:
--------------------------------------
GitHub user ijokarumawak opened a pull request:
https://github.com/apache/nifi/pull/1306
NIFI-1202: Site-to-Site batch settings.
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
- Added batchCount, batchSize, batchDuration to limit flow files to be
included in a single Site-to-Site transaction.
- Added batch throttling logic when StandardRemoteGroupPort transfers
flow files to a remote input port using the batch limit configurations,
so that users can limit batch
not only for pulling data, but also pushing data.
- Added destination list shuffle to provide better load distribution.
Previously, the load distribution algorithm produces the same host
consecutively.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ijokarumawak/nifi nifi-1202
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1306.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1306
----
commit 511cdbb8c0418d7bbab7cf0dabbf3ca18100246b
Author: Koji Kawamura <[email protected]>
Date: 2016-11-18T06:31:39Z
NIFI-1202: Site-to-Site batch settings.
- Added batchCount, batchSize, batchDuration to limit flow files to be
included in a single Site-to-Site transaction.
- Added batch throttling logic when StandardRemoteGroupPort transfers
flow files to a remote input po rt using the batch limit configurations,
so that users can limit batch not only for pulling data, but also pushing
data.
- Added destination list shuffle to provide better load distribution.
Previously, the load distribution algorithm produced the same host
consecutively.
----
> Allow user to configure Batch Size for site-to-site
> ---------------------------------------------------
>
> Key: NIFI-1202
> URL: https://issues.apache.org/jira/browse/NIFI-1202
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Core UI, Documentation & Website
> Reporter: Mark Payne
> Assignee: Mark Payne
> Attachments: WIP-added-batch-settings-ui.png,
> configure-remote-port-window.png
>
>
> Currently, there is no way for a user to specify the batch size that
> Site-to-Site will use. The framework decides this for you. However, if we
> want to use the List/Fetch Pattern, it will be helpful to specify a small
> batch size so that a small number of things that are listed are still well
> distributed across the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
