[
https://issues.apache.org/jira/browse/NIFI-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362014#comment-17362014
]
ASF subversion and git services commented on NIFI-8689:
-------------------------------------------------------
Commit 64f600d0ce28be3856c1abdf947d05058ea548a7 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=64f600d ]
NIFI-8689: This closes #5150. Avoid flushing the socket buffer unnecessarily
when sending a series of FlowFiles via site-to-site
Signed-off-by: Joe Witt <[email protected]>
> Site-to-Site client is constantly flushing the socket's OutputStream
> --------------------------------------------------------------------
>
> Key: NIFI-8689
> URL: https://issues.apache.org/jira/browse/NIFI-8689
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
>
> When a RemoteProcessGroup is sending data to another NiFi instance, the
> protocol should establish a transaction and then send a sequence of FlowFiles
> following a pattern along the lines of:
> {code:java}
> <FlowFile Follows><FlowFile Attributes><FlowFile Content>
> <FlowFile Follows><FlowFile Attributes><FlowFile Content>
> <FlowFile Follows><FlowFile Attributes><FlowFile Content>
> <Finished Transaction>{code}
> However, currently, the protocol is flushing the Socket's output buffer each
> that that it indicates that a FlowFile follows, and again after each
> FlowFile. So it's more like:
> {code:java}
> <FlowFile Follows>*Flush Buffer*
> <FlowFile Attributes><FlowFile Content>*Flush Buffer*
> <FlowFile Follows>*Flush Buffer*
> <FlowFile Attributes><FlowFile Content>*Flush Buffer*
> <FlowFile Follows>*Flush Buffer*
> <FlowFile Attributes><FlowFile Content>*Flush Buffer*{code}
> As a result, when sending a large number of smaller FlowFiles, we end up
> constantly flushing data to the socket, which results in dramatically worse
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
