Mark Payne created NIFI-8689:
--------------------------------
Summary: Site-to-Site client is constantly flushing the socket's
OutputStream
Key: NIFI-8689
URL: https://issues.apache.org/jira/browse/NIFI-8689
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
When a RemoteProcessGroup is sending data to another NiFi instance, the
protocol should establish a transaction and then send a sequence of FlowFiles
following a pattern along the lines of:
{code:java}
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<Finished Transaction>{code}
However, currently, the protocol is flushing the Socket's output buffer each
that that it indicates that a FlowFile follows, and again after each FlowFile.
So it's more like:
{code:java}
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*{code}
As a result, when sending a large number of smaller FlowFiles, we end up
constantly flushing data to the socket, which results in dramatically worse
performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
