David Handermann created NIFI-10737:
---------------------------------------
Summary: ListenBeats Unable to Process Large Batches
Key: NIFI-10737
URL: https://issues.apache.org/jira/browse/NIFI-10737
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.18.0, 1.17.0, 1.16.0
Reporter: David Handermann
Assignee: David Handermann
Refactoring of ListenBeats released in version 1.16.0 introduced processing
issues when reading batches of messages larger than 2048 bytes.
The refactored implementation based on Netty introduced a {{BeatsFrameDecoder}}
extension of the Netty {{{}ByteToMessageDecoder{}}}, which the Netty framework
invokes multiple times with variables contents of the input {{{}ByteBuf{}}}.
The implementation does not handle scenarios where a batch of multiple messages
is larger than the buffer size of 2048. As a result of this behavior, the
Decoder does not construct a complete message. Without the complete message,
the Processor does not send an acknowledgement for the received message,
resulting in errors on sending clients, such as the following:
{noformat}
Failed to publish events caused by: read tcp
192.168.1.100:54623->192.168.1.200:5555: i/o timeout
{noformat}
This error occurs on sending client while waiting to receive an acknowledgement
packet for the messages transmitted.
As a result of not receiving the acknowledgement, the client attempts to send
the same messages multiple times, resulting in duplication and failure to send
additional messages.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
