Arvind Kandpal created NIFI-16610:
-------------------------------------
Summary: StreamDemarcator misses a multi-byte delimiter that
follows a partial match
Key: NIFI-16610
URL: https://issues.apache.org/jira/browse/NIFI-16610
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Arvind Kandpal
Assignee: Anshu Singh
StreamDemarcator does not find a multi-byte delimiter when the bytes just
before it contain a partial match of the delimiter.
In nextToken(), when a byte does not match delimiterBytes[j], the match index j
is reset to 0, but the current byte is not compared again with
delimiterBytes[0]. If that byte is the start of the real delimiter, the
delimiter is skipped and the two tokens are returned as one.
StreamDemarcator is used by PublishKafka (DelimitedStreamKafkaRecordConverter)
when the Message Demarcator property is set. With a multi-byte demarcator,
messages can be silently combined into one Kafka message.
Steps to reproduce (StreamDemarcator with maxDataSize 1000):
{noformat}
data "msg1\r\r\nmsg2" delimiter "\r\n" actual [msg1\r\r\nmsg2] expected
[msg1\r, msg2]
data "x---END--y" delimiter "--END--" actual [x---END--y] expected
[x-, y]
data "helloaaab" delimiter "aab" actual [helloaaab] expected
[helloa]
{noformat}
Single-byte delimiters such as "\n" are not affected.
Code: nifi-utils, StreamDemarcator.java, nextToken()
The existing tests in StreamDemarcatorTest only use delimiters where this case
does not happen (for example "daa").
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)