[
https://issues.apache.org/jira/browse/FLINK-5771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861022#comment-15861022
]
Colin Breame commented on FLINK-5771:
-------------------------------------
The problem is in the loop found at line 570 of DelimitedInputFormat.java:
{code}
while (this.readPos < this.limit && i <
this.delimiter.length) {
if ((this.readBuffer[this.readPos++]) ==
this.delimiter[i]) {
i++;
} else {
i = 0;
}
}
{code}
> DelimitedInputFormat does not correctly handle muli-byte delimiters
> -------------------------------------------------------------------
>
> Key: FLINK-5771
> URL: https://issues.apache.org/jira/browse/FLINK-5771
> Project: Flink
> Issue Type: Bug
> Components: filesystem-connector
> Affects Versions: 1.2.0
> Reporter: Colin Breame
> Attachments: Test.java, test.txt
>
>
> The DelimitedInputFormat does not correctly handle multi-byte delimiters.
> The reader sometimes misses a delimiter if it is preceded by the first byte
> from the delimiter. This results in two records (or more) being returned
> from a single call to nextRecord.
> See attached test case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
