[
https://issues.apache.org/jira/browse/FLINK-8331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16311497#comment-16311497
]
ASF GitHub Bot commented on FLINK-8331:
---------------------------------------
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/5218#discussion_r159670504
--- Diff:
flink-core/src/main/java/org/apache/flink/types/parser/StringValueParser.java
---
@@ -45,6 +45,12 @@ public void enableQuotedStringParsing(byte
quoteCharacter) {
@Override
public int parseField(byte[] bytes, int startPos, int limit, byte[]
delimiter, StringValue reusable) {
+ if (startPos == limit) {
+ setErrorState(ParseErrorState.EMPTY_COLUMN);
+ reusable.setValueAscii(bytes, startPos, limit -
startPos);
--- End diff --
change to `reusable.setValueAscii(bytes, startPos, 0);`
also this early out means that the later check `if (limit == startPos)` in
line 99 will never be true and can be removed.
> FieldParsers do not correctly set EMPT_COLUMN error state
> ---------------------------------------------------------
>
> Key: FLINK-8331
> URL: https://issues.apache.org/jira/browse/FLINK-8331
> Project: Flink
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.5.0, 1.4.1
> Reporter: Fabian Hueske
> Assignee: sunjincheng
>
> Some {{FieldParser}} do not correctly set the EMPTY_COLUMN error state if a
> field is empty.
> Instead, they try to parse the field value from an empty String which fails,
> e.g., in case of the {{DoubleParser}} with a {{NumberFormatException}}.
> The {{RowCsvInputFormat}} has a flag to interpret empty fields as {{null}}
> values. The implementation requires that all {{FieldParser}} correctly return
> the EMPTY_COLUMN error state in case of an empty field.
> Affected {{FieldParser}}:
> - BigDecParser
> - BigIntParser
> - DoubleParser
> - FloatParser
> - SqlDateParser
> - SqlTimeParser
> - SqlTimestampParser
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
