[
https://issues.apache.org/jira/browse/FLINK-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884154#comment-15884154
]
ASF GitHub Bot commented on FLINK-5907:
---------------------------------------
GitHub user KurtYoung opened a pull request:
https://github.com/apache/flink/pull/3417
[FLINK-5907] [java api] Fix trailing empty fields in CsvInputFormat
If there are 3 fields with field delimiter ",", both these two line should
be parsed successfully:
aaa,bbb,
aaa,bbb
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KurtYoung/flink flink-5907
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3417.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3417
----
commit 4e93fe1376d623eda5f8f68afb54583070968881
Author: Kurt Young <[email protected]>
Date: 2017-02-25T08:37:37Z
[FLINK-5907] [java api] Fix trailing empty fields in CsvInputFormat
----
> RowCsvInputFormat bug on parsing tsv
> ------------------------------------
>
> Key: FLINK-5907
> URL: https://issues.apache.org/jira/browse/FLINK-5907
> Project: Flink
> Issue Type: Bug
> Components: Java API
> Affects Versions: 1.2.0
> Reporter: Flavio Pompermaier
> Assignee: Kurt Young
> Labels: csv, parsing
> Attachments: test.tsv
>
>
> The following snippet reproduce the problem (using the attached file as
> input):
> {code:language=java}
> char fieldDelim = '\t';
> TypeInformation<?>[] fieldTypes = new TypeInformation<?>[51];
> for (int i = 0; i < fieldTypes.length; i++) {
> fieldTypes[i] = BasicTypeInfo.STRING_TYPE_INFO;
> }
> int[] fieldMask = new int[fieldTypes.length];
> for (int i = 0; i < fieldMask.length; i++) {
> fieldMask[i] = i;
> }
> RowCsvInputFormat csvIF = new RowCsvInputFormat(new Path(testCsv),
> fieldTypes, "\n", fieldDelim +"",
> fieldMask, true);
> csvIF.setNestedFileEnumeration(true);
> DataSet<Row> csv = env.createInput(csvIF);
> csv.print()
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
