Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85866221
Cool, thanks. Will have a look shortly.
Did you rebase to the latest master? We had a few build issues with the
master branch a few days ago.
It is a
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-86011099
Oops, I pushed a intermediate commit a8a5c37. I will fix it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-86011531
Sure, no problem :-)
Can I check it now or do you need a bit more time?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-86011782
@fhueske You can check it now :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-86079595
@chiwanpark excellent job, thanks!
Will merge it after a final round of Travis tests passed.
---
If your project is set up for it, you can reply to this email
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/426
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018880
--- Diff:
flink-scala/src/main/scala/org/apache/flink/api/scala/ExecutionEnvironment.scala
---
@@ -223,8 +224,11 @@ class ExecutionEnvironment(javaEnv:
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018277
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -52,103 +50,86 @@
public static final String
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018316
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -52,103 +50,86 @@
public static final String
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85455383
Very good! Let me know, when you want me to have a look again :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018151
--- Diff:
flink-scala/src/main/java/org/apache/flink/api/scala/operators/ScalaCsvInputFormat.java
---
@@ -98,98 +123,66 @@ public void setFields(int[]
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85447697
I see the issue with the non-deterministic field order and FLINK-1665 as
follows. Both, FLINK-1665 and Option 3, solve the problem of non-deterministic
field order.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018812
--- Diff:
flink-scala/src/main/java/org/apache/flink/api/scala/operators/ScalaCsvInputFormat.java
---
@@ -19,66 +19,91 @@
package
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018473
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -52,103 +50,86 @@
public static final String
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r27018597
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -52,103 +50,86 @@
public static final String
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85830275
Hi, I updated this PR.
* Remove `pojoType(Class? targetType)` method in `CsvReader` to force the
user to explicitly specify the fields order.
* Add
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-84977748
Hi @chiwanpark
Thanks for updating the PR! :-)
I was gone for a few days. Will have a look at your PR shortly.
---
If your project is set up for it, you can
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26991431
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -234,9 +301,23 @@ public OUT readRecord(OUT reuse, byte[] bytes,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26992871
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/CsvInputFormatTest.java
---
@@ -684,4 +693,249 @@ private void testRemovingTrailingCR(String
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26993897
--- Diff:
flink-scala/src/main/scala/org/apache/flink/api/scala/ExecutionEnvironment.scala
---
@@ -247,16 +252,27 @@ class ExecutionEnvironment(javaEnv:
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85261367
Hi @chiwanpark, the PR improved a lot! Besides a few inline comments, the
logic and test coverage looks pretty good. Unfortunately, the PR does not seem
to build
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26993011
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/CsvInputFormatTest.java
---
@@ -684,4 +693,249 @@ private void testRemovingTrailingCR(String
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26994167
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/io/CsvReaderWithPOJOITCase.java
---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26993339
--- Diff:
flink-scala/src/main/java/org/apache/flink/api/scala/operators/ScalaCsvInputFormat.java
---
@@ -219,73 +92,24 @@ public OUT readRecord(OUT reuse,
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-85285193
@fhueske Hi, Thanks for your kindly advice! I will fix them soon.
About the order of POJO fields, I think also that the option 3 is good.
However,
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-82885499
I updated this PR.
* Change method of obtaining `Field` object from using `PojoTypeInfo` to
saving field names. (Thanks @fhueske for advice!)
*
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-78876571
Hello. I have a question about object reuse in `readRecord` method of
`ScalaCsvInputFormat`. In java implementation, `CsvInputFormat` reuse result
object. But in
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-79066240
@aljoscha Thanks! I understand about it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26215940
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/CsvInputFormatTest.java
---
@@ -684,4 +693,178 @@ private void testRemovingTrailingCR(String
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26203612
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -64,26 +70,45 @@
private transient int commentCount;
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26205234
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -152,6 +177,38 @@ public void setFields(boolean[] sourceFieldMask,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26206307
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -234,9 +291,29 @@ public OUT readRecord(OUT reuse, byte[] bytes,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26206381
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -234,9 +291,29 @@ public OUT readRecord(OUT reuse, byte[] bytes,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26205969
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -152,6 +177,38 @@ public void setFields(boolean[] sourceFieldMask,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26206133
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -152,6 +177,38 @@ public void setFields(boolean[] sourceFieldMask,
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r26206329
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -234,9 +291,29 @@ public OUT readRecord(OUT reuse, byte[] bytes,
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-78401359
@fhueske Thanks for your kindly advice. I will fix as soon as possible.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-77912526
@teabot: We've created this JIRA for the feature you've suggested:
https://issues.apache.org/jira/browse/FLINK-1665
---
If your project is set up for it, you can reply
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r25846058
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -64,15 +66,25 @@
private transient int commentCount;
Github user teabot commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-77347164
I like that this request adds what I would consider to be a very useful
feature. Data is often stored in text delimited formats and one of the nice
features of Flink is
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-77293135
@fhueske Oh, you are right. Currently, users cannot decide order of fields.
I will add a parameter to set order of fields.
---
If your project is set up for it, you
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/426#issuecomment-77272091
How are fields in the CSV file mapped to POJO fields? I assume it is the
order of fields in the POJO type information, right? Is that order the same as
in the POJO
Github user chiwanpark commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r25760453
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -235,8 +252,21 @@ public OUT readRecord(OUT reuse, byte[] bytes,
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r25237152
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -82,6 +92,13 @@ public CsvInputFormat(Path filePath, String
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/426#discussion_r25237861
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
@@ -235,8 +252,21 @@ public OUT readRecord(OUT reuse, byte[] bytes,
GitHub user chiwanpark opened a pull request:
https://github.com/apache/flink/pull/426
[FLINK-1512] Add CsvReader for reading into POJOs.
This PR contains following changes.
* `CsvInputFormat` and `ScalaCsvInputFormat` can receive POJO type as
generic parameter
* Add
46 matches
Mail list logo
