[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-06-01 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/2020 The input formats still have a leftover field that stores the split. After that, the only thing that remains is the API methods. Also what was the reason for the new code in

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-06-01 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65347119 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java --- @@ -104,8 +106,21 @@ public void

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/2020 The changes look good, we just have to figure out what to do about the methods on `StreamExecutionEnvironment`. One thing I'd like to get rid of is

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65216093 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java --- @@ -104,8 +106,21 @@ public void

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65212938 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -0,0 +1,336 @@

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65212915 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -0,0 +1,336 @@

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65212384 --- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/StreamExecutionEnvironment.scala --- @@ -483,28 +500,31 @@ class

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65209503 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java --- @@ -897,21 +900,21 @@ public

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65209426 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java --- @@ -877,15 +880,15 @@ public

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persistent

2016-05-31 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r65207037 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java --- @@ -82,6 +82,8 @@ */ private static int