[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-27 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r64883424 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java --- @@ -135,16 +135,22 @@ protected static void

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-24 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r64395572 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java --- @@ -104,8 +106,25 @@ public void

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-24 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r64392589 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java --- @@ -104,8 +106,25 @@ public void

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-24 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/2020#issuecomment-221227518 CC: @StephanEwen By the way, it might not look like it but the only additional methods this introduces on `StreamExecutionEnvironment` are are these three:

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/2020#issuecomment-220988362 Thanks, the changes look good. R: @StephanEwen for taking a look at the API, you would only look at `StreamExecutionEnvironment`, for this. --- If your

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread kl0u
Github user kl0u commented on the pull request: https://github.com/apache/flink/pull/2020#issuecomment-220986678 @aljoscha Thanks a lot for the comments! I integrated them already. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/2020#issuecomment-220981894 All in all, very good work! One thing I'd like to change is the order of parameters in the `readFile` methods. For these telescoping methods is usual to append

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r64221544 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/DataStreamSource.java --- @@ -44,6 +44,11 @@ public

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread aljoscha
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2020#discussion_r64220867 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FilePathFilter.java --- @@ -0,0 +1,74 @@ +/* + *

[GitHub] flink pull request: [FLINK-2314] Make Streaming File Sources Persi...

2016-05-23 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2020 [FLINK-2314] Make Streaming File Sources Persistent This PR solves FLINK-2314 and combines a number of sub-tasks. In addition, it solves FLINK-3896 which was introduced as part of this task.