[GitHub] flink issue #2361: [FLINK-3318][cep] Add support for quantifiers to CEP's pa...

2017-01-26 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2361 I was talking about a pattern like `begin().oneOrMany().where("a")`. So that you just expect as many consecutive `a`'s as possible. --- If your project is set up for it, you can reply to

[GitHub] flink issue #3191: [FLINK-5529] [FLINK-4752] [docs] Improve / extends window...

2017-01-26 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3191 @aljoscha thanks for the review. I integrated the comments. I also include @fhueske as he also contributed to this. --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink issue #2361: [FLINK-3318][cep] Add support for quantifiers to CEP's pa...

2017-01-26 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2361 Well for iii), the idea behind the new type of state is that the `NFA` will see one `PotentiallyFinal` element, but it will continue receiving, until the first non-eligible element. So if the pattern

[GitHub] flink issue #2361: [FLINK-3318][cep] Add support for quantifiers to CEP's pa...

2017-01-26 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2361 Hi @mushketyk ! Thanks a lot! The problem is in the `NFACompiler` and more specifically the way the pattern is compiled. I have not come up with a concrete solution yet but I would say

[GitHub] flink issue #2361: [FLINK-3318][cep] Add support for quantifiers to CEP's pa...

2017-01-25 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2361 Hi @mushketyk ! Yes. In this case we would expect to have everything apart from the "end" event in the result, right? --- If your project is set up for it, you can reply to this email and

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97789351 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/State.java --- @@ -43,7 +43,14 @@ public State(final String name, final StateType

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97790026 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97790105 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97790271 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97790228 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97790385 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97789988 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97789900 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -130,26 +114,166

[GitHub] flink pull request #2361: [FLINK-3318][cep] Add support for quantifiers to C...

2017-01-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2361#discussion_r97789775 --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/compiler/NFACompiler.java --- @@ -80,25 +82,17 @@ // return

[GitHub] flink pull request #3201: [FLINK-5620] Fix unstable ContinuousFileProcessing...

2017-01-25 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/3201 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3201: [FLINK-5620] Fix unstable ContinuousFileProcessingTest

2017-01-25 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3201 @aljoscha Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #3201: [FLINK-5620] Fix unstable ContinuousFileProcessing...

2017-01-24 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3201 [FLINK-5620] Fix unstable ContinuousFileProcessingTest This fixes at least one possible problem that may render the test unstable, which is that all the tests in the class create files with the same

[GitHub] flink issue #3134: [FLINK-5450] Fix restore from legacy log message

2017-01-23 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 Thanks a lot! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3134: [FLINK-5450] Fix restore from legacy log message

2017-01-23 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 @rmetzger I see. Then my PR fixes it, as the problem was that when retrieving timers, we were first putting them in the temporary list in the `initializeState` and the re-register them

[GitHub] flink issue #3180: [FLINK-5532] Make window assigners for aligned window ope...

2017-01-23 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3180 Thanks @fhueske and @StephanEwen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3180: [FLINK-5532] Make window assigners for aligned win...

2017-01-23 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/3180#discussion_r97283933 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/SlidingAlignedProcessingTimeWindows.java --- @@ -35,7 +35,7

[GitHub] flink pull request #3191: [FLINK-5529] [FLINK-4752] [docs] Improve / extends...

2017-01-23 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3191 [FLINK-5529] [FLINK-4752] [docs] Improve / extends windowing documentation This PR is for both the issues in the title. It refactors/improves/extends the documentation of the windowing logic

[GitHub] flink pull request #3180: [FLINK-5532] Make window assigners for aligned win...

2017-01-20 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3180 [FLINK-5532] Make window assigners for aligned window operators non-extendable Makes the TumblingAlignedProcessingTimeWindows and the SlidingAlignedProcessingTimeWindows final so that users cannot

[GitHub] flink issue #3130: [FLINK-5502] Add the function migration guide in docs.

2017-01-19 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3130 Thanks @alpinegizmo for the review! I integrated your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #3134: [FLINK-5450] Fix restore from legacy log message

2017-01-18 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 From what I understood from the JIRA, it is only when restoring from legacy. If not, then the problem is deeper. @rmetzger ? --- If your project is set up for it, you can reply to this email and have

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2017-01-17 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 Thanks a lot @zentol . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #3134: [FLINK-5450] Fix restore from legacy log message

2017-01-17 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3134 [FLINK-5450] Fix restore from legacy log message You can merge this pull request into a Git repository by running: $ git pull https://github.com/kl0u/flink window-op-logging Alternatively you

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2017-01-16 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 Hi @zentol ! I integrated some remaining comments could you have a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request #3130: [FLINK-5502] Add the function migration guide in d...

2017-01-16 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3130 [FLINK-5502] Add the function migration guide in docs. This PR introduces the migration guide from Flink-1.1 to Flink-1.2 for user-defined functions. You can merge this pull request into a Git

[GitHub] flink pull request #3094: [FLINK-5443] [FLINK-5318] RollingSink backwards co...

2017-01-13 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/3094 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3075: [FLINK-5296] Expose the old AlignedWindowOperators throug...

2017-01-12 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3075 Thanks @aljoscha . Will do that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3075: [FLINK-5296] Expose the old AlignedWindowOperators...

2017-01-12 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/3075 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #3075: [FLINK-5296] Expose the old AlignedWindowOperators throug...

2017-01-11 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3075 @aljoscha thanks for the review. I changed the names of the tests and also put them in the right classes. Let me know what you think. --- If your project is set up for it, you can reply

[GitHub] flink pull request #3094: [FLINK-5443] [FLINK-5318] RollingSink backwards co...

2017-01-11 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3094 [FLINK-5443] [FLINK-5318] RollingSink backwards compatible and migration to BucketingSink. This PR makes the RollingSink backwards compatible and allows to migrate from a RollingSink (deprecated

[GitHub] flink pull request #3075: [FLINK-5296] Expose the old AlignedWindowOperators...

2017-01-11 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/3075#discussion_r95593492 --- Diff: flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/WindowTranslationTest.scala --- @@ -48,6 +49,45 @@ import org.junit.Test

[GitHub] flink pull request #3075: [FLINK-5296] Expose the old AlignedWindowOperators...

2017-01-11 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/3075#discussion_r95593424 --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/TimeWindowTranslationTest.java --- @@ -51,12 +53,48

[GitHub] flink issue #3075: [FLINK-5296] Expose the old AlignedWindowOperators throug...

2017-01-11 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3075 @aljoscha I integrated your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #3075: [FLINK-5296] Expose the old AlignedWindowOperators throug...

2017-01-10 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3075 @aljoscha Thanks for the review! I will integrate the comments and ping you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #2929: [FLINK-5247] Fix incorrect check in allowedLateness() met...

2017-01-06 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2929 Hi @mindprince . What do you mean "in the file"? The allowed lateness can always be specified in the program by the developer, right? So logging a message is always a valid approach. -

[GitHub] flink pull request #3075: [FLINK-5296] Expose the old AlignedWindowOperators...

2017-01-06 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3075 [FLINK-5296] Expose the old AlignedWindowOperators through special as… This PR allows the user to use the deprecated `AccumulatingProcessingTimeWindowOperator

[GitHub] flink issue #2871: [FLINK-5163] Ports the production functions to the new st...

2016-12-13 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2871 @aljoscha Yes thanks a lot! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink pull request #2871: [FLINK-5163] Ports the production functions to the...

2016-12-13 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/2871 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2016-12-13 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 I think your argument is valid. This is how I had it before, but I changed it because it becomes too verbose, given that the `ListState.get()` just gives you an Iterable that you have to iterate over

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2016-12-12 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 @zentol Thanks for the review. I integrated your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2016-12-12 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 The test should cover if something semantically wrong happened during restoring. For example, if you expected some state that never came, then the test should fail. This is not a matter

[GitHub] flink issue #2939: [FLINK-5113] Ports all functions in the tests to the new ...

2016-12-12 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2939 @zentol Thanks for the review. I integrated your comments and I am waiting for Travis. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] flink issue #2871: [FLINK-5163] Ports the production functions to the new st...

2016-12-12 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2871 @aljoscha Thanks a lot for the review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #2939: [FLINK-5113] Ports all functions in the tests to t...

2016-12-05 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2939 [FLINK-5113] Ports all functions in the tests to the new checkpointing interface. This PR only touches testing code. It refactors all the tests that were using the `Checkpointed

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759808 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/KeyedStream.java --- @@ -173,67 +174,67 @@ public KeyedStream(DataStream

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89760994 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java --- @@ -50,7 +46,7 @@ * @param Output

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759655 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/ConnectedStreams.java --- @@ -234,62 +235,62 @@ public

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759704 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/ConnectedStreams.java --- @@ -234,62 +235,62 @@ public

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89761615 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java --- @@ -119,7 +115,7

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89762025 --- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/ConnectedStreams.scala --- @@ -101,30 +101,31 @@ class ConnectedStreams[IN1

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89761309 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java --- @@ -50,7 +46,7 @@ * @param Output

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89761398 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java --- @@ -78,7 +74,7 @@ * @throws

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89761160 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java --- @@ -50,7 +46,7 @@ * @param Output

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759545 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/ConnectedStreams.java --- @@ -234,62 +235,62 @@ public

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759846 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/KeyedStream.java --- @@ -173,67 +174,67 @@ public KeyedStream(DataStream

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89760102 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/ProcessFunction.java --- @@ -24,32 +24,24 @@ import

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89762169 --- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/KeyedStream.scala --- @@ -54,28 +54,28 @@ class KeyedStream[T, K

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89762073 --- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/ConnectedStreams.scala --- @@ -101,30 +101,31 @@ class ConnectedStreams[IN1

[GitHub] flink pull request #2788: [FLINK-5026] Rename TimelyFlatMap to Process

2016-11-28 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2788#discussion_r89759486 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/ConnectedStreams.java --- @@ -234,62 +235,62 @@ public

[GitHub] flink pull request #2871: [FLINK-5163] Ports the production functions to the...

2016-11-25 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2871 [FLINK-5163] Ports the production functions to the new state abstraction. This includes the following functions: 1) `StatefulSequenceSource` 2) `MessageAcknowledgingSourceBase` 3

[GitHub] flink issue #2845: [FLINK-5096] Make the RollingSink rescalable.

2016-11-25 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2845 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #2845: [FLINK-5096] Make the RollingSink rescalable.

2016-11-25 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2845#discussion_r89603474 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/test/java/org/apache/flink/streaming/connectors/fs/RollingSinkITCase.java --- @@ -638,6

[GitHub] flink issue #2845: [FLINK-5096] Make the RollingSink rescalable.

2016-11-25 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2845 Hi @zentol ! Thanks for the review. I integrated the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #2845: [FLINK-5096] Make the RollingSink rescalable.

2016-11-24 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2845 Thanks for the review @zentol ! I integrated your comments and rebased to the master. Let's see what travis has to say and then merge it if you have no further comments ;) --- If your project

[GitHub] flink pull request #2845: [FLINK-5096] Make the RollingSink rescalable.

2016-11-21 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2845 [FLINK-5096] Make the RollingSink rescalable. This PR integrates the RollingSink with the new state abstractions, so that the sink can resume execution from a checkpoint, with different

[GitHub] flink issue #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-16 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2797 Done @zentol --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-16 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88211157 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-16 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88209971 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/test/java/org/apache/flink/streaming/connectors/fs/bucketing

[GitHub] flink issue #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-16 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2797 Hi @zentol . I integrated your last comments. Now for the your last question, the answer is that the only way to distinguish between the two types of files is by their filename (prefix

[GitHub] flink issue #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2797 Sorry I just amended my previous commit with the comment integration. :S --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink issue #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2797 The subtask matters for files that were opened but they did not make it into a checkpoint because a failure occurred after they were opened but before a checkpoint barrier arrived

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88110293 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88105517 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java

[GitHub] flink issue #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2797 Thanks for the first review @zentol . On the comment about the diff reduction, you are referring to the removal of the method in the intermediate commits that was not used? If yes

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88063735 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java

[GitHub] flink pull request #2797: [FLINK-5056] Makes the BucketingSink rescalable.

2016-11-15 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2797#discussion_r88063330 --- Diff: flink-streaming-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java

[GitHub] flink pull request #2763: [FLINK-5021] Makes the ContinuousFileReaderOperato...

2016-11-11 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/2763 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2765: [FLINK-5027] FileSource finishes successfully with a wron...

2016-11-10 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2765 Thanks a lot @tillrohrmann --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink issue #2759: [FLINK-5020] Make the GenericWriteAheadSink rescalable.

2016-11-09 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2759 Hi @zentol . I just rebased to the master, as there were conflicts. After travis passes could you have a look and merge this if there are no more comments? --- If your project is set up

[GitHub] flink issue #2765: [FLINK-5027] FileSource finishes successfully with a wron...

2016-11-07 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2765 @tillrohrmann Thanks for the review. I integrated the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #2765: [FLINK-5027] FileSource finishes successfully with...

2016-11-07 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2765 [FLINK-5027] FileSource finishes successfully with a wrong path Adds a test in the `run()` of the `ContinuousFileMonitoringFunction` that checks if the user-specified path is valid. If not, the job

[GitHub] flink pull request #2759: [FLINK-5020] Make the GenericWriteAheadSink rescal...

2016-11-07 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2759#discussion_r86757109 --- Diff: flink-streaming-connectors/flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/CassandraConnectorITCase.java

[GitHub] flink pull request #2763: [FLINK-5021] Makes the ContinuousFileReaderOperato...

2016-11-07 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2763 [FLINK-5021] Makes the ContinuousFileReaderOperator rescalable. This is the last PR that completes the refactoring of the `ContinuousFileReaderOperator` so that it can be rescalable

[GitHub] flink pull request #2759: [FLINK-5020] Make the GenericWriteAheadSink rescal...

2016-11-04 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2759 [FLINK-5020] Make the GenericWriteAheadSink rescalable. As the issue suggests, this is the final commit to make the GenericWriteAheadSink re-scalable. In essence, what it does is that it replaces

[GitHub] flink issue #2707: [FLINK-4939] GenericWriteAheadSink: Decouple the creating...

2016-11-04 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2707 @zentol let me know if you have any more comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink pull request #2707: [FLINK-4939] GenericWriteAheadSink: Decouple the c...

2016-11-04 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2707#discussion_r86561350 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericWriteAheadSink.java --- @@ -77,111 +84,144 @@ public

[GitHub] flink pull request #2707: [FLINK-4939] GenericWriteAheadSink: Decouple the c...

2016-11-04 Thread kl0u
Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/2707#discussion_r86561357 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/GenericWriteAheadSink.java --- @@ -77,111 +84,144 @@ public

[GitHub] flink issue #2747: [FLINK-4960] Allow the AbstractStreamOperatorTestHarness ...

2016-11-04 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2747 Thanks @aljoscha . Will do that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #2747: [FLINK-4960] Allow the AbstractStreamOperatorTestH...

2016-11-04 Thread kl0u
Github user kl0u closed the pull request at: https://github.com/apache/flink/pull/2747 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #2747: [FLINK-4960] Allow the AbstractStreamOperatorTestHarness ...

2016-11-03 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2747 @aljoscha thanks for the comments. I integrated them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] flink issue #2707: [FLINK-4939] GenericWriteAheadSink: Decouple the creating...

2016-11-03 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2707 Hi @zentol thanks for the review. I suppose that the comment that it does not work, implies that it does not work when rescaling right? Not in general. If I am right, then this is expected. As you can

[GitHub] flink issue #2747: [FLINK-4960] Allow the AbstractStreamOperatorTestHarness ...

2016-11-03 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2747 R: @aljoscha --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #2747: [FLINK-4960] Allow the AbstractStreamOperatorTestH...

2016-11-03 Thread kl0u
GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/2747 [FLINK-4960] Allow the AbstractStreamOperatorTestHarness to test scaling down As the title describes, this PR just adds a method in the `AbstractStreamOperatorTestHarness` that allows multiple

[GitHub] flink issue #2718: [hotfix] Fixes the TimestampedInputSplit.EOS comparison.

2016-10-31 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2718 Thanks @mxm ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #2702: [FLINK-4942] Improve processing performance of HeapIntern...

2016-10-28 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2702 Are we sure that this is going to lead to improvements? I understand that both cases have pros and cons, as in one case we pay the price at each arriving timer, while in the other you pay it at once

[GitHub] flink issue #2718: [hotfix] Fixes the TimestampedInputSplit.EOS comparison.

2016-10-28 Thread kl0u
Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2718 @mxm I agree and I am planning to do it as soon as we fix some other design issues with the ContinuousFileReadOperator that concern rescaling. Thanks for the review. --- If your project is set up

<    2   3   4   5   6   7   8   9   10   >