[GitHub] flink issue #4456: [FLINK-7343][kafka] Increase Xmx for tests

2017-08-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4456 @NicoK sounds reasonable however this will drop `-UseGCOverheadLimit` flag. It should be fine when at the same time we increase the `Xmx`. Btw, having `UseGCOverheadLimit` seems a little bit fishy

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r131330545 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -51,39 +48,52

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r131332988 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,311

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r131334260 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,311

[GitHub] flink issue #4454: [hotfix][docs] Add section in docs about writing unit/int...

2017-08-04 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4454 @tzulitai maybe I will add a link to `TwoPhaseCommitSinkFunctionTest` for how to test `exactly-once`? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4454#discussion_r131342944 --- Diff: docs/dev/testing.md --- @@ -0,0 +1,102 @@ +--- +title: "Testing" +nav-parent_id: dev +na

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4454#discussion_r131337294 --- Diff: docs/dev/testing.md --- @@ -0,0 +1,102 @@ +--- +title: "Testing" +nav-parent_id: dev +na

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4454#discussion_r131343119 --- Diff: docs/dev/testing.md --- @@ -0,0 +1,102 @@ +--- +title: "Testing" +nav-parent_id: dev +na

[GitHub] flink pull request #4454: [hotfix][docs] Add section in docs about writing u...

2017-08-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4454#discussion_r131344197 --- Diff: docs/dev/testing.md --- @@ -0,0 +1,102 @@ +--- +title: "Testing" +nav-parent_id: dev +na

[GitHub] flink pull request #4470: [FLINK-7343] Simulate network failures in kafka at...

2017-08-03 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4470 [FLINK-7343] Simulate network failures in kafka at-least-once test We shouldn't fail KafkaServers directly, because they might not be able to flush the data (`log.flush.interval.***` properties

[GitHub] flink pull request #4515: Logging improvements in NetworkFailuresProxy

2017-08-15 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4515#discussion_r133231386 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/networking/NetworkFailureHandler.java --- @@ -92,7 +92,14 @@ public

[GitHub] flink pull request #4515: Logging improvements in NetworkFailuresProxy

2017-08-10 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4515 Logging improvements in NetworkFailuresProxy This PR consists of only one minor bug fix and one improvement in logging of `NetworkFailuresProxy` class used in tests. It's highly unlikely

[GitHub] flink pull request #4515: Logging improvements in NetworkFailuresProxy

2017-08-11 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4515#discussion_r132638260 --- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/networking/NetworkFailureHandler.java --- @@ -92,7 +92,14 @@ public

[GitHub] flink issue #4239: [FLINK-6988] flink-connector-kafka-0.11 with exactly-once...

2017-08-10 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4239 Indeed it seems like you are right. `read_committed` doesn't play along with long `max.transaction.timeout.ms`. I'm not sure about Beam, but in Flink we can not use one single `transactional.id

[GitHub] flink issue #4239: [FLINK-6988] flink-connector-kafka-0.11 with exactly-once...

2017-08-10 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4239 This solution (basically a pool with a fixed size of 2) would work, only if there would be at most one pending commit transaction. Which is not always true in Flink - there can be multiple

[GitHub] flink pull request #4558: [FLINK-7056][tests][hotfix] make sure the client a...

2017-08-17 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4558#discussion_r133674886 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobClientTest.java --- @@ -139,30 +139,35 @@ private static BlobKey prepareTestFile

[GitHub] flink pull request #4557: [hotifx][streaming] Simplify state of TwoPhaseComm...

2017-08-17 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4557 [hotifx][streaming] Simplify state of TwoPhaseCommitSinkFunction This simplify state of `TwoPhaseCommitSinkFunction`, by dropping unnecessary complication. This change is already covered

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-13 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 Good catch with with this spinning, I missed that. Checking per each iteration for assigned partitions is unfortunately costly, because there is no cheap `isEmpty()` method. The one that I

[GitHub] flink issue #4323: [FLINK-7175] Add first simplest Flink benchmark

2017-07-13 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4323 Thanks for pointing this out. Indeed we will probably have to move it to separate project in that case. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-13 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 Hmm, will blocking operation be appropriate here? This would prevent `shutdown()` from actually breaking the loop. I think we would need some timeout here? --- If your project is set up

[GitHub] flink pull request #4323: [FLINK-7175] Add first simplest Flink benchmark

2017-07-13 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4323 [FLINK-7175] Add first simplest Flink benchmark Example output: ``` Benchmark (objectReuse) (parallelism) (stateBackend) Mode Cnt Score Error

[GitHub] flink pull request #4255: [FLINK-7102] improve ClassLoaderITCase

2017-07-14 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4255#discussion_r127444872 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -137,6 +137,20 @@ public void

[GitHub] flink pull request #4323: [FLINK-7175] Add first simplest Flink benchmark

2017-07-17 Thread pnowojski
Github user pnowojski closed the pull request at: https://github.com/apache/flink/pull/4323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink issue #4323: [FLINK-7175] Add first simplest Flink benchmark

2017-07-17 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4323 Closing this PR. Benchmarks will be in separate repository. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #4310: [misc] Commit read offsets in Kafka integration te...

2017-07-12 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4310 [misc] Commit read offsets in Kafka integration tests Previously offsets were not committed so the same records could be read more then once. It was not a big issue, because so far

[GitHub] flink issue #4310: [misc] Commit read offsets in Kafka integration tests

2017-07-12 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4310 Yes and as far as I know, we are doing this. Why do you ask? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2....

2017-07-13 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4321 [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1 This patch fixes also an incompatibility with the latest Kafka 0.10.x and 0.11.x kafka-clients. You can merge this pull request into a Git

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-14 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 @tzulitai could you look at this PR and particularly into last commit (fixup). I'm not a big fan of mocks and mockito based tests and I would really be inclined to just drop this test

[GitHub] flink pull request #3672: [Flink-6072] TestCase CheckpointStateRestoreTest::...

2017-07-17 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/3672#discussion_r127641370 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/SubtaskState.java --- @@ -20,6 +20,7 @@ import

[GitHub] flink pull request #3672: [Flink-6072] TestCase CheckpointStateRestoreTest::...

2017-07-17 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/3672#discussion_r127641085 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointStateRestoreTest.java --- @@ -66,10 +66,19 @@ public void

[GitHub] flink pull request #3672: [Flink-6072] TestCase CheckpointStateRestoreTest::...

2017-07-17 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/3672#discussion_r127641519 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java --- @@ -949,8 +949,6 @@ public void run

[GitHub] flink issue #4240: [FLINK-7058] Fix scala-2.10 dependencies

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4240 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4255: [FLINK-7102] improve ClassLoaderITCase

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4255 ok, I missed that you didn't fail if the expected exception didn't happen :) Now it's LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink issue #4310: [misc] Commit read offsets in Kafka integration tests

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4310 I didn't know that you can have disconnected graph in Flink :) It shouldn't be caused by this commit, since it is included in my other PR. Rebased and let's make sure that it passes

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 -- IMO begin Mockito tests tends to repeat the implementation. Instead of testing for the effect, they tend to do the same thing as the actual code but in backwards. In other words, they have

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 I have added unit test for closing. I think it should be triggered/tested in one of the `ITCases`, but test is fairly easy so it shouldn't hurt us to have this tested explicitly. --- If your

[GitHub] flink issue #4310: [misc] Commit read offsets in Kafka integration tests

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4310 For consumer side or mapper side it is natural to use that kind of validating mappers, because you could just add them at the end of your pipeline. For producers tests it isn't, because

[GitHub] flink issue #4239: [FLINK-6988] flink-connector-kafka-0.11 with exactly-once...

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4239 `df6d5e0` to `5ff8106` should definitely be squashed, I left them only to make it easier for reviewers to follow the changes made in 0.11 vs 0.10 connectors (those changes would be invisible

[GitHub] flink pull request #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2....

2017-07-19 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4321#discussion_r128193424 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/test/java/org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThreadTest.java

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 I have also squashed previous fixups - there is only a new one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-21 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 Dropping this field would make it more error prone in the future if anyone would call `reassingPartitions()` from somewhere else. For me `hasAssignedPartitions` is tightly related to the state

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129777841 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java --- @@ -722,12 +697,30 @@ public ApplicationReport

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129771505 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java --- @@ -478,12 +485,15 @@ public static void runInteractiveCli

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129769550 --- Diff: flink-clients/src/main/java/org/apache/flink/client/cli/DefaultCLI.java --- @@ -58,7 +58,7 @@ public void addGeneralOptions(Options baseOptions

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129769520 --- Diff: flink-clients/src/main/java/org/apache/flink/client/cli/DefaultCLI.java --- @@ -77,10 +77,10 @@ public StandaloneClusterClient retrieveCluster

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129770778 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnCLI.java --- @@ -214,22 +218,21 @@ public void addGeneralOptions(Options baseOptions

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129774438 --- Diff: flink-yarn/src/test/java/org/apache/flink/yarn/YarnClusterDescriptorTest.java --- @@ -210,6 +207,7 @@ public void

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129774051 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java --- @@ -163,23 +161,17 @@ public

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129768986 --- Diff: flink-clients/src/main/java/org/apache/flink/client/cli/CustomCommandLine.java --- @@ -62,25 +62,27 @@ * Retrieves a client

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129772021 --- Diff: flink-yarn-tests/src/test/java/org/apache/flink/yarn/YarnTestBase.java --- @@ -549,7 +559,12 @@ protected void runWithArgs(String[] args, String

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129770769 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnCLI.java --- @@ -214,22 +218,21 @@ public void addGeneralOptions(Options baseOptions

[GitHub] flink pull request #4280: [FLINK-7125] [yarn] Remove Configuration loading f...

2017-07-27 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4280#discussion_r129881025 --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java --- @@ -478,12 +485,15 @@ public static void runInteractiveCli

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129531190 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129581412 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129523821 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129522314 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129531220 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-26 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4368#discussion_r129527454 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,342

[GitHub] flink issue #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-25 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4368 @EronWright, there shouldn't be any difficulties with that. I think that there is only one functional difference between `TwoPhaseCommitSourceFunction` and `PravegaWriter` - the first one

[GitHub] flink pull request #4368: [FLINK-7210] Introduce TwoPhaseCommitSinkFunction

2017-07-19 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4368 [FLINK-7210] Introduce TwoPhaseCommitSinkFunction This is intended to be a recommended base class for implementing exactly-once sinks in Flink You can merge this pull request into a Git

[GitHub] flink pull request #4239: [FLINK-6988] flink-connector-kafka-0.11 with exact...

2017-07-19 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4239#discussion_r128180978 --- Diff: flink-connectors/flink-connector-kafka-0.11/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer011.java --- @@ -0,0

[GitHub] flink pull request #4239: [FLINK-6988] flink-connector-kafka-0.11 with exact...

2017-07-19 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4239#discussion_r128201074 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/TwoPhaseCommitSinkFunction.java --- @@ -0,0 +1,317

[GitHub] flink issue #4386: (backport-1.3) [FLINK-7174] [kafka] Bump Kafka 0.10 depen...

2017-07-24 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4386 Looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4310: [misc] Commit read offsets in Kafka integration tests

2017-07-24 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4310 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4321: [FLINK-7174] Bump Kafka 0.10 dependency to 0.10.2.1

2017-07-24 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4321 Thanks @tzulitai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4239: [FLINK-6988] flink-connector-kafka-0.11 with exact...

2017-07-19 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4239#discussion_r128271792 --- Diff: flink-connectors/flink-connector-kafka-0.11/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer011.java --- @@ -0,0

[GitHub] flink issue #4310: [misc] Commit read offsets in Kafka integration tests

2017-07-19 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4310 yes, Travis passes @tzulitai :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #4209: [FLINK-7030] Build with scala-2.11 by default

2017-06-28 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4209 rebased and fixed conflict --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-06-29 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4213 Ops, sorry. I also have bundled 3.3.9 maven version with 2017.1.4 Intellij. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-06-29 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4213 I have the latest version of the Intellij (build in June), and it uses bundled Maven 3.5.9, while my local maven is 3.5.0. For me it replicates 100% times. Touch the pom.xml of some module

[GitHub] flink pull request #4213: [FLINK-7032] Overwrite inherited properties from p...

2017-06-28 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4213 [FLINK-7032] Overwrite inherited properties from parent pom You can merge this pull request into a Git repository by running: $ git pull https://github.com/pnowojski/flink java17

[GitHub] flink issue #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarantee at-l...

2017-06-30 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4206 Hmm, this is disturbing. Locally it works for me always. I have rewritten test so that it should be less prone to intermittent failures (longer reading from Kafka timeout). Hopefully

[GitHub] flink issue #4209: [FLINK-7030] Build with scala-2.11 by default

2017-06-30 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4209 @zentol build is passing :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4239: [FLINK-6988] Initial flink-connector-kafka-0.11 wi...

2017-06-30 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4239 [FLINK-6988] Initial flink-connector-kafka-0.11 with at-least-once semantic Couple of first commits are from other PRs #4206 #4209 #4213 You can merge this pull request into a Git repository

[GitHub] flink pull request #4240: [FLINK-7058] Fix scala-2.10 dependencies

2017-06-30 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4240 [FLINK-7058] Fix scala-2.10 dependencies First commit is from #4209 and should be ignored in this PR Before fixup: ``` $ mvn dependency:tree -pl flink-scala | grep quasi

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-07-04 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4213 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarantee at-l...

2017-07-04 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4206 @tzulitai applied changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarant...

2017-07-04 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4206#discussion_r125416757 --- Diff: flink-connectors/flink-connector-kafka-base/src/test/java/org/apache/flink/streaming/connectors/kafka/KafkaProducerTestBase.java --- @@ -172,6

[GitHub] flink pull request #4247: [FLINK-7066] Fix integration tests in airplane mod...

2017-07-03 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4247 [FLINK-7066] Fix integration tests in airplane mode You can merge this pull request into a Git repository by running: $ git pull https://github.com/pnowojski/flink airplane Alternatively

[GitHub] flink issue #4209: [FLINK-7030] Build with scala-2.11 by default

2017-07-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4209 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4240: [FLINK-7058] Fix scala-2.10 dependencies

2017-07-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4240 @zentol rebased --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-07-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4213 I have added one more sentence to the comment inside the pom. @zentol commit message is already quite verbose, but I have improved commit title. --- If your project is set up for it, you can

[GitHub] flink pull request #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarant...

2017-07-03 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4206#discussion_r125306972 --- Diff: flink-connectors/flink-connector-kafka-base/src/test/java/org/apache/flink/streaming/connectors/kafka/KafkaProducerTestBase.java --- @@ -172,6

[GitHub] flink issue #4247: [FLINK-7066] Fix integration tests in airplane mode

2017-07-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4247 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarantee at-l...

2017-07-03 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4206 Sorry, those tests were passing before rebase - after rebase I have accidentally reverted this previous fixup. I have re-introduced it now as a separate commit --- If your project is set up

[GitHub] flink issue #4206: [FLINK-6996] FlinkKafkaProducer010 doesn't guarantee at-l...

2017-07-06 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4206 thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4146: [FLINK-6008][blob] collection of BlobServer improv...

2017-06-29 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4146#discussion_r124727496 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobClient.java --- @@ -593,9 +594,7 @@ private void sendPutHeader(OutputStream

[GitHub] flink pull request #4146: [FLINK-6008][blob] collection of BlobServer improv...

2017-06-29 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4146#discussion_r124727900 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobServerConnection.java --- @@ -509,21 +509,7 @@ private void delete(InputStream

[GitHub] flink pull request #4146: [FLINK-6008][blob] collection of BlobServer improv...

2017-06-29 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4146#discussion_r124728498 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java --- @@ -227,7 +227,7 @@ static File getStorageLocation(File storageDir

[GitHub] flink pull request #4146: [FLINK-6008][blob] collection of BlobServer improv...

2017-06-29 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4146#discussion_r124730014 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobServerPutTest.java --- @@ -59,6 +60,110 @@ private final Random rnd

[GitHub] flink pull request #4209: [FLINK-7030] Build with scala-2.11 by default

2017-06-28 Thread pnowojski
GitHub user pnowojski opened a pull request: https://github.com/apache/flink/pull/4209 [FLINK-7030] Build with scala-2.11 by default You can merge this pull request into a Git repository by running: $ git pull https://github.com/pnowojski/flink scala211 Alternatively you can

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-06-29 Thread pnowojski
Github user pnowojski commented on the issue: https://github.com/apache/flink/pull/4213 Yes. Should I not do this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124544540 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124551917 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124544570 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124544513 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124545656 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124547387 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -70,7 +79,8 @@ * Algorithms must explicitly

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124547670 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -305,6 +334,41 @@ public static void main(String

[GitHub] flink pull request #4170: [FLINK-6358] [gelly] Write job details for Gelly e...

2017-06-28 Thread pnowojski
Github user pnowojski commented on a diff in the pull request: https://github.com/apache/flink/pull/4170#discussion_r124544307 --- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java --- @@ -103,6 +113,18 @@ .addClass

  1   2   3   4   5   6   7   8   9   10   >