[GitHub] flink issue #5823: [FLINK-9008] [e2e] Implements quickstarts end to end test

2018-05-18 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5823 Thanks @zhangminglei. This looks good to merge, will proceed to merge this. ---

[GitHub] flink issue #5374: [FLINK-8101][flink-connectors] Elasticsearch 5.3+ (Transp...

2018-05-18 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5374 @cjolif since we have now reached a conclusion on where the Elasticsearch connector should be improved in the future, could you maybe close this PR? I assume a new PR will be opened that subsumes

[GitHub] flink issue #6040: [FLINK-9349][Kafka Connector] KafkaConnector Exception wh...

2018-05-18 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6040 @snuyanzin the failing `YARNSessionCapacitySchedulerITCase` is known to be bit flaky, so you can safely ignore that for now. I'll take another look at your changes soon. Thanks! ---

[GitHub] flink pull request #6021: [FLINK-9374] [kinesis] Enable FlinkKinesisProducer...

2018-05-18 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6021#discussion_r189177308 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java --- @@ -326,6

[GitHub] flink issue #5941: [FLINK-8971] [e2e] Include broadcast / union state in gen...

2018-05-17 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5941 I think it is safe to merge this change. Will merge this now .. ---

[GitHub] flink issue #5955: [FLINK-8659] Add migration itcases for broadcast state.

2018-05-17 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5955 @kl0u yes, lets do that as a separate commit then. +1, this looks good to me. One final comment for the merge: When merging to `master`, we should have test savepoints for both

[GitHub] flink pull request #6021: [FLINK-9374] [kinesis] Enable FlinkKinesisProducer...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6021#discussion_r189163394 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java --- @@ -218,6

[GitHub] flink pull request #6021: [FLINK-9374] [kinesis] Enable FlinkKinesisProducer...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6021#discussion_r189164871 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java --- @@ -326,6

[GitHub] flink issue #6040: [FLINK-9349][Kafka Connector] KafkaConnector Exception wh...

2018-05-17 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6040 Thanks for the PR @snuyanzin! I had some comments, please let me know what you think. Also, some general contribution tips: 1. I would suggest the title of the PR to be something

[GitHub] flink pull request #6040: [FLINK-9349][Kafka Connector] KafkaConnector Excep...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6040#discussion_r189031751 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/test/java/org/apache/flink/streaming/connectors/kafka/internal/Flink9349Test.java --- @@ -0,0

[GitHub] flink pull request #6040: [FLINK-9349][Kafka Connector] KafkaConnector Excep...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6040#discussion_r189029077 --- Diff: flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/internals/AbstractFetcher.java

[GitHub] flink pull request #6040: [FLINK-9349][Kafka Connector] KafkaConnector Excep...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6040#discussion_r189031075 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/test/java/org/apache/flink/streaming/connectors/kafka/internal/Flink9349Test.java --- @@ -0,0

[GitHub] flink pull request #6040: [FLINK-9349][Kafka Connector] KafkaConnector Excep...

2018-05-17 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6040#discussion_r189031464 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/test/java/org/apache/flink/streaming/connectors/kafka/internal/Flink9349Test.java --- @@ -0,0

[GitHub] flink pull request #6038: [FLINK-9394] [e2e] Test rescaling when resuming fr...

2018-05-17 Thread tzulitai
GitHub user tzulitai opened a pull request: https://github.com/apache/flink/pull/6038 [FLINK-9394] [e2e] Test rescaling when resuming from externalized checkpoints ## What is the purpose of the change This PR further extends the `test_resume_externalized_checkpoints.sh

[GitHub] flink issue #6038: [FLINK-9394] [e2e] Test rescaling when resuming from exte...

2018-05-17 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6038 r: @StefanRRichter ---

[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6008#discussion_r188548653 --- Diff: flink-end-to-end-tests/test-scripts/test_resume_savepoint.sh --- @@ -33,8 +33,8 @@ else NUM_SLOTS=$NEW_DOP fi

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188511081 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188512949 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188511258 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188512862 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188512804 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #5955: [FLINK-8659] Add migration itcases for broadcast s...

2018-05-16 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5955#discussion_r188511280 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/StatefulJobWBroadcastStateMigrationITCase.java --- @@ -0,0 +1,418

[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/6008#discussion_r188507930 --- Diff: flink-end-to-end-tests/run-nightly-tests.sh --- @@ -43,121 +43,151 @@ EXIT_CODE=0 # printf &qu

[GitHub] flink issue #6008: [FLINK-9354][travis] Print execution times for nightly E2...

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6008 I see these in the Travis logs: ``` flink-end-to-end-tests/run-pre-commit-tests.sh: line 94: start_timer: command not found ... flink-end-to-end-tests/run-pre-commit-tests.sh: line

[GitHub] flink issue #6018: [FLINK-9372] Typo on Elasticsearch website link (elastic....

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6018 Thanks for catching this @medcv! Apparently elastic.io directs to a completely different website 😅. Merging this .. ---

[GitHub] flink pull request #5977: [FLINK-9295][kafka] Fix transactional.id collision...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5977#discussion_r188205000 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/functions/RuntimeContext.java --- @@ -71,6 +71,17 @@ @PublicEvolving

[GitHub] flink issue #6002: [FLINK-9350] Parameter baseInterval has wrong check messa...

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6002 LGTM, merging .. ---

[GitHub] flink issue #5984: [FLINK-9328][state] Fix RocksDBStateBackend restore probl...

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5984 Hi @Myasuka, it seems like your local branch is not properly rebased on the latest master. Could you try rebasing again and reopening the PR? ---

[GitHub] flink issue #6008: [FLINK-9354][travis] Print execution times for nightly E2...

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/6008 +1 LGTM ---

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188195971 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188195534 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188195999 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188196219 --- Diff: flink-yarn/src/main/scala/org/apache/flink/yarn/YarnJobManager.scala --- @@ -34,7 +34,6 @@ import

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188196115 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188194545 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188195349 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5823: [FLINK-9008] [e2e] Implements quickstarts end to e...

2018-05-15 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5823#discussion_r188194072 --- Diff: flink-end-to-end-tests/test-scripts/test_quickstarts.sh --- @@ -0,0 +1,123 @@ +#!/usr/bin/env bash

[GitHub] flink issue #5823: [FLINK-9008] [e2e] Implements quickstarts end to end test

2018-05-15 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5823 @medcv That sounds like something we should do as part of the release process (if we are going to do it), orthogonal to this PR here. ---

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187848661 --- Diff: flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch1.sh --- @@ -0,0 +1,66 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187847421 --- Diff: flink-end-to-end-tests/flink-elasticsearch1-test/src/main/java/org/apache/flink/elasticsearch1/test/Elasticsearch1SinkExample.java --- @@ -15,10

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187848008 --- Diff: flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch1.sh --- @@ -0,0 +1,66 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187847498 --- Diff: flink-end-to-end-tests/flink-elasticsearch1-test/src/main/java/org/apache/flink/elasticsearch1/test/Elasticsearch1SinkExample.java --- @@ -37,20

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187848355 --- Diff: flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch1.sh --- @@ -0,0 +1,66 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187847923 --- Diff: flink-end-to-end-tests/test-scripts/test_streaming_elasticsearch1.sh --- @@ -0,0 +1,66 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187847586 --- Diff: flink-end-to-end-tests/flink-elasticsearch2-test/pom.xml --- @@ -0,0 +1,140 @@ + + +http://maven.apache.org/POM/4.0.0

[GitHub] flink pull request #5761: [FLINK-8989] [e2eTests] Elasticsearch1&2&5 end to ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5761#discussion_r187847815 --- Diff: flink-end-to-end-tests/run-pre-commit-tests.sh --- @@ -53,6 +53,30 @@ if [ $EXIT_CODE == 0 ]; then EXIT_CODE=$? fi

[GitHub] flink pull request #5676: [FLINK-8910][tests] Automated end-to-end test for ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5676#discussion_r187845478 --- Diff: flink-end-to-end-tests/flink-local-recovery-and-allocation-test/src/main/java/org/apache/flink/streaming/tests

[GitHub] flink pull request #5676: [FLINK-8910][tests] Automated end-to-end test for ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5676#discussion_r187843886 --- Diff: flink-end-to-end-tests/test-scripts/test_local_recovery_and_scheduling.sh --- @@ -0,0 +1,115 @@ +#!/usr/bin/env bash

[GitHub] flink pull request #5676: [FLINK-8910][tests] Automated end-to-end test for ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5676#discussion_r187842776 --- Diff: flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java --- @@ -140,7 +140,8 @@ public static final

[GitHub] flink pull request #5676: [FLINK-8910][tests] Automated end-to-end test for ...

2018-05-14 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5676#discussion_r187844104 --- Diff: flink-end-to-end-tests/test-scripts/test_local_recovery_and_scheduling.sh --- @@ -0,0 +1,115 @@ +#!/usr/bin/env bash

[GitHub] flink issue #5969: [FLINK-9074] [e2e] Add e2e for resuming from externalized...

2018-05-13 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5969 Thanks a lot for the review @fhueske. Will merge this! ---

[GitHub] flink pull request #6004: [FLINK-8977] [e2e] End-to-end test for manual job ...

2018-05-13 Thread tzulitai
GitHub user tzulitai opened a pull request: https://github.com/apache/flink/pull/6004 [FLINK-8977] [e2e] End-to-end test for manual job resume after terminal failure ## What is the purpose of the change This PR is based on new e2e features introduced by #5941, #5990

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788278 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxy.java --- @@ -382,50

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788338 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxy.java --- @@ -353,19

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788213 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.java

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788363 --- Diff: flink-connectors/flink-connector-kinesis/src/test/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxyTest.java --- @@ -70,20

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788223 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.java

[GitHub] flink pull request #5992: [FLINK-8944] [Kinesis Connector] Use listShards in...

2018-05-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5992#discussion_r187788284 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxy.java --- @@ -382,50

[GitHub] flink issue #5929: [FLINK-8497] [connectors] KafkaConsumer throws NPE if top...

2018-05-11 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5929 @alexpf thanks for the updates. Please bear with me just a little, I'll try to get back to reviewing this PR ASAP after I finish with some ongoing tasks. ---

[GitHub] flink pull request #5990: [FLINK-9322][FLINK-9320] [e2e] Improvements to e2e...

2018-05-11 Thread tzulitai
GitHub user tzulitai opened a pull request: https://github.com/apache/flink/pull/5990 [FLINK-9322][FLINK-9320] [e2e] Improvements to e2e standalone chaos monkey test ## What is the purpose of the change This PR is based on #5941. Only the last 2 commits are relevant

[GitHub] flink pull request #5969: [FLINK-9074] [e2e] Add e2e for resuming from exter...

2018-05-09 Thread tzulitai
GitHub user tzulitai opened a pull request: https://github.com/apache/flink/pull/5969 [FLINK-9074] [e2e] Add e2e for resuming from externalized checkpoints ## What is the purpose of the change This PR adds an end-to-end test for resuming from externalized, retained

[GitHub] flink issue #5941: [FLINK-8971] [e2e] Include broadcast / union state in gen...

2018-05-08 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5941 @StefanRRichter the PR is ready for another review, thanks! ---

[GitHub] flink issue #5926: [FLINK-9073] [e2e-tests] Extend savepoint e2e tests for d...

2018-05-07 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5926 Thanks for the review @StefanRRichter! Will address your comment and merge this. ---

[GitHub] flink pull request #5926: [FLINK-9073] [e2e-tests] Extend savepoint e2e test...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5926#discussion_r186608526 --- Diff: flink-end-to-end-tests/run-nightly-tests.sh --- @@ -58,25 +58,97 @@ fi if [ $EXIT_CODE == 0 ]; then printf &qu

[GitHub] flink issue #5958: [FLINK-8500] Get the timestamp of the Kafka message from ...

2018-05-07 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5958 Thanks for the update @FredTing. I'll try to take another look at the PR within the next days. ---

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186606104 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/DeserializationSchema.java --- @@ -42,14 +42,22 @@ @Public public

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186341972 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186341885 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186341059 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186340944 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186342816 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186340110 --- Diff: docs/dev/stream/state/index.md --- @@ -49,6 +49,7 @@ Where to go next? - * [Working with State](state.html): Shows

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186339587 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -25,27 +25,25 @@ under the License. * ToC {:toc} -[Working with State

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186342376 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5922: [FLINK-8780] [docs] Add Broadcast State documentat...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5922#discussion_r186341293 --- Diff: docs/dev/stream/state/broadcast_state.md --- @@ -0,0 +1,279 @@ +--- +title: "The Broadcast State Pattern" +nav

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186338002 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/DeserializationSchema.java --- @@ -42,14 +42,22 @@ @Public public

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186337736 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/ConsumerRecordMetaInfo.java --- @@ -0,0 +1,73 @@ +/* + * Licensed

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186337890 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/ConsumerRecordMetaInfo.java --- @@ -0,0 +1,73 @@ +/* + * Licensed

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186338049 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/DeserializationSchema.java --- @@ -42,14 +42,22 @@ @Public public

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186337834 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/ConsumerRecordMetaInfo.java --- @@ -0,0 +1,73 @@ +/* + * Licensed

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186338721 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/serialization/DeserializationSchema.java --- @@ -42,14 +42,22 @@ @Public public

[GitHub] flink pull request #5958: [FLINK-8500] Get the timestamp of the Kafka messag...

2018-05-07 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5958#discussion_r186337301 --- Diff: flink-connectors/flink-connector-kafka-0.10/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka010Fetcher.java --- @@ -78,6

[GitHub] flink issue #5006: [hotfix][docs][QS] MInor cleanup of QS documentation

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5006 +1, LGTM. Will address Greg's last comment and merge this .. ---

[GitHub] flink issue #5956: [hotfix][javadocs] Correct the javadoc of StandaloneCheck...

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5956 LGTM, thanks @klion26. Merging this .. ---

[GitHub] flink issue #5890: [FLINK-8999] [e2e] Ensure the job has an operator with op...

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5890 Hi @zhangminglei, I think the current general purpose DataStream job already subsumes this PR; for example, the sequence source generator in the job already uses operator state. Moreover

[GitHub] flink issue #5918: [hotfix][ListCheckpointed]modify parallelism in the annot...

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5918 Thanks @maqingxiang! This LGTM, will merge this, thanks. ---

[GitHub] flink issue #5952: [FLINK-9287][kafka] Properly clean up resources in non EX...

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5952 As discussed offline, both approaches I mentioned wouldn't seem to work. I'll merge as is, let's keep an eye on it to see if the test is flaky. ---

[GitHub] flink issue #5950: [FLINK-9169] [state-backend] Allow absence of old seriali...

2018-05-06 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5950 Thanks for the review Stefan! Will merge this now .. ---

[GitHub] flink issue #5952: [FLINK-9287][kafka] Properly clean up resources in non EX...

2018-05-04 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5952 Would filtering the thread count by thread name be helpful here? Another possible approach to test this: Maybe we can simply verify that the current transaction producer is closed after

[GitHub] flink issue #5950: [FLINK-9169] [state-backend] Allow absence of old seriali...

2018-05-04 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5950 @StefanRRichter I have updated the PR. Also had to do a rebase due to conflicts. Regarding the thoughts you brought up: - Bubble up `UnloadableTypeSerializerException` approach: I

[GitHub] flink pull request #5950: [FLINK-9169] [state-backend] Allow absence of old ...

2018-05-04 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5950#discussion_r185994375 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializerSerializationUtil.java --- @@ -200,7 +197,7 @@ public static void

[GitHub] flink pull request #5950: [FLINK-9169] [state-backend] Allow absence of old ...

2018-05-04 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5950#discussion_r185995278 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializerSerializationUtil.java --- @@ -200,7 +197,7 @@ public static void

[GitHub] flink pull request #5950: [FLINK-9169] [state-backend] Allow absence of old ...

2018-05-04 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5950#discussion_r185994454 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializerSerializationUtil.java --- @@ -200,7 +197,7 @@ public static void

[GitHub] flink issue #5950: [FLINK-9169] [state-backend] Allow absence of old seriali...

2018-05-03 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5950 I think even with the `UnloadableTypeSerializerException` exception bubbling approach, we actually still need a flag in the serialization proxy to decide how to handle the exception

[GitHub] flink issue #5950: [FLINK-9169] [state-backend] Allow absence of old seriali...

2018-05-03 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5950 @StefanRRichter yes, now that you mentioned it, the `isSerializerPresenceRequiredFlag` does seem a bit awkward to be in the serialization proxy. Essentially, what it is only doing is serving

[GitHub] flink pull request #5929: [FLINK-8497] [connectors] KafkaConsumer throws NPE...

2018-05-03 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5929#discussion_r185759080 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09PartitionDiscoverer.java

[GitHub] flink issue #5939: [FLINK-8500] [Kafka Connector] Get the timestamp of the K...

2018-05-03 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5939 @StephanEwen we have some follow-up discussion on how to approach this on the JIRA ticket. Do you want to take a look at that? https://issues.apache.org/jira/browse/FLINK-8500 The general

[GitHub] flink pull request #5929: [FLINK-8497] [connectors] KafkaConsumer throws NPE...

2018-05-03 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5929#discussion_r185732772 --- Diff: flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09PartitionDiscoverer.java

[GitHub] flink pull request #5945: [FLINK-9169] [runtime] Allow KeyedBackendSerializa...

2018-05-03 Thread tzulitai
Github user tzulitai closed the pull request at: https://github.com/apache/flink/pull/5945 ---

[GitHub] flink issue #5945: [FLINK-9169] [runtime] Allow KeyedBackendSerializationPro...

2018-05-03 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/5945 I've opened a new PR #5950 which has a cleaner approach to this. That new PR subsumes this one. ---

<    1   2   3   4   5   6   7   8   9   10   >