[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2509 Perfect! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Thanks! If do you happen to find inappropriate changes in `FlinkKafkaConsumerBaseMigrationTest`, please let me know, will be happy to discuss and fix it :-) Merging this to `master` now ..

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2509 Thanks @tzulitai and @rmetzger ! Of course, feel free to proceed with this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Thanks for letting us know @kl0u! Yes, there are other pending PRs based on this. I just double checked the changes in `FlinkKafkaConsumerBaseMigrationTest` myself, and I think that

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2509 @tzulitai how does this fit your timeline. Are there PRs depending this or is this PR blocking your in any way? If so, I would propose that we merge it right away. --- If your project is set

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user kl0u commented on the issue: https://github.com/apache/flink/pull/2509 Hi @tzulitai and @rmetzger . I did not have time so far to look into it. I hope I will be able to do it till the end of the week. Is this ok? --- If your project is set up for it, you can reply to

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Last commit addresses consolidating the deserliazer settings for `KafkaOffsetHandlerImpl`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Thanks for the review @rmetzger! The final 2 commits have addressed all your comments. I'll also wait for @kl0u to have a look at the changes in `FlinkKafkaConsumerBaseMigration` before

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2509 I've tested the change locally and with great success. So once my comments are addressed, the change is good to be merged --- If your project is set up for it, you can reply to this email and have

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Note about 3rd commit: fixed failing `FlinkKafkaConsumerBaseMigrationTest`s after the rebase. The tests were failing due to the removal of `AbstractFetcher#restoreOffsets(...)` method as

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2509 Thanks a lot! I have your Kafka pull requests on my todo list. I hope I get to it soon. I'm really sorry. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Rebased on the "flink-connectors" change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Hi @rmetzger, I've addressed all comments. I'll leave comments inline of code on parts that addresses your more bigger comments, to help with the second-pass review. --- If your project is set up

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Thanks for the review @rmetzger :) I'll aim to address your comments and rebase by the end of this week (will tag you once it's ready). --- If your project is set up for it, you can reply to

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Rebased on recent Kafka consumer changes, fixed failing Kafka 0.10 exactly-once tests, and added integration tests (`testStartFromEarliestOffsets`, `testStartFromLatestOffsets`, and

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 @rmetzger @gyfora @koeninger Rebased this on the Kafka 0.10 connector and some other recent changes. This is ready for review now ;) I'd like to add tests for this after #2580, because #2580 adds a

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Rebasing + adding tests for the new functions now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2509 @tzulitai makes sense ! As for for the Map you are right, the multiple topic case slipped my mind :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2509 Thank you for working on this. I gave #2369 some love today to speed up things ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 I'll rebase this PR soon, probably will also wait for Kafka 0.10 connector to be merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Hi @gyfora, Yes, it is absolutely possible to add that. There's actually a JIRA for that feature too ([FLINK-3123](https://issues.apache.org/jira/browse/FLINK-3123)), so I'd say we can add

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2509 Hi, I like the proposed changes, do you think it would make sense to add the possibility to set specific offsets on a per partition basis? ``` kafka.setStartOffsets(Map