curcur commented on pull request #11725: URL: https://github.com/apache/flink/pull/11725#issuecomment-622771835
@AHeise and @pnowojski , I have updated the PR with respect to the API change suggested above. The API change is listed as the fourth commit "[FLINK-15670][connector] Kafka Shuffle API Part". The updated PR also includes amends suggested by Arvid in previous reviews. I left a reply for comments not yet resolved. Now, most of the code change is wrapped in the package "org.apache.flink.streaming.connectors.kafka.shuffle" to avoid effects on other parts. 1. I think now the only thing a bit struggling is "TwoPhaseCommitFunction". If I do not provide a watermark entry in that abstract function, I will end up exposing "currentTransactionHolder" as well as "TransactionHolder#handle". Your call. 2. I divide the "persistentKeyBy" to two functions: "writeKeyBy" and "readKeyBy". Hence people can call "readKeyBy" directly to reuse the written data. You can take a look at the API to see whether the changes make sense. Thanks in advance!! ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
