pvary opened a new pull request, #23587: URL: https://github.com/apache/flink/pull/23587
## What is the purpose of the change This is the implementation of [FLIP-372](https://cwiki.apache.org/confluence/display/FLINK/FLIP-372%3A+Allow+TwoPhaseCommittingSink+WithPreCommitTopology+to+alter+the+type+of+the+Committable) Add a possibility to the WithPreCommitTopology to transform not only the data, but the type of the data channelled through it. Please only check the last commit. The previous commits have an open pull request. We can rebase once they are merged. ## Brief change log - Added `TwoPhaseCommittingSinkWithPreCommitTopology` to decouple the writer response type from the committer input type - Updated `TwoPhaseCommittingSink` to be compatible with the previous implementation, but the interface is inherited from `TwoPhaseCommittingSinkWithPreCommitTopology` - Changed `WithPostCommitTopology` and `WithPreCommitTopology` to be a mixin like interfaces (no inheritance from `TwoPhaseCommittingSink`) - Changed the `SinkTransformationTranslator` to handle the new interfaces - Updated old test to use the new `TwoPhaseCommittingSinkWithPreCommitTopology` where it was relevant - Created new tests in `StreamingJobGraphGeneratorTest` to check if we can create sinks with `TwoPhaseCommittingSinkWithPreCommitTopology` and `TwoPhaseCommittingSink` too ## Verifying this change Added new tests, and verified that the old tests are working ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? not documented at the moment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
