rahil-c opened a new pull request, #7397: URL: https://github.com/apache/hudi/pull/7397
### Change Logs * Upgrade flink to 1.16 in Hudi #### Main Changes for this upgrade * When doing a hudi flink upgrade, similar to spark upgrades, we must also copy over the module from the latest version and then make changes to resolve incompatibilies. When starting this upgrade I copied over the `hudi-flink-1.15` module and renamed it to `hudi-flink-1.16` and then proceeded with making changes. Therefore the code under `[hudi-flink1.16.x/..` is essientially the same as 1.15 and has been duplicated for sake of keeping in line with a upgrade version ( only minor pom changes has been made to this new module) * The following three interface methods are new to `1.16`, introduced in this flink pr https://github.com/apache/flink/pull/20299. https://github.com/apache/flink/blob/release-1.16.0/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/OperatorCoordinator.java https://github.com/apache/flink/blob/release-1.15.1/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/OperatorCoordinator.java ``` @Override public void executionAttemptFailed(int i, int attemptNumber, @Nullable Throwable throwable) { // no operation } @Override public void executionAttemptReady(int i, int attemptNumber, SubtaskGateway subtaskGateway) { // no operation } @Override public void handleEventFromOperator(int i, int attemptNumber, OperatorEvent operatorEvent) throws Exception { // no operation } ``` * these new methods are renames of the 1.15 methods such as `subtaskFailed,subtaskReady,etc` but take in an additional parameter `attemptNumber`. In my implementations of these I opted to not use this additional param as in the flink code I saw certain implementations not using this. --- https://github.com/apache/flink/blob/release-1.15.1/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/sort/SortCodeGenerator.scala https://github.com/apache/flink/blob/release-1.16.0/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/sort/SortCodeGenerator.scala * In flink 1.16 `class SortCodeGenerator( tableConfig: ReadableConfig, classLoader: ClassLoader, val input: RowType, val sortSpec: SortSpec)` now has an additional parameter for `classLoader` ---- * For all the test classes that use the method `handleEventFromOperator` i have set the `attemptNumber` to `0` . This does not seem ideal to me but when checking in the flink github most of the tests set this param to 0. ( Will check with hudi flink committer on this) ### Impact High ### Risk level (write none, low medium or high below) medium risk, but users can still use other flink versions. _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update Docs will need to be updated ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
