Dear community, Nice to share Hudi community bi-weekly updates for 2022-03-14 ~ 2022-03-27 with updates on features, bug fixes.
======================================= Features [Core] Rebase Data Skipping flow to rely on MT Column Stats index [1] [Flink] Support backend switch in HoodieFlinkStreamer [2] [Flink] Support flink multiple versions [3] [Core] Provide an option to trigger clean every nth commit [4] [Flink] Flink bulk_insert support bucket hash index [5] [Core] Supporting Composite Expressions over Data Table Columns in Data Skipping flow [6] [1] https://issues.apache.org/jira/browse/HUDI-3514 [2] https://issues.apache.org/jira/browse/HUDI-3607 [3] https://issues.apache.org/jira/browse/HUDI-3665 [4] https://issues.apache.org/jira/browse/HUDI-1436 [5] https://issues.apache.org/jira/browse/HUDI-3701 [6] https://issues.apache.org/jira/browse/HUDI-3594 ======================================= Bugs [Flink] flink sync hive metadata supports table properties and serde properties [1] [Core] Automatically adjust write configs based on metadata table and write concurrency mode [2] [Core] Replace RDD with HoodieData in HoodieSparkTable and commit executors [3] [Core] Refactored Spark DataSource Relations to avoid code duplication [4] [Core] Fixing Column Stats index to properly handle first Data Table commit [5] [Core] Refactor hive sync tool / config to use reflection and standardize configs [6] [Core] Refactoring MergeOnReadRDD to avoid duplication, fetch only projected columns [7] [Core] Do not throw exception when instant to rollback does not exist in metadata table active timeline [8] [Core] OOM occurred when use bulk_insert cow table with flink BUCKET index [9] [1] https://issues.apache.org/jira/browse/HUDI-3589 [2] https://issues.apache.org/jira/browse/HUDI-3404 [3] https://issues.apache.org/jira/browse/HUDI-2439 [4] https://issues.apache.org/jira/browse/HUDI-3457 [5] https://issues.apache.org/jira/browse/HUDI-3663 [6] https://issues.apache.org/jira/browse/HUDI-2883 [7] https://issues.apache.org/jira/browse/HUDI-3396 [8] https://issues.apache.org/jira/browse/HUDI-3435 [9] https://issues.apache.org/jira/browse/HUDI-3716 Best, Leesf
