Dear community, Nice to share Hudi community bi-weekly updates for 2021-03-14 ~ 2021-03-28 with updates on features, bug fixes and tests.
======================================= Features [Flink Integration] Tweak hudi-flink-bundle module pom and reorganize the pacakges for hudi-flink module [1] [Flink Integration] Bounded source for stream writer [2] [Metadata Table] Improve performance of key lookups from base file in Metadata Table [3] [Core] Added locking capability to allow multiple writers [4] [Core] Implement HoodieTableSource.explainSource for all kinds of pushing down [5] [Flink Integration] Use PRIMARY KEY syntax to define record keys for Flink Hudi table [6] [Spark Integration] hudi write should uncache rdd, when the write operation is finnished [7] [Core] Add support for composite keys in NonpartitionedKeyGenerator [8] [Flink Integration] Flush as per data bucket for mini-batch write [9] [Core] Custom avro kafka deserializer [10] [Core] Improving config names and adding hive metastore uri config [11] [Flink Integration] Read optimized query type for Flink batch reader [12] [Core] Rename & standardize config to match other configs [13] [Flink Integration] Bump Flink version to 1.12.2 [14] [Java Client] Introduce HoodieBloomIndex to hudi-java-client [15] [1] https://issues.apache.org/jira/browse/HUDI-1684 [2] https://issues.apache.org/jira/browse/HUDI-1692 [3] https://issues.apache.org/jira/browse/HUDI-1552 [4] https://issues.apache.org/jira/browse/HUDI-845 [5] https://issues.apache.org/jira/browse/HUDI-1701 [6] https://issues.apache.org/jira/browse/HUDI-1688 [7] https://issues.apache.org/jira/browse/HUDI-1663 [8] https://issues.apache.org/jira/browse/HUDI-1653 [9] https://issues.apache.org/jira/browse/HUDI-1705 [10] https://issues.apache.org/jira/browse/HUDI-1650 [11] https://issues.apache.org/jira/browse/HUDI-1709 [12] https://issues.apache.org/jira/browse/HUDI-1710 [13] https://issues.apache.org/jira/browse/HUDI-1712 [14] https://issues.apache.org/jira/browse/HUDI-1495 [15] https://issues.apache.org/jira/browse/HUDI-1478 ======================================= Bugs [GCS] Fixing input stream detection of GCS FileSystem [1] [Core] Fixing null schema in bulk_insert row writer path [2] [Core] Fixing spark3 bundles [3] [Core] Fix a null value related bug for spark vectorized reader. [4] [Core] Fix MethodNotFound for HiveMetastore Locks [5] [1] https://issues.apache.org/jira/browse/HUDI-1496 [2] https://issues.apache.org/jira/browse/HUDI-1615 [3] https://issues.apache.org/jira/browse/HUDI-1568 [4] https://issues.apache.org/jira/browse/HUDI-1667 [5] https://issues.apache.org/jira/browse/HUDI-1728 Best, Leesf