Dear community, Happy new year. Nice to share Hudi community bi-weekly updates for 2021-12-19 ~ 2022-01-02 with updates on features, bug fixes.
======================================= Features [Core] Add table option to set utc timezone [1] [Spark] APurge drop partition for spark sql [2] [Spark] Support Spark 3.2 [3] [Flink] Support component data types for flink bulk_insert [4] [Spark] Add bucket hash index, compatible with the hive bucket [5] [1] https://issues.apache.org/jira/browse/HUDI-3014 [2] https://issues.apache.org/jira/browse/HUDI-3099 [3] https://issues.apache.org/jira/browse/HUDI-2811 [4] https://issues.apache.org/jira/browse/HUDI-3083 [5] https://issues.apache.org/jira/browse/HUDI-1951 ======================================= Bugs [Core] Fixing HoodieFileIndex partition column parsing for nested fields [1] [Core] Do not clean the log files right after compaction for metadata table [2] [Flink] Schedule Flink compaction in service [3] [Core] Adding ability to read entire data with HoodieIncrSource with empty checkpoint [4] [Core] drop table for spark sql [5] [Core] Excluding compaction instants from pending rollback info [6] [Core] Do not store rollback plan in inflight instant [7] [Core] Fixing AvroDFSSource does not use the overridden schema to deserialize Avro binaries [8] [Core] fix spark-sql query table that write with TimestampBasedKeyGenerator [9] [Core] Fix HiveSyncTool not sync schema [10] [Core] Fix the exception 'Not an Avro data file' when archive and clean [11] [Flink] Bootstrap when timeline have completed instant [12] [Flink] Cache compactionPlan in buffer [13] [Core] abstract partition filter logic to enable code reuse [14] [Core] Fix HiveSyncTool drop partitions using JDBC or hivesql or hms [15] [Spark] Fix insert error after adding columns on Spark 3.2.0 [16] [Spark] Fix merge/insert/show partitions error on Spark3.2 [17] [Spark] fix ctas error in spark3.1.1 [18] [1] https://issues.apache.org/jira/browse/HUDI-3008 [2] https://issues.apache.org/jira/browse/HUDI-3032 [3] https://issues.apache.org/jira/browse/HUDI-2547 [4] https://issues.apache.org/jira/browse/HUDI-3011 [5] https://issues.apache.org/jira/browse/HUDI-3060 [6] https://issues.apache.org/jira/browse/HUDI-3101 [7] https://issues.apache.org/jira/browse/HUDI-3102 [8] https://issues.apache.org/jira/browse/HUDI-2374 [9] https://issues.apache.org/jira/browse/HUDI-3093 [10] https://issues.apache.org/jira/browse/HUDI-3106 [11] https://issues.apache.org/jira/browse/HUDI-2675 [12] https://issues.apache.org/jira/browse/HUDI-3124 [13] https://issues.apache.org/jira/browse/HUDI-3120 [14] https://issues.apache.org/jira/browse/HUDI-3095 [15] https://issues.apache.org/jira/browse/HUDI-3107 [16] https://issues.apache.org/jira/browse/HUDI-3134 [17] https://issues.apache.org/jira/browse/HUDI-3136 [18] https://issues.apache.org/jira/browse/HUDI-3131 Best, Leesf
