Dear community, Nice to share Hudi community weekly update for 2020-05-10 ~ 2020-05-17 with updates on graduation, features, discussion, bug fix and tests.
======================================= Discussion [Graduation] A voting thread to graduate Hudi to Apache TLP has been started, please chime in and vote, click the below link and reply. [1] [Release] A discussion about doing a 0.5.3 patch set release has been approved, and a bug fix version will be released next. [2] [Writer Core] A discussion about supporting log append scenario with better write and asynchronous compaction, it aims to lower the write enlarge and process small files[3] [Core] An idea about enhancing preCombine API for Mongo Oplog integration, which wants to pass schema argument to preCombine API . [4] ======================================= Features [Multi Engine] Remove spark context in ClientUtils and HoodieIndex [5] [Multi Engine] Replace part of spark context by hadoop configuration in HoodieTable [6] [Multi Engine] Replace jsc.hadoopConfiguration by hadoop configuration in hudi-client testcase [7] [Multi Engine] Replace part of spark context by hadoop configuration in AbstractHoodieClient and HoodieReadClient [8] [FileSystem] Add support for alluxio [9] [Hive Integration] Writer supports useJdbc configuration when hive synchronization is enabled [10] [Writer Core] Add ability to specify time unit for TimestampBasedKeyGenerator [11] ======================================= Bugs [CLI] cleaner repair command should only inspect clean metadata files. [13] [Hive Integration] Shade avro and parquet-avro in hudi-hive-sync-bundle [8] [Writer Core] Adding proper default to hudi metadata fields and proper handling to rewrite routine [14] [Spark Integration] Handle empty commit in incremental pulling [15] [Deltastreamer] Register avro schema if infered from SQL transformation [16] [Deltastreamer] Avoid exception when getSchemaProvider [17] ======================================= Tests [Test] Restructure test packages [19] [Test] Add unit test for HDFSParquetImportCommand [20] [1] https://lists.apache.org/thread.html/r8039c8eece636df8c81a24c26965f5c1556a3c6404de02912d6455b4%40%3Cgeneral.incubator.apache.org%3E [2] https://lists.apache.org/thread.html/r1fb5ad5547f55f40b20306dac90a711c9c0e29f6855f63b6b2118987%40%3Cdev.hudi.apache.org%3E [3] https://lists.apache.org/thread.html/rb292ff49a2a23dcdd380f9b4b34898aba86f7eb7450b7f282a786571%40%3Cdev.hudi.apache.org%3E [4] https://lists.apache.org/thread.html/r6b676dd828def5493deed729007b745ff2e5f945943bdb66b50f22d9%40%3Cdev.hudi.apache.org%3E [5] https://issues.apache.org/jira/browse/HUDI-870 [6] https://issues.apache.org/jira/browse/HUDI-880 [7] https://issues.apache.org/jira/browse/HUDI-886 [8] https://issues.apache.org/jira/browse/HUDI-881 [9] https://issues.apache.org/jira/browse/HUDI-869 [10] https://issues.apache.org/jira/browse/HUDI-889 [11] https://issues.apache.org/jira/browse/HUDI-843 [12] https://issues.apache.org/jira/browse/HUDI-902 [13] https://issues.apache.org/jira/browse/HUDI-884 [14] https://issues.apache.org/jira/browse/HUDI-884 [15] https://issues.apache.org/jira/browse/HUDI-793 [16] https://issues.apache.org/jira/browse/HUDI-528 [17] https://issues.apache.org/jira/browse/HUDI-723 [18] https://issues.apache.org/jira/browse/HUDI-902 [19] https://issues.apache.org/jira/browse/HUDI-811 [20] https://issues.apache.org/jira/browse/HUDI-701 Best, Leesf
