Dear community, Nice to share Hudi community weekly update for 2020-05-17 ~ 2020-05-24 with updates on features, discussion, bug fix and tests.
======================================= Discussion [Writer Core] A discussion about supporting log append scenario with better write and asynchronous compaction, hudi now could lookup the index and merge small files even inserting now records, it makes the insert a little bit slow. [1] [Writer Core] A discussion to write failed records to .hoodie/errors/ folder, and expose the metrics as well for monitoring purpose. [2] [Hudi Usage] If you are using hudi, please feel free to comment in here. [3] ======================================= Features [Writer Core] Reduce listing .hoodie folder when using timeline server [4] [Index] Adding Simple Index to Hoodie [5] [Writer Core] Allow multiple operations to be executed within a single commit [6] [Cleaner] Enable Incremental cleaning and embedded timeline-server by default [7] [Metrics] Implement datadog metrics reporter [8] ======================================= Bugs [Spark Integration] Nested structs containing decimal types lead to null pointer exception. [9] [Hive Integration] fix NullPointerException in HoodieCompactor [10] ======================================= Tests [Test] Add unit test for RollbacksCommand [11] [Test] Add unit test for SavepointsCommand[12] [Test] Improve Unit test coverage of HoodieAvroUtils around default values [13] [Test] Add unit test for StatsCommand [14] [1] https://lists.apache.org/thread.html/rb292ff49a2a23dcdd380f9b4b34898aba86f7eb7450b7f282a786571%40%3Cdev.hudi.apache.org%3E [2] https://lists.apache.org/thread.html/r55da626d1a682dbc14ac1f0ad6511f3a14b9540d32ec0e6c118b1b03%40%3Cdev.hudi.apache.org%3E [3] https://github.com/apache/incubator-hudi/issues/661 [4] https://jira.apache.org/jira/browse/HUDI-895 [5] https://jira.apache.org/jira/browse/HUDI-407 [6] https://jira.apache.org/jira/browse/HUDI-858 [7] https://jira.apache.org/jira/browse/HUDI-846 [8] https://jira.apache.org/jira/browse/HUDI-836 [9] https://jira.apache.org/jira/browse/HUDI-863 [10] https://jira.apache.org/jira/browse/HUDI-888 [11] https://jira.apache.org/jira/browse/HUDI-705 [12] https://jira.apache.org/jira/browse/HUDI-706 [13] https://jira.apache.org/jira/browse/HUDI-803 [14] https://jira.apache.org/jira/browse/HUDI-707 Best, Leesf
