Dear community,
Nice to share Hudi community weekly update for 2020-01-28 ~ 2020-02-03 with updates on development, features, improvement, tests. The community is mainly focus on the releasing 0.5.1-incubating(including some docs update) this week. Development [Release] Apache Hudi(incubating) 0.5.1 has been released, the version 0.5.1-incubating-rc1 has reviewed and voted on the release candidate #1 for the version 0.5.1. All information about the release is available on hudi release web page. [1] [Disscussion] A disscussion about suggestion for removing HoodieWriteClient, based on the great degree of usage for HoodieSparkSource. The disscussion conclusion is that datasource and deltastreamer are all built on top of the HoodieWriteClient. [2] [1] https://lists.apache.org/thread.html/r00a103880bc2722ed9b5d729b4cea06ad0d45fe2abb1eaf7b2f59f71%40%3Cdev.hudi.apache.org%3E [2] https://lists.apache.org/thread.html/rbd77e7e95aa31a6813c096f5473d27528b5e86b5e1712e48f5bef8eb%40%3Cdev.hudi.apache.org%3E Features [Hudi DAG] Include custom names for spark HUDI spark DAG stages for easier understanding, providing descriptive names for the stages which makes it easier to visualize the HUDI DAG. [3] [3] https://issues.apache.org/jira/browse/HUDI-92 Improvement [Spark Integration] Relocate spark-avro dependency by maven-shade-plugin, spark-avro module is not included with spark-shell by default. [4] [Utilities] Optimize the steps of building with scala 2.12 version, for dev/change-scala-version.sh execution. [5] [4] https://issues.apache.org/jira/browse/HUDI-584 [5] https://issues.apache.org/jira/browse/HUDI-585 Tests [Unit Test] Improve unit test coverage for org.apache.hudi.common.table.HoodieTimeline, added new test cases for class HoodieTimeline, HoodieDefaultTimeline and HoodieActiveTimeline. [6] [Testing] Support spark version upgrade and fix the test failure on hudi_test_suite_refactor branch, for the reason that Avro dependency has been a bulit-in module in Spark. [7] [Jacoco Coverage] Fix generation of jacococ coverage reports after unit tests are run, surefire plugin's argLine is moved into a property. [8] [Test Coverage] Increase test coverage for HoodieReadClient, add tests for methods that have no coverage, increased coverage to 98% from 45%. [9] [6] https://issues.apache.org/jira/browse/HUDI-566 [7] https://issues.apache.org/jira/browse/HUDI-587 [8] https://issues.apache.org/jira/browse/HUDI-591 [9] https://issues.apache.org/jira/browse/HUDI-594 Best, Nicholas
