Dear community, Nice to share Hudi community weekly update for 2020-02-03 ~ 2020-02-09 with updates on development, features and bug fix.
Development [Code Cleanup] A discussion to unify Hudi code cleanup and improvement has reached a consensus, and the PR to judge where a PR is MINOR has been merged.[1] [Relocate dependency] A disscussion to relocate spark-avro dependency by maven-shade-plugin is ongoing, you would chime in and give your thought.[2] [Hudi & Beam Integration] A disscussion about where Hudi DAG would be written in Apache Beam is brought up, to make hudi compatible with Beam.[3] [Transformer Enhancement] A disscussion to refactor and enhance Hudi Transformer is brought up to make Transformer more universal and flexible, please chime in if you are interest.[4] [Hudi Usage] A ML to report Hudi usage, and please report the usage in [5], and we will add it to powered-by page[7] [Slack and Jira Integration] A disscussion to integrate Jira in slack is brought up, and after discussion, we will send thread in #general channel to ML[8] [FAQs] A disscussion to add common errors and solutions to FAQs is brought up so that new folks would find solutions in the section first and improve hudi when look back at the common errors.[9] [1] https://lists.apache.org/thread.html/rc77546c7de7b2fa40330f889ed39dec7c0ab335e30e93d25745508ef%40%3Cdev.hudi.apache.org%3E [2] https://lists.apache.org/thread.html/r6bd85074eea3e807cde44ce9944c30966aa18c95e69b7bc9596d6398%40%3Cdev.hudi.apache.org%3E [3] https://lists.apache.org/thread.html/redcfecab2e2aba82760d86ccd9f8ef8d1cabaedaf21c20827b4edb50%40%3Cdev.hudi.apache.org%3E [4] https://lists.apache.org/thread.html/rfad2e71fc432922ca567432b7b6e1dd9c3bb102822177b73dbff2d90%40%3Cdev.hudi.apache.org%3E [5] https://lists.apache.org/thread.html/ra82d304380e3212533a533e9e3e16be0b17d39747000231a3c9a4adc%40%3Cdev.hudi.apache.org%3E [6] https://github.com/apache/incubator-hudi/issues/661 [7] https://lists.apache.org/thread.html/ra82d304380e3212533a533e9e3e16be0b17d39747000231a3c9a4adc%40%3Cdev.hudi.apache.org%3E [8] https://lists.apache.org/thread.html/r0575d916663f826a5078363ec913c53360afb372471061aa60fd380c%40%3Cdev.hudi.apache.org%3E [9] https://lists.apache.org/thread.html/r33559c5bb584cac24b2aa0b4f1cf6d42c883af07dd37f1099da991dd%40%3Cdev.hudi.apache.org%3E Features [Utilities] Optimize the steps of building with scala-2.12[10] [INDEX] Allow partition path to be updated with GLOBAL_BLOOM index[11] [CLI] Add 'commits show archived' command to CLI[12] [10] https://issues.apache.org/jira/browse/HUDI-585 [11] https://issues.apache.org/jira/browse/HUDI-499 [12] https://issues.apache.org/jira/browse/HUDI-571 Bugs [Utilities] Close KafkaConsumer every time[13] [13] https://issues.apache.org/jira/browse/HUDI-596 Best, Leesf
