Dear community, Nice to share Hudi community weekly update for 2020-07-04 ~ 2020-07-12 with updates on discussion, features and tests.
======================================= Discussion [Writer Core] A discussion about concurrent writes, users are writing multi hudi tables for different parittions as a solution, and the community is looking into adding support for parallel writers in 0.6.0. [1] [CLI] A discussion about exposing HUDI CLI as a Service so that we would execute hudi commond via spring boot service. [2] [Community Scale] A discussion to code, config, design walk through sessions to help more users coming into the community [3] [Hive Integration] A question about keeping Hive in Sync [4] ======================================= Features [Writer Core] Remove unnecessary maxEvent check and add some log in KafkaOffsetGen [5] [Writer Core] Implemented custom key generator [6] [Writer Core] Added support for multiple input formats in TimestampBasedKeyGenerator [7] ======================================= Bugs [Metrics] Support update metrics in HoodieDeltaStreamerMetrics [8] [Hive Integration] Fix backward compatibility for com.uber inputformats [9] ======================================= Tests [Test] Add functional test suite for hudi-utilities [10] [Test] Remove duplicate assertNoWriteErrors() [11] [Test] Make TestHBaseQPSResourceAllocator a unit test [12] [1] https://lists.apache.org/thread.html/rf0bebebcedfabaddca3285287ed6788547ecd8950952159a5dfcb5c0%40%3Cdev.hudi.apache.org%3E [2] https://lists.apache.org/thread.html/r3ec35df5d7388130acae74b8562f2dd94dce006ecc43ceb79ea94a11%40%3Cdev.hudi.apache.org%3E [3] https://lists.apache.org/thread.html/r05d9a04b31f04a87d70ab15c003ce890f64f73f9c3a642c0911bd8a6%40%3Cdev.hudi.apache.org%3E [4] https://lists.apache.org/thread.html/r1c0132826960be81902b04f761235c99f03f009ed45a317a3d2ff5a7%40%3Cdev.hudi.apache.org%3E [5] https://issues.apache.org/jira/browse/HUDI-1062 [6] https://issues.apache.org/jira/browse/HUDI-728 [7] https://issues.apache.org/jira/browse/HUDI-916 [8] https://issues.apache.org/jira/browse/HUDI-1004 [9] https://issues.apache.org/jira/browse/HUDI-1080 [10] https://issues.apache.org/jira/browse/HUDI-996 [11] https://issues.apache.org/jira/browse/HUDI-1069 [12] https://issues.apache.org/jira/browse/HUDI-994 Best, Leesf
