Dear community, Nice to share Hudi community bi-weekly updates for 2021-08-15 ~ 2021-08-29 with updates on features, bug fixes and tests.
======================================= Features [Flink Integration] Support Flink batch upsert [1] [Core] AHoodie columns sort partitioner for bulk insert [2] [1] https://issues.apache.org/jira/browse/HUDI-2316 [2] https://issues.apache.org/jira/browse/HUDI-2345 ======================================= Bugs [Core] fix FileSliceMetrics utils bug [1] [Core] HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException [2] [Core] Include _hoodie_operation meta column in removeMetadataFields [3] [Core] Use correct meta columns while preparing dataset for bulk insert [4] [Spark Integraion] Create Table If Not Exists Failed After Alter Table [5] [Flink Integration] Merge the data set for flink bounded source when changelog mode turns off [6] [Flink Integration] Optimize Bootstrap operator [7] [Spark Integration] Support referencing subquery with column aliases by table alias [8] [Flink Integration] The upgrade downgrade action of flink writer should be singleton [9] [Spark Integration] MERGE INTO doesn't work for tables created using CTAS [10] [Core] Catch Throwable in BoundedInMemoryExecutor [11] [Core] Use the caller classloader for ReflectionUtils [12] [Core] Refact HoodieFlinkStreamer to reuse the pipeline of HoodieTableSink [13] [Core] Optimizing overwriteField method with Objects.equals [14] [Flink Integration] Improvement flink streaming reader [15] [1] https://issues.apache.org/jira/browse/HUDI-2301 [2] https://issues.apache.org/jira/browse/HUDI-2167 [3] https://issues.apache.org/jira/browse/HUDI-1363 [4] https://issues.apache.org/jira/browse/HUDI-2322 [5] https://issues.apache.org/jira/browse/HUDI-2339 [6] https://issues.apache.org/jira/browse/HUDI-2340 [7] https://issues.apache.org/jira/browse/HUDI-2342 [8] https://issues.apache.org/jira/browse/HUDI-2259 [9] https://issues.apache.org/jira/browse/HUDI-2352 [10] https://issues.apache.org/jira/browse/HUDI-2357 [11] https://issues.apache.org/jira/browse/HUDI-2368 [12] https://issues.apache.org/jira/browse/HUDI-2321 [13] https://issues.apache.org/jira/browse/HUDI-2229 [14] https://issues.apache.org/jira/browse/HUDI-2365 [15] https://issues.apache.org/jira/browse/HUDI-2371 ====================================== Tests [Tests] Adding spark delete node to integ test suite [1] [Tests] Add basic "hoodie_is_deleted" unit tests to TestDataSource classes [2] [Tests] Refactor HoodieSparkSqlWriterSuite to add setup and teardown [3] [1] https://issues.apache.org/jira/browse/HUDI-2349 [2] https://issues.apache.org/jira/browse/HUDI-2359 [3] https://issues.apache.org/jira/browse/HUDI-2264 Best, Leesf
