Davis Zhang created HUDI-8914:
---------------------------------
Summary: Test coverage for table creation
Key: HUDI-8914
URL: https://issues.apache.org/jira/browse/HUDI-8914
Project: Apache Hudi
Issue Type: Test
Reporter: Davis Zhang
[follow up of
https://github.com/apache/hudi/pull/12615|https://github.com/apache/hudi/pull/12615]
goal for this patch (open to splitting it up into multiple):
* for new table creation, we should default to table version 8.
* for new table creation, if user overrides "hoodie.write.table.version", we
will honor the overridden value.
We need to validate and may be fix all writers (spark-ds, spark-sql,
deltastreamer and spark structured streaming ingestion)
|Deltastreamer :
[hudi/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java#L421]
Line 421 in
[ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||HoodieTableMetaClient initializeEmptyTable(HoodieTableMetaClient.TableBuilder
tableBuilder, String partitionColumns,|
[hudi/hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java#L428]
Line 428 in
[ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||public void testTableCreation() throws Exception {|
spark-sql and spark-ds: guess this patch already have tests.
spark-streaming:
[hudi/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestStructuredStreaming.scala|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestStructuredStreaming.scala#L208]
Line 208 in
[ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||def testStructuredStreaming(tableType: HoodieTableType): Unit = {|
|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
