vinishjail97 commented on code in PR #772:
URL: https://github.com/apache/incubator-xtable/pull/772#discussion_r3494552617


##########
xtable-core/src/main/java/org/apache/xtable/hudi/HudiConversionTarget.java:
##########
@@ -607,7 +635,15 @@ private HoodieWriteConfig getWriteConfig(
               HoodieMetadataConfig.newBuilder()
                   .enable(true)
                   .withProperties(properties)
-                  .withMetadataIndexColumnStats(true)
+                  // Hudi 1.2.0 couples the partition-stats index to the 
column-stats index. For
+                  // partitioned tables the
+                  // partition-stats generation path rebuilds a file-system 
view over the committed
+                  // external
+                  // parquet files and groups them by fileId; XTable's 
externally-registered files
+                  // have non-Hudi names
+                  // whose fileId cannot be parsed once the "_hudiext" marker 
is stripped, leading
+                  // to failures.

Review Comment:
   Thank you — addressed in 42bba3ad. I've reflowed the comment into clean 
lines and pointed it at a dedicated tracking issue (#832) that explains why the 
column-stats index is currently gated to un-partitioned tables.



##########
xtable-core/src/main/java/org/apache/xtable/hudi/HudiTableManager.java:
##########
@@ -54,6 +56,10 @@ public class HudiTableManager {
       "org.apache.hudi.keygen.TimestampBasedKeyGenerator";
   private static final String COMPLEX_KEY_GENERATOR = 
"org.apache.hudi.keygen.ComplexKeyGenerator";
   private static final String SIMPLE_KEY_GENERATOR = 
"org.apache.hudi.keygen.SimpleKeyGenerator";
+  // Hudi 1.x spark query defaults to "default" database and spark read query 
picks up delta
+  // instead, 0.x doesn't have the same problem.
+  // TODO: https://app.clickup.com/t/18029943/ENG-23339
+  private static final String DEFAULT_DATABASE_NAME = "default_hudi";

Review Comment:
   Good catch — removed in 42bba3ad. The `DEFAULT_DATABASE_NAME` constant and 
the `setDatabaseName(...)` call are both gone, so newly created tables fall 
back to Hudi's default database, matching the pre-upgrade behaviour on `main`. 
I re-ran `TestHudiTableManager`, `TestHudiConversionTarget` and 
`ITHudiConversionTarget` after the change and they all pass.



##########
xtable-hudi-support/xtable-hudi-support-extensions/pom.xml:
##########
@@ -26,7 +26,7 @@
         <version>0.2.0-SNAPSHOT</version>
     </parent>
 
-    
<artifactId>xtable-hudi-support-extensions_${scala.binary.version}</artifactId>
+    <artifactId>xtable-hudi-support-extensions_2.12</artifactId>

Review Comment:
   Agreed — reverted in 42bba3ad. The module artifactId is back to 
`xtable-hudi-support-extensions_${scala.binary.version}` so it stays tied to 
the build-time Scala version.



##########
xtable-hudi-support/xtable-hudi-support-extensions/pom.xml:
##########
@@ -38,7 +38,7 @@
 
         <dependency>
             <groupId>org.apache.xtable</groupId>
-            <artifactId>xtable-core_${scala.binary.version}</artifactId>
+            <artifactId>xtable-core_2.12</artifactId>

Review Comment:
   Done in 42bba3ad — the `xtable-core` dependencies are back to 
`xtable-core_${scala.binary.version}` as well (both the compile dependency and 
the test-jar entry).



##########
xtable-core/src/main/java/org/apache/xtable/hudi/HudiTableManager.java:
##########
@@ -102,11 +109,15 @@ HoodieTableMetaClient initializeHudiTable(String 
tableDataPath, InternalTable ta
     boolean hiveStylePartitioningEnabled =
         DataLayoutStrategy.HIVE_STYLE_PARTITION == table.getLayoutStrategy();
     try {
-      return HoodieTableMetaClient.withPropertyBuilder()
+      return HoodieTableMetaClient.newTableBuilder()
           .setCommitTimezone(HoodieTimelineTimeZone.UTC)
           .setHiveStylePartitioningEnable(hiveStylePartitioningEnabled)
           .setTableType(HoodieTableType.COPY_ON_WRITE)
+          // Pin new tables to table version 6 for now. Table version 9 (Hudi 
1.x) support will be

Review Comment:
   Added in 42bba3ad — the v6-pin comment now links the umbrella issue #762 
(Hudi version upgrade), under which the table-version-9 follow-up work is 
tracked.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to