jerryshao commented on code in PR #14659:
URL: https://github.com/apache/gravitino/pull/14659#discussion_r4509190121


##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -714,6 +717,19 @@ private static void applyColumnDefaultValue(
         capabilities.columnDefaultValue().unsupportedMessage() + " Illegal 
column: " + columnName);
   }
 
+  /**
+   * Rejects setting {@code initialDefaultValue} on create/add-column. 
Catalogs that support write
+   * defaults (e.g. MySQL) still do not apply Iceberg-style initial defaults, 
so reusing {@link
+   * Capability#columnDefaultValue()} would accept a value the catalog 
silently drops. Until a
+   * catalog can write initial defaults, require the unset sentinel.
+   */
+  private static void applyColumnInitialDefaultValue(
+      String columnName, Expression initialDefaultValue) {
+    Preconditions.checkArgument(
+        INITIAL_DEFAULT_VALUE_NOT_SET.equals(initialDefaultValue),
+        "Setting column initial default value is not supported. Illegal 
column: " + columnName);

Review Comment:
   The error message is only visible to the caller and does not say which 
catalogs would support it. Consider naming the catalog capability so users know 
the value is read-only for now.



##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ConvertUtil.java:
##########
@@ -76,9 +89,112 @@ public static IcebergColumn 
fromNestedField(Types.NestedField nestedField) {
         .withNullable(nestedField.isOptional())
         .withComment(nestedField.doc())
         
.withType(IcebergDataTypeConverter.CONVERTER.toGravitino(nestedField.type()))
+        .withDefaultValue(
+            fromIcebergLiteral(
+                nestedField.writeDefaultLiteral(),
+                nestedField.type(),
+                Column.DEFAULT_VALUE_NOT_SET))
+        .withInitialDefaultValue(
+            fromIcebergLiteral(
+                nestedField.initialDefaultLiteral(),
+                nestedField.type(),
+                Column.INITIAL_DEFAULT_VALUE_NOT_SET))
         .build();
   }
 
+  private static Expression fromIcebergLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    if (icebergDefault == null) {
+      return unset;
+    }
+    return toGravitinoLiteral(icebergDefault, type, unset);
+  }
+
+  private static Expression toGravitinoLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    Object value = icebergDefault.value();
+    if (value == null) {
+      return Literals.NULL;
+    }
+
+    switch (type.typeId()) {
+      case BOOLEAN:
+        return Literals.booleanLiteral((Boolean) value);
+      case INTEGER:
+        return Literals.integerLiteral((Integer) value);
+      case LONG:
+        return Literals.longLiteral((Long) value);
+      case FLOAT:
+        return Literals.floatLiteral((Float) value);
+      case DOUBLE:
+        return Literals.doubleLiteral((Double) value);
+      case STRING:
+        return Literals.stringLiteral(value.toString());
+      case DATE:
+        return Literals.dateLiteral(DateTimeUtil.dateFromDays((Integer) 
value));
+      case TIME:
+        return Literals.timeLiteral(DateTimeUtil.timeFromMicros((Long) value));
+      case TIMESTAMP:
+        {
+          Types.TimestampType timestampType = (Types.TimestampType) type;
+          if (timestampType.shouldAdjustToUTC()) {
+            // Iceberg timestamptz is a UTC instant (micros). Expose it as UTC 
LocalDateTime while
+            // keeping TimestampType.withTimeZone so the type still says "with 
time zone".
+            return Literals.of(
+                DateTimeUtil.timestamptzFromMicros((Long) 
value).toLocalDateTime(),

Review Comment:
   timestamptz is converted to a UTC LocalDateTime but typed as with-time-zone, 
so the zone is lost and clients may read it in local time. Is this the intended 
wire convention, and is it tested end to end through the DTO?



##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -705,6 +707,7 @@ private static void applyColumnNotNull(
 
   private static void applyColumnDefaultValue(Column column, Capability 
capabilities) {
     applyColumnDefaultValue(column.name(), column.defaultValue(), 
capabilities);
+    applyColumnInitialDefaultValue(column.name(), 
column.initialDefaultValue());

Review Comment:
   Any column loaded from an Iceberg v3 table with an initial default now fails 
here when reused in createTable or addColumn, since this check rejects every 
non-unset value. Clients that copy loaded columns will break; consider ignoring 
the field for catalogs that do not support it, or add a test for this round 
trip.



##########
maintenance/optimizer/build.gradle.kts:
##########
@@ -26,9 +26,11 @@ plugins {
 
 val scalaVersion: String =
   project.properties["scalaVersion"] as? String ?: 
extra["defaultScalaVersion"].toString()
-val sparkVersion: String = libs.versions.spark33.get()
+// Match maintenance/jobs and catalog Iceberg 1.11 so IsolatedClassLoader's 
shared
+// org.apache.iceberg.* types include NestedField default APIs used by 
lakehouse-iceberg.
+val sparkVersion: String = libs.versions.spark35.get()

Review Comment:
   Switching the optimizer module from Spark 3.3 to Spark 3.5 looks unrelated 
to column defaults and can affect users of that module. Can this go in a 
separate PR, or can you explain why it is required here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to