lasdf1234 commented on code in PR #14659:
URL: https://github.com/apache/gravitino/pull/14659#discussion_r4509282906


##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -705,6 +707,7 @@ private static void applyColumnNotNull(
 
   private static void applyColumnDefaultValue(Column column, Capability 
capabilities) {
     applyColumnDefaultValue(column.name(), column.defaultValue(), 
capabilities);
+    applyColumnInitialDefaultValue(column.name(), 
column.initialDefaultValue());

Review Comment:
   Already covered by `testLoadThenCreateRejectsColumnsCarryingInitialDefault`: 
load-then-create with a set `initialDefaultValue` is intentionally rejected for 
now (create/write of initial defaults is unsupported). We can revisit ignoring 
the field later if we want copy-table to succeed.



##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -714,6 +717,19 @@ private static void applyColumnDefaultValue(
         capabilities.columnDefaultValue().unsupportedMessage() + " Illegal 
column: " + columnName);
   }
 
+  /**
+   * Rejects setting {@code initialDefaultValue} on create/add-column. 
Catalogs that support write
+   * defaults (e.g. MySQL) still do not apply Iceberg-style initial defaults, 
so reusing {@link
+   * Capability#columnDefaultValue()} would accept a value the catalog 
silently drops. Until a
+   * catalog can write initial defaults, require the unset sentinel.
+   */
+  private static void applyColumnInitialDefaultValue(
+      String columnName, Expression initialDefaultValue) {
+    Preconditions.checkArgument(
+        INITIAL_DEFAULT_VALUE_NOT_SET.equals(initialDefaultValue),
+        "Setting column initial default value is not supported. Illegal 
column: " + columnName);

Review Comment:
   Updated the error message to state that `initialDefaultValue` is read-only 
for now: catalogs may return it on load (e.g. Iceberg v3), but setting it on 
create/add-column is not supported by any catalog capability yet.



##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ConvertUtil.java:
##########
@@ -76,9 +89,112 @@ public static IcebergColumn 
fromNestedField(Types.NestedField nestedField) {
         .withNullable(nestedField.isOptional())
         .withComment(nestedField.doc())
         
.withType(IcebergDataTypeConverter.CONVERTER.toGravitino(nestedField.type()))
+        .withDefaultValue(
+            fromIcebergLiteral(
+                nestedField.writeDefaultLiteral(),
+                nestedField.type(),
+                Column.DEFAULT_VALUE_NOT_SET))
+        .withInitialDefaultValue(
+            fromIcebergLiteral(
+                nestedField.initialDefaultLiteral(),
+                nestedField.type(),
+                Column.INITIAL_DEFAULT_VALUE_NOT_SET))
         .build();
   }
 
+  private static Expression fromIcebergLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    if (icebergDefault == null) {
+      return unset;
+    }
+    return toGravitinoLiteral(icebergDefault, type, unset);
+  }
+
+  private static Expression toGravitinoLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    Object value = icebergDefault.value();
+    if (value == null) {
+      return Literals.NULL;
+    }
+
+    switch (type.typeId()) {
+      case BOOLEAN:
+        return Literals.booleanLiteral((Boolean) value);
+      case INTEGER:
+        return Literals.integerLiteral((Integer) value);
+      case LONG:
+        return Literals.longLiteral((Long) value);
+      case FLOAT:
+        return Literals.floatLiteral((Float) value);
+      case DOUBLE:
+        return Literals.doubleLiteral((Double) value);
+      case STRING:
+        return Literals.stringLiteral(value.toString());
+      case DATE:
+        return Literals.dateLiteral(DateTimeUtil.dateFromDays((Integer) 
value));
+      case TIME:
+        return Literals.timeLiteral(DateTimeUtil.timeFromMicros((Long) value));
+      case TIMESTAMP:
+        {
+          Types.TimestampType timestampType = (Types.TimestampType) type;
+          if (timestampType.shouldAdjustToUTC()) {
+            // Iceberg timestamptz is a UTC instant (micros). Expose it as UTC 
LocalDateTime while
+            // keeping TimestampType.withTimeZone so the type still says "with 
time zone".
+            return Literals.of(
+                DateTimeUtil.timestamptzFromMicros((Long) 
value).toLocalDateTime(),

Review Comment:
   Yes, this is the intended convention. Gravitino timestamp literals use 
`LocalDateTime` and put timezone-ness on `TimestampType.withTimeZone()`, so an 
Iceberg timestamptz UTC instant is exposed as that instant's UTC 
`LocalDateTime` with a with-time-zone type (clients should treat the value as 
UTC). Added `testFromNestedFieldTimestamptzDefaultsRoundTripThroughDtoJson` to 
cover ConvertUtil → ColumnDTO → JSON.



##########
maintenance/optimizer/build.gradle.kts:
##########
@@ -26,9 +26,11 @@ plugins {
 
 val scalaVersion: String =
   project.properties["scalaVersion"] as? String ?: 
extra["defaultScalaVersion"].toString()
-val sparkVersion: String = libs.versions.spark33.get()
+// Match maintenance/jobs and catalog Iceberg 1.11 so IsolatedClassLoader's 
shared
+// org.apache.iceberg.* types include NestedField default APIs used by 
lakehouse-iceberg.
+val sparkVersion: String = libs.versions.spark35.get()

Review Comment:
   Required here because CI fails without it. Lakehouse now calls Iceberg 1.11 
`NestedField.writeDefaultLiteral()` / `initialDefaultLiteral()` on table load. 
Optimizer ITs load the lakehouse catalog through `IsolatedClassLoader`, where 
`org.apache.iceberg.*` is shared from the test classpath — staying on Spark 3.3 
pulls Iceberg 1.8.1 and those methods are missing (`NoSuchMethodError`). See 
the comment above `sparkVersion` in this file. A separate PR only works if it 
lands first; this change alone would leave optimizer CI red.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to