jerryshao commented on code in PR #14659:
URL: https://github.com/apache/gravitino/pull/14659#discussion_r4512593758


##########
common/src/main/java/org/apache/gravitino/dto/util/DTOConverters.java:
##########
@@ -1651,4 +1673,47 @@ public static PolicyContent fromDTO(PolicyContentDTO 
policyContentDTO) {
     throw new IllegalArgumentException(
         "Unsupported policy content type: " + policyContentDTO.getClass());
   }
+
+  private static boolean isBinaryOrFixed(Type dataType) {
+    return dataType instanceof Types.BinaryType || dataType instanceof 
Types.FixedType;
+  }
+
+  private static String literalValueToDtoString(Object value, Type dataType) {
+    // byte[].toString() is an object identity and cannot preserve the 
literal's bytes.
+    if (isBinaryOrFixed(dataType) && value instanceof byte[]) {
+      return BINARY_HEX_PREFIX + BINARY_HEX.encode((byte[]) value);
+    }
+    return value.toString();
+  }
+
+  /**
+   * Decodes {@code 0x}-prefixed hex for binary/fixed. Unprefixed values keep 
their previous string
+   * interpretation for provider-specific handling (#14609).
+   */
+  private static Object literalValueFromDtoString(String value, Type dataType) 
{
+    if (!isBinaryOrFixed(dataType) || value == null) {
+      return value;
+    }
+    if (!value.regionMatches(true, 0, BINARY_HEX_PREFIX, 0, 
BINARY_HEX_PREFIX.length())) {

Review Comment:
   Any binary/fixed default string starting with 0x (case-insensitive) is now 
decoded as hex for every catalog, so an existing JDBC default like '0xAB' 
silently changes meaning, and a non-hex value after the prefix now throws. 
Please scope the decoding to the Iceberg path or add a compatibility test for 
existing JDBC binary defaults.



##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -716,6 +718,7 @@ private static void applyColumnNotNull(
 
   private static void applyColumnDefaultValue(Column column, Capability 
capabilities) {
     applyColumnDefaultValue(column.name(), column.defaultValue(), 
capabilities);
+    applyColumnInitialDefaultValue(column.name(), 
column.initialDefaultValue());

Review Comment:
   This rejects any non-unset initial default wherever applyColumnDefaultValue 
runs. Please confirm it only runs on create/add-column and not on load or alter 
paths, otherwise a client that reads an Iceberg v3 column and sends it back 
will fail; add a test for that round trip.



##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ConvertUtil.java:
##########
@@ -76,9 +93,113 @@ public static IcebergColumn 
fromNestedField(Types.NestedField nestedField) {
         .withNullable(nestedField.isOptional())
         .withComment(nestedField.doc())
         
.withType(IcebergDataTypeConverter.CONVERTER.toGravitino(nestedField.type()))
+        .withDefaultValue(
+            fromIcebergLiteral(
+                nestedField.writeDefaultLiteral(),
+                nestedField.type(),
+                Column.DEFAULT_VALUE_NOT_SET))
+        .withInitialDefaultValue(
+            fromIcebergLiteral(
+                nestedField.initialDefaultLiteral(),
+                nestedField.type(),
+                Column.INITIAL_DEFAULT_VALUE_NOT_SET))
         .build();
   }
 
+  private static Expression fromIcebergLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    if (icebergDefault == null) {
+      return unset;
+    }
+    return toGravitinoLiteral(icebergDefault, type, unset);
+  }
+
+  private static Expression toGravitinoLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    Object value = icebergDefault.value();
+    if (value == null) {
+      return Literals.NULL;
+    }
+
+    switch (type.typeId()) {
+      case BOOLEAN:
+        return Literals.booleanLiteral((Boolean) value);
+      case INTEGER:
+        return Literals.integerLiteral((Integer) value);
+      case LONG:
+        return Literals.longLiteral((Long) value);
+      case FLOAT:
+        return Literals.floatLiteral((Float) value);
+      case DOUBLE:
+        return Literals.doubleLiteral((Double) value);
+      case STRING:
+        return Literals.stringLiteral(value.toString());
+      case DATE:
+        return Literals.dateLiteral(DateTimeUtil.dateFromDays((Integer) 
value));
+      case TIME:
+        return Literals.timeLiteral(DateTimeUtil.timeFromMicros((Long) value));
+      case TIMESTAMP:
+        {
+          Types.TimestampType timestampType = (Types.TimestampType) type;
+          if (timestampType.shouldAdjustToUTC()) {
+            // Iceberg timestamptz is a UTC instant (micros). Expose it as UTC 
LocalDateTime while
+            // keeping TimestampType.withTimeZone so the type still says "with 
time zone".
+            return Literals.of(
+                DateTimeUtil.timestamptzFromMicros((Long) 
value).toLocalDateTime(),

Review Comment:
   A timestamptz default is turned into a UTC LocalDateTime, so the instant is 
only recoverable by assuming UTC. Is that the intended contract for clients, 
and is it documented?



##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ConvertUtil.java:
##########
@@ -76,9 +93,113 @@ public static IcebergColumn 
fromNestedField(Types.NestedField nestedField) {
         .withNullable(nestedField.isOptional())
         .withComment(nestedField.doc())
         
.withType(IcebergDataTypeConverter.CONVERTER.toGravitino(nestedField.type()))
+        .withDefaultValue(
+            fromIcebergLiteral(
+                nestedField.writeDefaultLiteral(),
+                nestedField.type(),
+                Column.DEFAULT_VALUE_NOT_SET))
+        .withInitialDefaultValue(
+            fromIcebergLiteral(
+                nestedField.initialDefaultLiteral(),
+                nestedField.type(),
+                Column.INITIAL_DEFAULT_VALUE_NOT_SET))
         .build();
   }
 
+  private static Expression fromIcebergLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    if (icebergDefault == null) {
+      return unset;
+    }
+    return toGravitinoLiteral(icebergDefault, type, unset);
+  }
+
+  private static Expression toGravitinoLiteral(
+      Literal<?> icebergDefault, Type type, Expression unset) {
+    Object value = icebergDefault.value();
+    if (value == null) {
+      return Literals.NULL;
+    }
+
+    switch (type.typeId()) {
+      case BOOLEAN:
+        return Literals.booleanLiteral((Boolean) value);
+      case INTEGER:
+        return Literals.integerLiteral((Integer) value);
+      case LONG:
+        return Literals.longLiteral((Long) value);
+      case FLOAT:
+        return Literals.floatLiteral((Float) value);
+      case DOUBLE:
+        return Literals.doubleLiteral((Double) value);
+      case STRING:
+        return Literals.stringLiteral(value.toString());
+      case DATE:
+        return Literals.dateLiteral(DateTimeUtil.dateFromDays((Integer) 
value));
+      case TIME:
+        return Literals.timeLiteral(DateTimeUtil.timeFromMicros((Long) value));
+      case TIMESTAMP:
+        {
+          Types.TimestampType timestampType = (Types.TimestampType) type;
+          if (timestampType.shouldAdjustToUTC()) {
+            // Iceberg timestamptz is a UTC instant (micros). Expose it as UTC 
LocalDateTime while
+            // keeping TimestampType.withTimeZone so the type still says "with 
time zone".
+            return Literals.of(
+                DateTimeUtil.timestamptzFromMicros((Long) 
value).toLocalDateTime(),
+                
org.apache.gravitino.rel.types.Types.TimestampType.withTimeZone(
+                    FromIcebergType.PRECISION_MICROSECOND));
+          }
+          return Literals.of(
+              DateTimeUtil.timestampFromMicros((Long) value),
+              
org.apache.gravitino.rel.types.Types.TimestampType.withoutTimeZone(
+                  FromIcebergType.PRECISION_MICROSECOND));
+        }
+      case TIMESTAMP_NANO:
+        {
+          Types.TimestampNanoType timestampNanoType = 
(Types.TimestampNanoType) type;
+          if (timestampNanoType.shouldAdjustToUTC()) {
+            // Same UTC-instant → LocalDateTime convention as TIMESTAMP with 
zone above.
+            return Literals.of(
+                DateTimeUtil.timestamptzFromNanos((Long) 
value).toLocalDateTime(),
+                
org.apache.gravitino.rel.types.Types.TimestampType.withTimeZone(
+                    FromIcebergType.PRECISION_NANOSECOND));
+          }
+          return Literals.of(
+              DateTimeUtil.timestampFromNanos((Long) value),
+              
org.apache.gravitino.rel.types.Types.TimestampType.withoutTimeZone(
+                  FromIcebergType.PRECISION_NANOSECOND));
+        }
+      case DECIMAL:
+        {
+          Types.DecimalType decimalType = (Types.DecimalType) type;
+          return Literals.decimalLiteral(
+              Decimal.of((BigDecimal) value, decimalType.precision(), 
decimalType.scale()));
+        }
+      case UUID:
+        return Literals.of(
+            ((UUID) value).toString(), 
org.apache.gravitino.rel.types.Types.UUIDType.get());
+      case FIXED:
+        return Literals.of(
+            toByteArray(value),
+            
org.apache.gravitino.rel.types.Types.FixedType.of(((Types.FixedType) 
type).length()));
+      case BINARY:
+        return Literals.of(
+            toByteArray(value), 
org.apache.gravitino.rel.types.Types.BinaryType.get());
+      default:
+        // Nested types cannot carry non-null Iceberg defaults; other V3 types 
are left unset
+        // rather than failing table load.
+        LOG.warn("Dropping Iceberg column default of unsupported type {} as 
unset", type.typeId());

Review Comment:
   Unsupported default types are logged at warn on every table load, which can 
flood logs for hot tables. Consider debug level or log once per type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to