mchades commented on code in PR #14609:
URL: https://github.com/apache/gravitino/pull/14609#discussion_r4468582330
##########
spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/SparkTableChangeConverter.java:
##########
@@ -93,13 +139,95 @@ public org.apache.gravitino.rel.TableChange
toGravitinoTableChange(TableChange c
(TableChange.UpdateColumnDefaultValue) change;
return org.apache.gravitino.rel.TableChange.updateColumnDefaultValue(
updateColumnDefaultValue.fieldNames(),
- Literals.stringLiteral(updateColumnDefaultValue.newDefaultValue()));
+ convertDefaultValue(
+ updateColumnDefaultValue.newDefaultValue(),
+ updateColumnDefaultValue.fieldNames(),
+ columns));
} else {
throw new UnsupportedOperationException(
String.format("Unsupported table change %s",
change.getClass().getName()));
}
}
+ private Expression convertDefaultValue(
+ String sql, String[] fieldNames, Supplier<Column[]> columns) {
+ // Spark uses an empty SQL string for DROP DEFAULT; a string literal
containing no characters
+ // is represented by two quote characters instead.
+ if (sql.isEmpty()) {
+ throw new UnsupportedOperationException("DROP DEFAULT is not supported
by Gravitino");
+ }
+ org.apache.spark.sql.catalyst.expressions.Expression parsed;
+ Object value;
+ try {
+ parsed = CatalystSqlParser$.MODULE$.parseExpression(sql);
+ boolean literal =
+ parsed instanceof Literal
+ || (parsed instanceof UnaryMinus && ((UnaryMinus)
parsed).child() instanceof Literal)
+ || (parsed instanceof UnaryPositive
+ && ((UnaryPositive) parsed).child() instanceof Literal);
+ Preconditions.checkArgument(
+ literal && parsed.resolved() && parsed.foldable(),
+ "Only literal column defaults are supported: %s",
+ sql);
+ value = parsed.eval(null);
+ } catch (Exception e) {
+ throw new IllegalArgumentException("Unsupported column default SQL: " +
sql, e);
+ }
+ if (parsed.dataType() instanceof TimestampType
Review Comment:
[P2] DATE/STRING defaults still bypass target-aware timestamp coercion. With
Spark's session zone set to `Asia/Shanghai` and a UTC MySQL connection, `SET
DEFAULT DATE '2026-11-04'` on a TIMESTAMP column is valid in Spark 3.5/4.0 and
should represent `2026-11-03T16:00:00Z`. Since this branch only handles
timestamp-typed expressions, JDBC receives `'2026-11-04'` and MySQL 8.0.45
stores `2026-11-04T00:00:00Z` instead. Spark 4.0 also accepts ordinary string
defaults such as `'2026-11-04 12:34:56.123456'`, which have the same eight-hour
shift; Spark 3.5 rejects that string case. Please apply target-aware coercion
to these accepted DATE/STRING assignments, or explicitly reject unsupported
combinations, and add regression tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]