github-actions[bot] commented on code in PR #68934:
URL: https://github.com/apache/doris/pull/68934#discussion_r4245321064
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTypeMapping.java:
##########
@@ -150,12 +150,16 @@ private static ConnectorType
fromPrimitive(Type.PrimitiveType primitive,
return ConnectorType.of("TIMESTAMPTZ",
ICEBERG_DATETIME_SCALE_MS, 0);
}
return ConnectorType.of("DATETIMEV2",
ICEBERG_DATETIME_SCALE_MS, 0);
+ case TIMESTAMP_NANO:
+ // TIMESTAMP_NS is a wall-clock type; zoned nanoseconds need a
distinct carrier.
+ return ConnectorType.of(((Types.TimestampNanoType)
primitive).shouldAdjustToUTC()
+ ? "UNSUPPORTED" : "TIMESTAMP_NS");
Review Comment:
[P2] Convert timestamp-nano literals for filtered Iceberg rewrites. This new
mapping makes v3 `timestamp_ns` columns queryable, but
`IcebergPredicateConverter.extractIcebergLiteral` still handles only
`TypeID.TIMESTAMP` for datetime/string literals. A `rewrite_data_files` call
with `WHERE ts_ns = '2024-01-02 03:04:05.123456789'` therefore drops its sole
predicate, and `RewriteDataFilePlanner` rejects the operation as unpushable.
Convert these literals to exact epoch nanoseconds for `TIMESTAMP_NANO` and
cover a filtered rewrite; normal scans also currently lose file pruning.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTypeMapping.java:
##########
@@ -218,6 +222,8 @@ static Type toIcebergPrimitive(ConnectorType type) {
// Legacy parity: timestamp WITHOUT zone (datetime scale
dropped — Iceberg timestamps are
// microsecond). DorisTypeToIcebergType.atomic returns
TimestampType.withoutZone().
return Types.TimestampType.withoutZone();
+ case "TIMESTAMP_NS":
Review Comment:
[P2] Format timestamp-nano write defaults before Iceberg writes. This
mapping makes an unzoned v3 `timestamp_ns` field writable, but
`IcebergWriteSchemaContext` formats every non-null `writeDefault()` while
building the sink schema and its `toDorisSql` switch has no `TIMESTAMP_NANO`
arm. An INSERT into a table with such a default throws `Unsupported Iceberg
write-default type` before planning, even if the INSERT supplies that column.
Add a nanosecond-preserving SQL literal conversion and cover both explicit and
omitted-column INSERTs.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTypeMapping.java:
##########
@@ -218,6 +222,8 @@ static Type toIcebergPrimitive(ConnectorType type) {
// Legacy parity: timestamp WITHOUT zone (datetime scale
dropped — Iceberg timestamps are
// microsecond). DorisTypeToIcebergType.atomic returns
TimestampType.withoutZone().
return Types.TimestampType.withoutZone();
+ case "TIMESTAMP_NS":
+ return Types.TimestampNanoType.withoutZone();
Review Comment:
[P2] Parse timestamp-nano defaults accepted by ADD COLUMN. This new
`TIMESTAMP_NS` mapping lets a v3 Iceberg table add a nanosecond timestamp
column, and FE validation accepts `DEFAULT '2024-01-02 03:04:05.123456789'`.
`IcebergConnectorMetadata.toAddColumnChange` then passes the unchanged text to
`IcebergSchemaBuilder.parseDefaultLiteral`, whose `TIMESTAMP_NANO` arm calls
`Long.parseLong` and rejects the ALTER before updating the schema. Convert the
validated timestamp to exact epoch nanoseconds here and cover ADD COLUMN with a
nine-digit default. This is separate from formatting an existing write default
during INSERT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
