github-actions[bot] commented on code in PR #69163:
URL: https://github.com/apache/doris/pull/69163#discussion_r4291750960
##########
fe/be-java-extensions/paimon-connector/src/main/java/org/apache/doris/paimon/PaimonArrowBatchAdapter.java:
##########
@@ -163,9 +164,12 @@ public Timestamp getTimestamp(int index, int precision) {
case MICROSECOND:
return Timestamp.fromMicros(value);
case NANOSECOND:
- return Timestamp.fromEpochMillis(
- Math.floorDiv(value, 1_000_000L),
- (int) Math.floorMod(value,
1_000_000L));
+ // Paimon serializers trust target precision.
Truncate the positive
+ // fractional remainder so negative epochs and
leading zero digits
+ // obey the same narrowing rule, including
nested values and keys.
+ int nanos = (int) Math.floorMod(value,
1_000_000L);
+ return
Timestamp.fromEpochMillis(Math.floorDiv(value, 1_000_000L),
+ nanos / nanosPerUnit * nanosPerUnit);
Review Comment:
[P2] Check the nanos range after target-precision narrowing
For an existing Paimon `TIMESTAMP(7)` column, inserting the valid Doris
minimum `1677-09-21 00:12:43.145224192` floors it here to `.145224100`, 92 ns
below `Long.MIN_VALUE`; `TIMESTAMP(8)` similarly becomes `.145224190`, 2 ns
below it. The writer accepts and commits that Timestamp, but both target types
map back to `TIMESTAMP_NS`, and the JNI reader's
`TypeNativeBytes.convertToTimestampNs` then throws from `Math.subtractExact`.
Reject a narrowed value that leaves the signed-nanos domain before passing it
to Paimon; otherwise a successful insert makes that value unreadable through
Doris. The precision7/8 tests and signed-boundary tests currently exercise
these cases separately.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergNereidsUtils.java:
##########
@@ -524,8 +524,13 @@ static Object extractNereidsLiteralValue(
}
// accept string value for date
return literal.getStringValue();
- case TIMESTAMP:
- case TIMESTAMP_NANO: {
+ case TIMESTAMP_NANO:
+ if (((org.apache.iceberg.types.Types.TimestampNanoType)
icebergType).shouldAdjustToUTC()) {
+ return null;
+ }
+ return Expressions.nanos(IcebergTimestampUtils.parseNanos(
+ literal.getStringValue(),
java.time.ZoneOffset.UTC));
Review Comment:
[P2] Normalize raw rewrite literals into the session timezone
`rewrite_data_files` passes its WHERE expression directly from the parser,
so this branch can receive `EqualTo(UnboundSlot(ts), StringLiteral('2024-01-01
00:00:00+08:00'))` without SQL literal coercion. In an `Asia/Shanghai` session,
`ALTER TABLE t EXECUTE rewrite_data_files("rewrite-all"="true") WHERE
ts='2024-01-01 00:00:00+08:00'` converts that value here to `2023-12-31
16:00:00`, while the normal `TIMESTAMP_NS` SQL predicate normalizes it to civil
midnight. With separate identity partitions/files, the requested midnight
partition is skipped and the unrelated 16:00 partition is rewritten. Parse raw
SQL literals with the session destination timezone before encoding their civil
fields as nanos; keep UTC for already-normalized metadata carriers. The added
rewrite regression uses a literal without an offset and misses this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
