github-actions[bot] commented on code in PR #65912:
URL: https://github.com/apache/doris/pull/65912#discussion_r3630874308
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -928,4 +953,76 @@ private Table getProcessedTable() throws UserException {
}
return baseTable;
}
+
+ @VisibleForTesting
+ static OptionalLong extractFileCreationTimeLowerBound(List<Expr>
conjuncts) {
+ OptionalLong result = OptionalLong.empty();
+ for (Expr conjunct : conjuncts) {
+ result = maxLowerBound(result,
extractFileCreationTimeLowerBound(conjunct));
+ }
+ return result;
+ }
+
+ private static OptionalLong extractFileCreationTimeLowerBound(Expr expr) {
+ if (expr instanceof CompoundPredicate) {
+ CompoundPredicate predicate = (CompoundPredicate) expr;
+ if (predicate.getOp() != CompoundPredicate.Operator.AND) {
+ return OptionalLong.empty();
+ }
+ return
maxLowerBound(extractFileCreationTimeLowerBound(predicate.getChild(0)),
+ extractFileCreationTimeLowerBound(predicate.getChild(1)));
+ }
+ if (!(expr instanceof BinaryPredicate)) {
+ return OptionalLong.empty();
+ }
+
+ BinaryPredicate predicate = (BinaryPredicate) expr;
+ BinaryPredicate.Operator operator = predicate.getOp();
+ SlotRef slot =
PaimonPredicateConverter.convertDorisExprToSlotRef(predicate.getChild(0));
Review Comment:
**P1: Do not derive this bound through an arbitrary cast of
`creation_time`.** `convertDorisExprToSlotRef()` strips any `CastExpr(SlotRef)`
without checking whether the cast preserves value/order. Doris rounds when
reducing DATETIMEV2 precision: a `$files` value `2026-07-21 01:02:56.600`
becomes `01:02:57` under `CAST(creation_time AS DATETIMEV2(0))`, so
`CAST(creation_time AS DATETIMEV2(0)) >= '2026-07-21 01:02:57'` is true. This
extractor nevertheless emits the raw `01:02:57.000` cutoff, causing Paimon to
discard the `.600` row before the residual runs. Restrict this optimization to
a direct slot (or a cast proven to be a precision-preserving no-op) and add a
rounding-boundary test.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -928,4 +953,76 @@ private Table getProcessedTable() throws UserException {
}
return baseTable;
}
+
+ @VisibleForTesting
+ static OptionalLong extractFileCreationTimeLowerBound(List<Expr>
conjuncts) {
+ OptionalLong result = OptionalLong.empty();
+ for (Expr conjunct : conjuncts) {
+ result = maxLowerBound(result,
extractFileCreationTimeLowerBound(conjunct));
+ }
+ return result;
+ }
+
+ private static OptionalLong extractFileCreationTimeLowerBound(Expr expr) {
+ if (expr instanceof CompoundPredicate) {
+ CompoundPredicate predicate = (CompoundPredicate) expr;
+ if (predicate.getOp() != CompoundPredicate.Operator.AND) {
+ return OptionalLong.empty();
+ }
+ return
maxLowerBound(extractFileCreationTimeLowerBound(predicate.getChild(0)),
+ extractFileCreationTimeLowerBound(predicate.getChild(1)));
+ }
+ if (!(expr instanceof BinaryPredicate)) {
+ return OptionalLong.empty();
+ }
+
+ BinaryPredicate predicate = (BinaryPredicate) expr;
+ BinaryPredicate.Operator operator = predicate.getOp();
+ SlotRef slot =
PaimonPredicateConverter.convertDorisExprToSlotRef(predicate.getChild(0));
+ LiteralExpr literal = convertToLiteral(predicate.getChild(1));
+ if (slot == null || literal == null) {
+ slot =
PaimonPredicateConverter.convertDorisExprToSlotRef(predicate.getChild(1));
+ literal = convertToLiteral(predicate.getChild(0));
+ operator = operator.commutative();
+ }
+ if (slot == null || literal == null
+ ||
!PAIMON_FILE_CREATION_TIME_COLUMN.equalsIgnoreCase(slot.getColumnName())
+ || (operator != BinaryPredicate.Operator.GE && operator !=
BinaryPredicate.Operator.GT)) {
+ return OptionalLong.empty();
+ }
+
+ Object value = new TimestampType(3).accept(new
PaimonValueConverter(literal));
+ if (!(value instanceof Timestamp)) {
+ return OptionalLong.empty();
+ }
+ long millis = ((Timestamp) value).getMillisecond();
Review Comment:
**P1: Keep the cutoff in the same time domain as Paimon's file filter.**
`getMillisecond()` here is Paimon's zone-free wall-clock encoding, but Paimon
1.3.1 compares `scan.file-creation-time-millis` against
`PojoDataFileMeta.creationTimeEpochMillis()`, which applies the BE JVM's
`ZoneId.systemDefault()`. For example, this emits `1784595723456` for local
`2026-07-21 01:02:03.456`; on an `Asia/Shanghai` BE, a file with local
`creation_time = 2026-07-21 05:00:00` maps to `1784581200000` and is pruned
even though the retained Doris predicate is true. Doris does not force the
embedded BE JVM to UTC, and the current tests hard-code the GMT value, so they
cannot catch this. Please avoid installing the option until its comparison
domain is provably aligned (an FE-only zone conversion is unsafe for mixed
FE/BE zones), and add non-UTC coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
