github-actions[bot] commented on code in PR #65912:
URL: https://github.com/apache/doris/pull/65912#discussion_r3631512403
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java:
##########
@@ -588,6 +592,15 @@ private static String[] splitParam(String value, String
delimiter) {
private Map<String, String> buildTableOptions(Map<String, String>
tableOptions) {
Map<String, String> options = new HashMap<>(tableOptions);
options.put(CoreOptions.READ_BATCH_SIZE.key(),
String.valueOf(batchSize));
+ String localMillis = params.get(PAIMON_OPTION_PREFIX +
DORIS_FILE_CREATION_TIME_LOCAL_MILLIS);
+ if (localMillis != null) {
+ long lowerBound = Long.parseLong(localMillis);
+ // Paimon converts every file creation timestamp with the BE JVM's
default zone.
+ // Convert the cutoff in this same JVM so mixed FE/BE zones cannot
strengthen it.
+ long epochMillis =
Timestamp.fromEpochMillis(lowerBound).toLocalDateTime()
Review Comment:
**P1: Do not strengthen cutoffs that fall inside a DST gap.** This backend
conversion fixes the fixed-offset FE/BE mismatch, but `LocalDateTime.atZone`
advances a nonexistent local time by the gap length. On an `America/New_York`
BE, `2026-03-08 02:30` becomes `03:30-04:00` (`1772955000000`), while a Paimon
file with the valid wall-clock `creation_time = 03:15` maps to `1772954100000`.
Doris's retained predicate accepts `03:15 >= 02:30`, but
`FileCreationTimeStartingScanner` prunes that manifest entry. This is distinct
from the existing zone-mismatch thread because both values use the same BE zone
here; the mapping itself is non-monotonic through a gap. Detect a gap and use a
conservative cutoff (or omit the pushdown), and add a spring-forward test.
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java:
##########
@@ -588,6 +592,15 @@ private static String[] splitParam(String value, String
delimiter) {
private Map<String, String> buildTableOptions(Map<String, String>
tableOptions) {
Map<String, String> options = new HashMap<>(tableOptions);
options.put(CoreOptions.READ_BATCH_SIZE.key(),
String.valueOf(batchSize));
+ String localMillis = params.get(PAIMON_OPTION_PREFIX +
DORIS_FILE_CREATION_TIME_LOCAL_MILLIS);
+ if (localMillis != null) {
+ long lowerBound = Long.parseLong(localMillis);
+ // Paimon converts every file creation timestamp with the BE JVM's
default zone.
+ // Convert the cutoff in this same JVM so mixed FE/BE zones cannot
strengthen it.
+ long epochMillis =
Timestamp.fromEpochMillis(lowerBound).toLocalDateTime()
+ .atZone(ZoneId.systemDefault()).toInstant().toEpochMilli();
+ options.put(CoreOptions.SCAN_FILE_CREATION_TIME_MILLIS.key(),
String.valueOf(epochMillis));
Review Comment:
**P1: Preserve a stronger existing file-creation cutoff.** In Paimon 1.3.1,
`FilesTable.options()` is empty, but `FilesTable.copy(dynamicOptions)` forwards
these options to its wrapped `FileStoreTable`, whose `copyInternal` merges them
over persisted schema options. If the source table already has
`scan.mode=from-file-creation-time` with cutoff `2000`, a query-derived cutoff
of `1000` overwrites it here; a file at `1500` is then emitted and also passes
the weaker Doris residual, even though the configured scan previously excluded
it. Treat the query bound as an additional restriction by preserving the
stronger existing cutoff (or skip this optimization for conflicting scan
options), and cover this with a real `FilesTable` test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
