JingsongLi commented on code in PR #10601:
URL: https://github.com/apache/paimon/pull/10601#discussion_r4374882914


##########
paimon-format/src/main/java/org/apache/parquet/filter2/predicate/ParquetFilters.java:
##########
@@ -390,17 +428,17 @@ private Comparable<?> toParquetObject(Object value, 
FieldRef fieldRef) {
 
             if (value instanceof Timestamp) {
                 Timestamp timestamp = (Timestamp) value;
-                timestampColumn(fieldRef, fileSchema, caseSensitive);
-                int precision = getTimestampPrecision(type);
-                if (precision <= 3) {
-                    // milliseconds
-                    return timestamp.getMillisecond();
-                } else if (precision <= 6) {
-                    // microseconds
-                    return timestamp.toMicros();
+                switch (timestampUnit(
+                        fieldRef, timestampColumn(fieldRef, fileSchema, 
caseSensitive))) {
+                    case MILLIS:
+                        return timestamp.getMillisecond();
+                    case MICROS:
+                        return getTimestampPrecision(type) <= 3
+                                ? firstMicroOfMilli(timestamp)

Review Comment:
   [P1] Preserve finer timestamp literals when pushing less-than to MICROS
   
   For a precision-3 field, this truncates a literal's submillisecond component 
even though predicate evaluation preserves it. A read row at `...00.123` 
satisfies `lessThan(ts, ...00.123456)`, but `visitLessThan` reuses this 
conversion and emits `< ...123000` microseconds, excluding the matching 
row/group. `PredicateBuilder` accepts the unmodified Timestamp literal and 
`Timestamp.compareTo` includes its nanos; there is no contract that quantizes 
the literal to the field precision.
   
   Using actual head-written Iceberg MICROS files, I reproduced this for 
TIMESTAMP(3) and TIMESTAMP_LTZ(3) at zero, negative, and positive epochs: an 
unfiltered read returns one row and its residual predicate is true, while the 
pushed read returns zero. Replacing only this filter class with exact 
merge-base c9a36a81d7 retains the row from the same files through the same 
reader, because the previous unit-mismatch fallback declines pushdown. The 
added `testMillisTimestampOnMicrosFile` already supplies a 456000-nanosecond 
component but asserts the truncated filter instead of checking row retention. 
Please use operator-aware widening, or conservatively decline pushdown when the 
literal is finer than the read field, and add full-reader assertions for these 
boundaries.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to