lilei1128 opened a new pull request, #8797:
URL: https://github.com/apache/paimon/pull/8797
### Purpose
Prevent incorrect query results when the timestamp logical type in a
Parquet
file is incompatible with the current Paimon field type.
Validate the Parquet timestamp unit and adjustedToUTC flag before
converting
Paimon predicates. Skip predicate pushdown when the file uses a different
timestamp unit or UTC adjustment. Also extract the common primitive field
lookup logic for reuse by decimal and timestamp validation.
### Tests
Add ParquetFiltersTest#testTimestampFileUnitMismatchCannotPushDown to
verify
that predicates are not pushed down when the Parquet file and read type use
different MILLIS/MICROS units.
Existing timestamp tests verify that predicates are still pushed down when
the Parquet timestamp schema matches the read type.
Manually verified with Spark SQL that timestamp equality queries return the
expected row after applying the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
