JingsongLi opened a new pull request, #8999:
URL: https://github.com/apache/paimon/pull/8999
## Purpose
Follow up on #8995 to prevent Parquet predicate pushdown from silently
dropping rows when the file value is converted lossily before it is exposed as
the declared table type.
A physical `DOUBLE` may round to a matching declared `FLOAT`, and an
out-of-range physical `INT64` wraps when read as `TINYINT` or `SMALLINT`.
Comparing the predicate against the original physical value can therefore prune
a row group that contains rows matching after conversion.
## Changes
- Do not push declared `FLOAT` predicates to physical `DOUBLE` columns.
- Do not push declared `TINYINT` or `SMALLINT` predicates to physical
`INT64` columns.
- Keep the lossless `INT64` to `INTEGER`, `INT32` to `BIGINT`, and `FLOAT`
to `DOUBLE` paths unchanged.
- Add end-to-end regression coverage for all three lossy narrowing cases.
## Tests
```shell
mvn -pl paimon-format \
-DwildcardSuites=none \
-Dtest=ParquetTypeWideningTest,ParquetFiltersTest,FileTypeNotMatchReadTypeTest \
test
```
53 tests passed; checkstyle, Spotless, and Maven enforcer checks also passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
