sundapeng opened a new pull request, #9010:
URL: https://github.com/apache/paimon/pull/9010
### Purpose
Follow up on #8995 and #8999 by rejecting Parquet `INT32` values with
non-integer logical annotations when the declared Paimon type is `BIGINT`.
A physical `INT32` may safely widen to `BIGINT` when it is unannotated or
uses an integer logical annotation. However, annotations such as `DECIMAL`,
`DATE`, and `TIME` change the stored value's meaning. Treating those values as
generic integers can return incorrect results, and pushing a `BIGINT` predicate
to such a column can filter rows incorrectly.
This change:
- rejects annotated non-integer `INT32` values in the vectorized `BIGINT`
reader;
- skips `BIGINT` predicate pushdown for the same incompatible file schema;
- covers `DECIMAL`, `DATE`, and `TIME` read paths and the predicate path.
### Tests
- `mvn -pl paimon-format -DskipITs
-Dtest=ParquetFiltersTest,ParquetTypeWideningTest test`
- 52 tests, 0 failures, 0 errors
Draft while the full `paimon-format` verification and CI checks are running.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
