leaves12138 opened a new pull request, #8505: URL: https://github.com/apache/paimon/pull/8505
### Purpose Some externally produced Parquet files can store values with a wider physical type than the Paimon table schema expects. For example, an `INT` field may be stored as Parquet `INT64` and a `FLOAT` field may be stored as Parquet `DOUBLE`. When such columns are dictionary encoded, the vectorized reader currently selects the updater only from the Paimon logical type and calls methods such as `decodeToInt`, which fails for dictionaries such as `PlainLongDictionary`. ### Changes - Use a narrowing updater when reading Paimon `INT` from Parquet `INT64`. - Use a narrowing updater when reading Paimon `FLOAT` from Parquet `DOUBLE`. - Keep overflow checks for `INT64 -> INT` via `Math.toIntExact`. - Add regression tests for `INT64 -> INT` and `DOUBLE -> FLOAT` reads. ### Tests - `mvn -pl paimon-format -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none -Dtest=FileTypeNotMatchReadTypeTest test` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
