leaves12138 opened a new pull request, #9137: URL: https://github.com/apache/paimon/pull/9137
## Purpose Paimon's vectorized Parquet readers were originally copied from Spark in #4982. The original Spark `VectorizedDeltaBinaryPackedReader` assigned: ```java valuesRead = total - remaining; ``` This replaces the cumulative number of values consumed from the current page with the size of the latest read call. As a result: - multiple read or skip calls do not maintain the correct page-level read count; - a zero-length read after partial consumption resets `valuesRead` to `0`; - the next read can incorrectly replay the separately encoded first value of the DELTA_BINARY_PACKED page. Spark now accumulates the count with `valuesRead += total` in [apache/spark@1071110](https://github.com/apache/spark/commit/107111001a0b5f5258a463481deb1a039902f511). This PR synchronizes that correction to Paimon. This is complementary to #9127. That PR prevents an invalid zero-length read at an RLE row-range boundary, while this change makes the DELTA_BINARY_PACKED reader maintain correct cumulative state independently. ## Changes - Accumulate `valuesRead` across DELTA_BINARY_PACKED read and skip calls. - Add regression coverage for cumulative bounds checking across multiple calls. - Add regression coverage ensuring a zero-length read does not reset the current decoding position. - Cover both INT32 and INT64 DELTA_BINARY_PACKED readers through the shared test suite. ## Testing ```bash mvn -pl paimon-format -am \ -DwildcardSuites=none \ -DfailIfNoTests=false \ -Dsurefire.failIfNoSpecifiedTests=false \ -Dtest='DeltaEncodingTest*' test ``` Result: 30 tests passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
