abstractdog opened a new pull request, #3773: URL: https://github.com/apache/parquet-java/pull/3773
### Rationale for this change `ValuesReader.skip(int n)` defaults to a naive loop of `skip()` calls. For dictionary-encoded columns each `skip()` bottoms out in `RunLengthBitPackingHybridDecoder.readInt()` — a mode switch, array indexing, and a value the caller immediately discards. Any filter-then-skip path (column-index row ranges, hash-join probe filtering, runtime filters) pays this cost per skipped row. `RleSkipBenchmark`, JMH throughput, JDK 17, 100 k values/op: | pattern | bitWidth | `readInt()` loop | `skipInts` | speedup | |---------|---------:|-----------------:|-----------:|--------:| | rle | 8 | 1.38 B/s | 104.9 B/s | ~76× | | packed | 8 | 0.89 B/s | 4.12 B/s | ~4.6× | | mixed | 8 | 0.96 B/s | 6.99 B/s | ~7.2× | ### What changes are included in this PR? - New `RunLengthBitPackingHybridDecoder.skipInts(int)` — re-uses `readNext()` per run, then advances `currentCount` by `min(n, currentCount)` instead of walking every value through `readInt()`. - `skip(int)` overrides on `DictionaryValuesReader` and `RunLengthBitPackingHybridValuesReader` delegating to `decoder.skipInts(n)`. - `parquet-benchmarks / RleSkipBenchmark` JMH benchmark. `readNext()` and the `ValuesReader.skip(int)` default are unchanged. ### Are these changes tested? `TestRunLengthBitPackingHybridDecoderSkip` covers RLE-only, PACKED-only, mixed, zero-skip, full-skip, mid-run partial skip, `bitWidth = 0`, and a randomised 4 K-value read/skip interleaving cross-checked against a `readInt()` reference. All 700 `parquet-column` tests pass. ### Are there any user-facing changes? No. Additive and binary-compatible: no signatures change; `skipInts(n)` is semantically identical to N discarded `readInt()`s. Existing callers of `ValuesReader.skip(int)` pick up the fast path with no code change. <!-- Closes #3772 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
