parthpathak1201 opened a new pull request, #50550:
URL: https://github.com/apache/arrow/pull/50550
### Rationale for this change
`LevelDecoder::Decode` called `FindMinMax` on every decoded batch, scanning
all levels (O(n)) for range validation. This showed up as a CPU hotspot in
the repeated-field skip path (GH-50548). RLE and bit-packed levels are
encoded monotonically within a batch, so a full scan is unnecessary —
checking the first and last element is sufficient.
### What changes are included in this PR?
- Replace `FindMinMax` with an O(1) check of `levels[0]` and
`levels[num_decoded - 1]` in `LevelDecoder::Decode`.
- Remove the now-unused `#include "parquet/level_comparison.h"` from
`column_reader.cc`.
- Simplify the error message (no longer reports min/max values).
### Are these changes tested?
Existing SkipRecords and ReadRecords tests pass (18 tests). Benchmarks
show 4.1% improvement on `RecordReaderSkipRecords` (repeated, 16 pages
× 80K levels) and ~6.7% on single-call `SkipRecords(59000)`.
### Are there any user-facing changes?
No behavioral change. The validation still catches out-of-range levels.
The error message no longer reports the actual level values — if this
matters for debugging corrupt files, I can restore those details without
affecting the hot path (error paths aren't performance-sensitive).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
