sunchao opened a new issue, #10846:
URL: https://github.com/apache/arrow-rs/issues/10846
### Describe the bug
`RunEndBuffer::get_physical_indices` keeps visiting later physical runs
after it has mapped every requested logical index. A tiny prefix selection from
a large run-end buffer therefore scales with the entire backing buffer. A small
logical slice can also scan runs beyond its end.
### To reproduce
On `900ec3ee38276ab651e210c4a85b38f8a8a61bcf`, compare:
```rust
use arrow_buffer::RunEndBuffer;
let run_count = 1_048_576;
let run_ends = (1..=run_count as i32).collect::<Vec<_>>();
let buffer = RunEndBuffer::new(run_ends.into(), 0, run_count);
assert_eq!(buffer.get_physical_indices(&[0_u32, 2]).unwrap(), [0, 2]);
let sliced = buffer.slice(run_count / 2, 3);
assert_eq!(
sliced.get_physical_indices(&[0_u32, 2]).unwrap(),
[run_count / 2, run_count / 2 + 2],
);
```
The results are correct, but the implementation scans many runs after the
last requested one. Dedicated benchmarks cover these cases and an all-index
control at 1,024 and 1,048,576 runs.
### Expected behavior
Stop visiting physical runs once no remaining requested index can use them.
Preserve original physical indices, reordered/duplicate requests, slices, and
bounds errors.
### Additional context
AI assistance: Codex helped investigate the implementation and generate the
benchmark, reproduction, and report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
