efegokdemir opened a new pull request, #11483: URL: https://github.com/apache/arrow-rs/pull/11483
# Which issue does this PR close? - Closes #11478 # Rationale for this change With sparse row selections, the Mask strategy can decode far more rows than the requested batch size before filtering. For variable-width columns, that can overflow the decoder's offset buffer even when the selected output is small. The selection strategy should not change whether a read succeeds. # What changes are included in this PR? Mask cursor chunks now stop after at most `batch_size` decoded rows, and `read_mask_batch` applies the same bound across chunks accumulated for an output batch. Sparse masks can therefore return smaller batches while preserving all selected rows. # Are these changes tested? Added cursor coverage for sparse selections with and without loaded row ranges, plus a reader regression test that selects three rows across large gaps. The new tests failed before the fix and pass after it. The cursor tests (7), row-selection regression (1), and read-plan tests (13) pass; formatting and diff checks pass. Clippy passed with allowances for lint differences between the installed 1.98 toolchain and the repository's pinned 1.99 toolchain. AI assistance: Codex generated the implementation and regression tests; the changes were reviewed and validated locally. # Are there any significant user-facing changes? No public API change. Sparse selections may produce smaller record batches than the requested batch size, while still returning every selected row. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
