adriangb opened a new pull request, #11241: URL: https://github.com/apache/arrow-rs/pull/11241
> [!NOTE] > This PR is stacked on https://github.com/apache/arrow-rs/pull/11238. Only the last commit is new (+87/−12). I will rebase when https://github.com/apache/arrow-rs/pull/11238 merges. # Which issue does this PR close? - Part of https://github.com/apache/arrow-rs/issues/11234 (step H). # Rationale for this change With `FetchGranularity::Batch` from https://github.com/apache/arrow-rs/pull/11238, the predicate and output readers read each window with the `Selectors` policy. The row-group mode resolves the configured `RowSelectionPolicy` (by default `Auto`), which reads dense, fragmented selections with a mask, and is faster for them. With this PR, both modes resolve the configured policy. # What changes are included in this PR? `IncrementalRowGroup::window_plan` resolves the configured policy for each window: | Resolved strategy | Plan | |---|---| | `Selectors` | selectors (as before) | | `Mask` | mask, limited to the rows of the loaded pages with `loaded_row_ranges_for_projection`, as `prepare_selection_for_page_skipping` does for a row group | A mask decodes all rows of a batch. Without the limit, it can read a page that the decoder does not hold. `LoadedRowRanges::ranges` is no longer test-only. # Are these changes tested? Yes. - `window_plan_uses_the_configured_policy`: with `Mask`, the window plan uses a mask cursor; with `Selectors`, it does not. It fails without this change. - The equivalence tests of https://github.com/apache/arrow-rs/pull/11238 already vary the policy (`Auto`, `Mask`, `Selectors`) with page-skipping selections. With this PR they run the mask path. 4 of them fail if the mask is not limited to the loaded pages. I did not benchmark this change. The speed-up is the same one that the row-group mode gets from `Auto`. # Are there any user-facing changes? No API change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
