efegokdemir opened a new pull request, #12015: URL: https://github.com/apache/arrow-rs/pull/12015
# Which issue does this PR close? - Closes #10648 # Rationale for this change `try_new_with_row_groups` can choose mask selection for a fragmented `RowSelection`, but it had no information about which pages were loaded. The mask could cross a skipped page and fail while decoding a sparse `RowGroups`. # What changes are included in this PR? `RowGroups` can now expose the page index used to prune pages. `try_new_with_row_groups` uses it to restrict mask decoding to loaded row ranges while retaining the `Auto` strategy. The loaded-range calculation is shared with the push decoder, and a regression test compares page-pruned reads with the high-level reader. AI disclosure: AI assistance was used to investigate the issue and generate the implementation and regression test. # Are these changes tested? All 1,487 Parquet library unit tests passed; one existing test that allocates about 2 GiB remains ignored. Formatting and spelling checks passed. The Parquet lib/tests Clippy run passed with targeted lint allowances; without them, Clippy reports issues in unchanged code on the installed Rust 1.98.1 toolchain. The repository pins Rust 1.99.1. # Are there any significant user-facing changes? `RowGroups` gains a defaulted optional page-index method, so existing implementations remain source-compatible. Implementations that prune pages can provide the index they used, allowing mask selection to stay within loaded pages. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
