haohuaijin opened a new pull request, #10702: URL: https://github.com/apache/arrow-rs/pull/10702
# Which issue does this PR close? - Closes #10624. # Rationale for this change DataFusion makes row-group-local selection decisions (`ParquetAccessPlan`), but the reader APIs only accept selected row groups plus a single global `RowSelection`. Callers must concatenate per-row-group selections into one global selection, which arrow-rs then re-partitions back into per-row-group selections during decoding. This round trip is wasted work and loses each selection's representation (bitmap vs. selector). # What changes are included in this PR? - New public API on the push decoder: `RowGroupSelection` (a row group index plus an optional row-group-local `RowSelection`) and `ParquetPushDecoderBuilder::with_row_group_selections`. Entries decode in the supplied order, omitted row groups are skipped, `None` reads the whole row group, and each selection keeps its bitmap or selector representation. - Mutually exclusive with `with_row_groups` / `with_row_selection`: the setters share an internal state machine (`RowGroupPlan`) that reports conflicting combinations as an error from `build()` regardless of call order. The legacy API combination is unchanged. - `build()` validates per-row-group plans eagerly: out-of-bounds indices and selections longer than their row group are errors; shorter selections skip the trailing rows. - `ParquetPushDecoder::into_builder` preserves remaining local selections (still in local coordinates), so adaptive scans compose with the new API. - Minor behavior improvement: an out-of-bounds index from `with_row_groups` on the push decoder now returns a `ParquetError` during decoding instead of panicking. The sync and async builders are unchanged; the async builder already delegates to the push decoder, so extending the API to it is a small follow-up if needed. # Are these changes tested? Yes, new tests cover bitmap- and selector-backed local selections (including out-of-order row groups and short selections), skip/replace semantics, mutual exclusion in all four call orders, build-time validation, `into_builder` round-trips, and the unchanged legacy combination. All existing tests pass. # Are there any user-facing changes? New public API: `RowGroupSelection` and `ParquetPushDecoderBuilder::with_row_group_selections`, with doc examples. No breaking changes; one behavior change: out-of-bounds `with_row_groups` indices on the push decoder now error during decoding instead of panicking. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
