sunchao opened a new issue, #11164: URL: https://github.com/apache/arrow-rs/issues/11164
## Describe the enhancement Expose a small, read-only way for a caller-driven Parquet I/O scheduler to ask which byte ranges the push decoder would need for its next one or two selected row groups, without advancing the decoder. Also allow recovering the original `AsyncFileReader` from a `ParquetRecordBatchStreamBuilder` after metadata or Bloom-filter preparation. ## Motivation The push decoder already separates decoding from I/O through `NeedsData`. A caller that wants to prepare the next read, however, needs more than a row-group index: projection, row selections, offset/limit, and bytes already buffered all affect the actual request. Reimplementing that planning outside the decoder can fetch unnecessary bytes or drift from the demand path. Advancing the decoder merely to discover an optional future request changes ordered execution. For example, a scan may read only one column from a wide file and select a small portion of the next row group. A preview should describe the ranges the existing planner would request, rather than treating the whole row group as the unit of I/O. It must remain an advisory snapshot: later buffer or decoder changes can invalidate it, and the caller must reconcile it with actual ordered `NeedsData` requests. There is also an ownership gap when preparing a custom asynchronous reader. Moving that reader into a stream builder allows metadata and Bloom-filter reads, but the caller cannot then recover the same reader to use in its own push-decoder I/O loop. Recreating the reader can lose state such as cached metadata or a file-version token. A consuming `into_inner` method would return it without additional I/O; callers would retain any metadata they still need separately. ## Proposed scope The range preview would be available only at a row-group boundary with no row predicates, and would reuse the demand planner on a cloned remaining plan. It would not evaluate predicates, start I/O, decode batches, manage memory, or implement a scheduler. Unsupported states would return `None`; no remaining selected work would return an empty preview. Existing decoding behavior would remain unchanged. The initial implementation would bound each request to one or two row groups. This is an API proposal for external scheduling, not a claim of measured query-time improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
