adriangb commented on PR #11165: URL: https://github.com/apache/arrow-rs/pull/11165#issuecomment-5785051590
@sunchao agreed on both points. Before implementation, can we also confirm: 1. One entry per page when the file has an offset index. One entry per column chunk when it does not. 2. A lazy iterator with no depth parameter. The caller stops at its own budget. 3. `into_inner()` moves to a separate PR. I suggest we do not add the opt-in flag: as far as I can tell the annotations come from metadata the planner already reads, and the lazy iterator only computes the entries that the caller consumes. We can add a flag later if a benchmark shows a cost. @alamb, your earlier comment proposed ranges that exclude buffered data. Are you OK with the buffer-independent plan? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
