hhhizzz commented on PR #10288: URL: https://github.com/apache/arrow-rs/pull/10288#issuecomment-4998265375
@alamb š A brief update since your last note: I simplified the implementation and added the remaining regression coverage for batch shape and nested columns with different page boundaries. I also realized that, in earlier iterations, I let the scope grow by pursuing too many detailed performance improvements at once. AI-assisted tools make it easy to explore and implement related ideas quickly, but they do not reduce the reviewer's cognitive load. That scope was therefore asking too much from reviewers. I have now narrowed this PR to one behavioral goal: making Mask execution correct across sparse pages without falling back to selectors. Future optimizations can be evaluated and reviewed separately in focused follow-up PRs. This is also how I plan to scope my contributions going forward: establish the behavioral change first, then optimize incrementally with separate evidence. The PR has 604 additions and 168 deletions overall, but when `#[cfg(test)]` code is counted as tests, **_the core runtime change is +244/-116 across four files, or a net increase of 128 lines._** Tests are +360/-52 and account for most of the net growth. I also revised the description around the `LoadedRowRanges` invariant and how safe decode chunks are coalesced into one logical batch. All current CI checks are green ā Whenever your schedule allows, Iād really appreciate your review of the approach š Iād like to keep building context in Parquet predicate pushdown and gradually help maintain this area, including reviewing related issues and PRs and taking follow-up work. Please feel free to tag me wherever I can be useful. Thank you! š -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
