haohuaijin opened a new issue, #10926: URL: https://github.com/apache/arrow-rs/issues/10926
### Is your feature request related to a problem or challenge? `RowFilter` evaluates each `ArrowPredicate` separately, so same-projection predicates may decode or replay the same column more than once. #10859 reuses one decoded stream and evaluates later predicates only on surviving rows. This can avoid repeated decoding or predicate-cache replay, but fragmented or dense survivors may be expensive to materialize. Fusion can therefore regress some workloads. ### Describe the solution you'd like When consecutive predicates use the same projection, the Parquet reader should be able to evaluate them from one decoded stream. This avoids repeated decoding or predicate-cache replay while preserving predicate order and ensuring that each predicate sees only rows accepted by earlier predicates. Fusion should not be an unconditional rewrite: as described above, materializing survivors can cost more than the repeated work it removes for some execution shapes. How to decide when fusion applies — a conservative structural rule, caller configuration, or something smarter — is left open for discussion. #10859 currently uses a conservative structural gate (a single top-level, non-repeated leaf projection, and no active row limit). ### Describe alternatives you've considered _No response_ ### Additional context - Arrow: #10859, #10774, #10776 - DataFusion: apache/datafusion#24426, apache/datafusion#24705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
