zhuqi-lucas commented on PR #22460: URL: https://github.com/apache/datafusion/pull/22460#issuecomment-4531800058
> Thanks for the review @zhuqi-lucas ! > > Do you think this will help with #22450? My thought was that it would make it very cheap to check e.g. on every batch, row group boundary, etc. if we need to re-prune using stats. Does that pan out you think? Thank you @adriangb, yes , this should fit #22450 cleanly. The RG-boundary "did the filter change?" check is exactly the pattern this tracker is built for — steady-state O(1) atomic loads per still-incomplete filter, no tree walk. The `complete` flag is a bonus on top: once all dynamic filters upstream have called `mark_complete()`, the RG pruner can stop polling and stay on the fast path. I'll wire this into #22450 once this PR merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
