jonahgao commented on issue #13994: URL: https://github.com/apache/datafusion/issues/13994#issuecomment-2574578429
> Another approach may be to add your own [OptimizerRule](https://github.com/apache/datafusion/blob/4e877a08d224d992a8cbcc9a14f59468e312b13f/datafusion-examples/examples/optimizer_rule.rs#L35). You can select the filters in your [TableScan](https://github.com/apache/datafusion/blob/4e877a08d224d992a8cbcc9a14f59468e312b13f/datafusion/expr/src/logical_plan/plan.rs#L2486) and return the unnecessary ones to the parent Filter plan. You can also declare that the data source does not support pushing down any filters, and then, within a custom optimization rule similar to `PushDownFilter`, push the filters to your own data source. If this is a common requirement from the community, we could try to integrate this logic into DataFusion, such as pushing down all supported filters to data sources at once only during the final optimization step. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
