Hi Cory, You need to override supports_filters_pushdown in your TableProvider implementation. DataFusion will call this for each filter expression to see which ones can be pushed down.
We should add this information to the documentation at https://arrow.apache.org/datafusion/library-user-guide/custom-table-providers.html .. I filed an issue for this: https://github.com/apache/arrow-datafusion/issues/7676 Hope that helps. Andy. On Wed, Sep 27, 2023 at 8:50 AM Cory Isaacson <[email protected]> wrote: > I am building a custom data source and need to know how to enable push > down filters. Here is an example query: > > SELECT a > FROM t1 > WHERE b = 123 > > DataFusion filters the data itself, but I want it to push down the filter > (it does push down projections correctly). > > I added the filters arg to `create_execution_plan` and save the filters in > the enclosing data source struct. I checked and the filters are always > empty. > > However, I do see the filter in the LogicalPlan. > > Please let me know how to do this, or point me to an example. > > Thanks in advance for any assistance. > > Cory > > > > -- > Cory Isaacson > http://www.coryisaacson.com >
