Hi JIabao, I think the current interface can already satisfy your requirements. The connector can reject all the filters by returning the input filters as `Result#remainingFilters`.
So maybe we don't need to introduce a new method to disable pushdown, but just introduce an option for the specific connector. Best, Jark On Tue, 24 Oct 2023 at 16:38, Leonard Xu <xbjt...@gmail.com> wrote: > Thanks @Jiabao for kicking off this discussion. > > Could you add a section to explain the difference between proposed > connector level config `scan.filter-push-down.enabled` and existing query > level config `table.optimizer.source.predicate-pushdown-enabled` ? > > Best, > Leonard > > > 2023年10月24日 下午4:18，Jiabao Sun <jiabao....@xtransfer.cn.INVALID> 写道： > > > > Hi Devs, > > > > I would like to start a discussion on FLIP-377: support configuration to > disable filter pushdown for Table/SQL Sources[1]. > > > > Currently, Flink Table/SQL does not expose fine-grained control for > users to enable or disable filter pushdown. > > However, filter pushdown has some side effects, such as additional > computational pressure on external systems. > > Moreover, Improper queries can lead to issues such as full table scans, > which in turn can impact the stability of external systems. > > > > Suppose we have an SQL query with two sources: Kafka and a database. > > The database is sensitive to pressure, and we want to configure it to > not perform filter pushdown to the database source. > > However, we still want to perform filter pushdown to the Kafka source to > decrease network IO. > > > > I propose to support configuration to disable filter push down for > Table/SQL sources to let user decide whether to perform filter pushdown. > > > > Looking forward to your feedback. > > > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=276105768 > > > > Best, > > Jiabao > >
