(resending to because forgot to reply to dev list as well) Thanks so much for the feedback Hongshun,
One alternative we have started considering is adding a method like `supportsFilterReuse()` (or `supportsDisjunctivePredicates()`) directly to the existing `SupportsFilterPushDown` interface, similar to how `SupportsProjectionPushDown.supportsNestedProjection()` is structured to allow the connector owner to opt into an extension of the same ability. The rationale for this design is that filter-reuse is not really independent from filter-pushdown, need filter-pushdown for any real utility to come from reusing the filter. Similar to `SupportsProjectionPushDown.supportsNestedProjection()`, this keeps the same filter-reuse concept in the same interface rather than requiring an entirely new source ability interface to be implemented. Thanks again for the feedback and curious to hear your view on this. I am going to try and have a public draft PR for this feature soon. Daniel On Wed, May 6, 2026 at 3:37 AM Hongshun Wang <[email protected]> wrote: > Hi Daniel, > This works looks fantastic to me ! > > > "Should this optimization remain a job-level flag consistent with the > established pattern, or should we pursue finer-grained scope (per-table or > per-scan) for v1?" > > I think that finer-grained scope (per-table or per-scan) would be better. > My suggestion is to add a SupportReuseFilter(like > what org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown > does) to Source. Only a source SupportReuseFilter return true, need to > apply this ability. > > Best, > Hongshun > > On Fri, May 1, 2026 at 12:25 AM Daniel Rossos via dev < > [email protected]> wrote: > >> Hi Everyone, >> >> I would like to discuss a new FLIP (FLIP-XXX: Table Planner Source Filter >> Reuse). >> >> Brief background, today scans of the same table with different >> FilterPushDownSpec values produce independent source readers because their >> digests differ. For sources where scan operations are expensive (BigQuery >> Storage Read API sessions, JDBC query execution etc), this results in >> multiple source scans when one would suffice. >> >> We have a public draft of the FLIP[1], as well as a working prototype on >> our internal fork (to be shared soon and linked in the thread). >> >> The main open question from the FLIP I'd most value early feedback on is >> the optimization's configuration scope: >> "Should this optimization remain a job-level flag consistent with the >> established pattern, or should we pursue finer-grained scope (per-table or >> per-scan) for v1?" >> >> Thanks a ton in advance for the feedback, >> >> Daniel >> >> [1] >> >> https://docs.google.com/document/d/1CcdogFWShLdybEBhRNvu4E7zSc0ep3hJQIlsDCnr_nc/edit?usp=sharing >> >
