Massive +1 from me. Delta is starting to transition to DSv2 as well, and this solves a major gap we were concerned about. THANK YOU.
On Thu, Feb 19, 2026 at 8:08 PM Anton Okolnychyi <[email protected]> wrote: > Thanks, Szehon! > > This will help address one of the long-standing limitations in DSv2 that > is a common cause of regressions or even blockers for DSv2 adoption. I am > looking forward to implementation. > > - Anton > > ср, 18 лют. 2026 р. о 14:30 Szehon Ho <[email protected]> пише: > >> Hi all, >> >> I would like to propose enhancements for partition filter pushdown, for >> DSV2 data sources that support partitioning (ie, those with partition >> stats). >> >> Some DSV2 data sources, for example table formats like Apache Iceberg, >> lack partition filtering in many queries, compared to Spark-native data >> sources that directly use Catalyst (like Parquet). This proposal can >> bridge that gap while simplifying the data source logic. >> >> JIRA: https://issues.apache.org/jira/browse/SPARK-55596 >> SPIP doc: >> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo >> >> Look forward to comments and feedback. >> >> Thanks, >> Szehon >> >
