wangyum opened a new pull request, #59258: URL: https://github.com/apache/spark/pull/59258
### What changes were proposed in this pull request? This PR extends Catalyst constraint propagation to support transitive inequality chaining: 1. Infers transitive inequality constraints (e.g., a < 5 from a < b and b < 5, a > 5 from a > b and b > 5). 2. Supports strict and non-strict inequalities (<, <=, >, >=) as well as literal bindings (=). 3. Propagates range constraints across equi-joins and left-outer joins, enabling partition pruning and data skipping on fact tables (as shown in TPC-H Q12). ### Why are the changes needed? When range conditions connect attributes (e.g., **windowed joins** or **event-attribution queries** such as b.time >= a.time with a.time >= '2020-03-01'), Catalyst previously did not infer the corresponding bound on b.time, leading to full-table scans. This optimization allows pushing down the inferred range filters into data sources. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? - Added unit tests to InferFiltersFromConstraintsSuite and ConstraintPropagationSuite. - Regenerated TPC-H Q12 plan stability golden files. - This feature has been running stably in our production environment for more than 6 years. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor (Grok 4.7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
