LuciferYang opened a new pull request, #59186: URL: https://github.com/apache/spark/pull/59186
### What changes were proposed in this pull request? `InferFiltersFromConstraints` plants redundant filters derived from a plan's constraints so they can be pushed down. When a constraint carries a `With` (a common-expression definition kept, for instance, inside a conditional branch by `RewriteWithExpression`, per SPARK-58902) whose definition reads only a column that is an equi-join key, the rule substitutes it through the key and plants a filter that still contains the `With` on the other side. This adds `With.inlineDefinitions`, which rewrites a `With` into the expression it stands for, and uses it in `InferFiltersFromConstraints` to inline any `With` in a predicate the rule plants, at both of its planting sites. The inferred filter now holds the plain predicate instead of the `With`. This also precedes https://github.com/apache/spark/pull/58845 (SPARK-59494), which lets a both-sides `With` survive in a join condition; with this in place that change does not have to touch `InferFiltersFromConstraints`. ### Why are the changes needed? A `With` is opaque to filter translation: `DataSourceStrategy` cannot translate one, so a planted filter carrying a `With` is never pushed to the data source, losing the pushdown the inferred filter exists for. Inlining is safe here: only deterministic constraints are planted (both `inferNewFilter` and the constraint set filter on `deterministic`), and inlining a deterministic definition yields the same value at each reference as memoizing it once, so results are unchanged. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New test in `InferFiltersFromConstraintsSuite`, `SPARK-59922: an inferred filter does not carry a With`: a conditional-branch `With` reading an equi-join key is inferred onto the other side, and the planted filter is asserted to carry no `With`. It fails without the inline and passes with it. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
