cloud-fan opened a new pull request, #56271: URL: https://github.com/apache/spark/pull/56271
### What changes were proposed in this pull request? Followup to https://github.com/apache/spark/pull/56209. This adds an internal conf `spark.sql.subexpressionElimination.filterExec.enabled` (default `true`) that gates subexpression elimination (CSE) in `FilterExec` whole-stage codegen specifically. When set to `false`, `FilterExec` falls back to the existing predicate codegen path (`generatePredicateCode`) that loads input columns lazily and short-circuits; subexpression elimination elsewhere (e.g. `ProjectExec`) is unaffected. The conf is checked alongside the existing `spark.sql.subexpressionElimination.enabled` gate in `FilterExec.doConsume`. ### Why are the changes needed? CSE in `FilterExec` codegen was introduced recently (SPARK-56032 and followups) and is a relatively new codegen path that has already needed several correctness/performance followups. The global `spark.sql.subexpressionElimination.enabled` flag is too coarse to disable it in isolation -- turning it off also disables CSE for projections and other operators. A dedicated flag gives a targeted kill-switch should a regression surface in the FilterExec path, without sacrificing CSE elsewhere. The default stays `true`, so behavior is unchanged. ### Does this PR introduce _any_ user-facing change? No. The conf is internal and defaults to `true`, preserving the current behavior. ### How was this patch tested? New unit test in `WholeStageCodegenSuite`: with a genuine common subexpression in the filter predicates, flipping `spark.sql.subexpressionElimination.filterExec.enabled` off (while leaving global subexpression elimination on) makes `FilterExec` fall back to the lazy non-CSE path -- the shared subexpression is re-evaluated per use, matching the code generated when CSE is globally disabled. Existing SPARK-56032 FilterExec CSE tests continue to exercise the default-on path. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude (Claude Code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
