bhollis-dbx opened a new pull request, #59283: URL: https://github.com/apache/spark/pull/59283
### What changes were proposed in this pull request? Lower `UpdateFields` to a compact evaluator that stores unchanged field ordinals as metadata and evaluates changed fields separately. Support interpreted execution and code generation without expanding the Catalyst tree by struct width. ### Why are the changes needed? Reconstructing every field makes Catalyst plan size proportional to struct width, increasing optimizer, binding, and code-generation costs for wide and nested structs. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added Catalyst coverage for expression growth by update depth and struct width. Added SQL coverage for nullable, nondeterministic, reused, nested, short-circuited, omitted, and throwable expressions across interpreted and code-generated execution. Ran the focused `ReplaceUpdateFieldsExpressionSuite`, `OptimizeWithFieldsSuite`, `ComplexTypesSuite`, and `ColumnExpressionSuite` tests. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
