Adam Binford created SPARK-37466:
------------------------------------
Summary: Support subexpression elimination in lambda functions
Key: SPARK-37466
URL: https://issues.apache.org/jira/browse/SPARK-37466
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.0
Reporter: Adam Binford
https://issues.apache.org/jira/browse/SPARK-37019 will add codegen support for
higher order functions. However we can't support subexpression elimination
inside of lambda functions because subexpressions are evaluated once per row at
the beginning of the codegen. Common expressions inside lambda functions can
easily result in performance degradation due to multiple evaluations of the
same expression. Subexpression elimination inside of lambda functions needs to
be handled specially to be evaluated once per function invocation.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
