Xianyang Liu created SPARK-44913:
------------------------------------
Summary: DS V2 supports push down V2 UDF that has magic method
Key: SPARK-44913
URL: https://issues.apache.org/jira/browse/SPARK-44913
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.4.1
Reporter: Xianyang Liu
Right now we only support pushing down the V2 UDF that has not a magic method.
Because the V2 UDF will be analyzed into the `ApplyFunctionExpression` which
could be translated and pushed down. However, a V2 UDF that has the magic
method will be analyzed into `StaticInvoke` or `Invoke` that can not be
translated into V2 expression and then can not be pushed down to the data
source. The magic method is suggested. So this PR adds the support of pushing
down the V2 UDF that has a magic method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
