[
https://issues.apache.org/jira/browse/SPARK-32154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150132#comment-17150132
]
Apache Spark commented on SPARK-32154:
--------------------------------------
User 'Ngone51' has created a pull request for this issue:
https://github.com/apache/spark/pull/28979
> Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst
> type
> ---------------------------------------------------------------------------------
>
> Key: SPARK-32154
> URL: https://issues.apache.org/jira/browse/SPARK-32154
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: wuyi
> Priority: Major
>
> Users now could register a UDF with Instant/LocalDate as return type even
> with
> spark.sql.datetime.java8API.enabled=false. However, the UDF can only be
> really used with spark.sql.datetime.java8API.enabled=true. This could make
> users confused.
> The problem is we use ExpressionEncoder to ser/deser types when registering
> the UDF, but use Catalyst converters to ser/deser types, which is under
> control of spark.sql.datetime.java8API.enabled, when executing UDF.
> If we could also use ExpressionEncoder to ser/deser types, similar to what we
> do for input parameter types, the, UDF could support Instant/LocalDate, event
> other combined complex types as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
