Wenchen Fan created SPARK-30127:
-----------------------------------
Summary: UDF should work for case class like Dataset operations
Key: SPARK-30127
URL: https://issues.apache.org/jira/browse/SPARK-30127
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 3.0.0
Reporter: Wenchen Fan
Currently, Spark UDF can only work on data types like java.lang.String,
o.a.s.sql.Row, Seq[_], etc. This is inconvenient if you want to apply an
operation on one column, and the column is struct type. You must access data
from a Row object, instead of your domain object like Dataset operations. It
will be great if UDF can work on types that are supported by Dataset, e.g. case
classes.
Note that, there are multiple ways to register a UDF, and it's only possible to
support this feature if the UDF is registered using Scala API that provides
type tag, e.g. `def udf[RT: TypeTag, A1: TypeTag](f: Function1[A1, RT])`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
