[
https://issues.apache.org/jira/browse/SPARK-28889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zachary S Ennenga updated SPARK-28889:
--------------------------------------
Description:
Looking at `org.apache.spark.sql.catalyst.expressions.Cast`, UDTs do not
support any sort of casting except for identity casts, IE:
{code:java}
case (udt1: UserDefinedType[_], udt2: UserDefinedType[_]) if udt1.userClass ==
udt2.userClass =>
true
{code}
I propose we add an additional piece of functionality here to allow UDTs to
define their own canCast and cast functions to allow users to define their own
cast mechanisms.
An example of how this might look:
{code:java}
case (fromType, toType: UserDefinedType[_]) =>
toType.canCast(fromType) // Returns boolean
{code}
{code:java}
case (fromType, toType: UserDefinedType[_]) =>
toType.cast(fromType) // Returns Casting function
{code}
The UDT base class would contain a default implementation that replicates
current behavior (IE no casting).
was:
Looking at `org.apache.spark.sql.catalyst.expressions.Cast`, UDTs do not
support any sort of casting except for identity casts, IE:
{code:java}
case (udt1: UserDefinedType[_], udt2: UserDefinedType[_]) if udt1.userClass ==
udt2.userClass =>
true
{code}
I propose we add an additional piece of functionality here to allow UDTs to
define their own canCast and cast functions to allow users to define their own
cast mechanisms.
An example of how this might look:
{code:java}
case (fromUDT: UserDefinedType[_], toUDT: UserDefinedType[_]) =>
toUDT.canCast(fromUDT) // Returns boolean
{code}
{code:java}
case (fromUDT: UserDefinedType[_], toUDT: UserDefinedType[_]) =>
toUDT.cast(fromUDT) // Returns toUDT
{code}
The UDT base class would contain a default implementation that replicates
current behavior (IE no casting).
> Allow UDTs to define custom casting behavior
> --------------------------------------------
>
> Key: SPARK-28889
> URL: https://issues.apache.org/jira/browse/SPARK-28889
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.3
> Reporter: Zachary S Ennenga
> Priority: Minor
>
> Looking at `org.apache.spark.sql.catalyst.expressions.Cast`, UDTs do not
> support any sort of casting except for identity casts, IE:
> {code:java}
> case (udt1: UserDefinedType[_], udt2: UserDefinedType[_]) if udt1.userClass
> == udt2.userClass =>
> true
> {code}
> I propose we add an additional piece of functionality here to allow UDTs to
> define their own canCast and cast functions to allow users to define their
> own cast mechanisms.
> An example of how this might look:
> {code:java}
> case (fromType, toType: UserDefinedType[_]) =>
> toType.canCast(fromType) // Returns boolean
> {code}
> {code:java}
> case (fromType, toType: UserDefinedType[_]) =>
> toType.cast(fromType) // Returns Casting function
> {code}
> The UDT base class would contain a default implementation that replicates
> current behavior (IE no casting).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]