GitHub user priyankagargnitk opened a pull request:
https://github.com/apache/spark/pull/21679
SPARK-24695: To add support to return Calendar interval from udf.
## What changes were proposed in this pull request?
This change adds capability to return Calender interval from udf.
Earlier, the udf of Type (String => CalendarInterval) was throwing
Exception stating:
Schema for type org.apache.spark.unsafe.types.CalendarInterval is not
supported
java.lang.UnsupportedOperationException: Schema for type
org.apache.spark.unsafe.types.CalendarInterval is not supported
at
org.apache.spark.sql.catalyst.ScalaReflection391anonfun.apply(ScalaReflection.scala:781)
## How was this patch tested?
Added test case in ScalaReflectionSuite.scala and
ExpressionEncoderSuite.scala
Also, tested by creating an udf that returns Calendar interval.
jira entry for detail: https://issues.apache.org/jira/browse/SPARK-24695
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/priyankagargnitk/spark SPARK-24695
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21679.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21679
----
commit bd805299cc9802597c165a6de1667a7b02ad48ae
Author: Priyanka Garg <prigarg@...>
Date: 2018-06-30T07:23:56Z
SPARK-24695: To add support to return Calender interval from udf.
## What changes were proposed in this pull request?
This change adds capability to return Calender interval from udf.
Earlier, the udf of Type (String => CalendarInterval) was throwing
Exception stating:
Schema for type org.apache.spark.unsafe.types.CalendarInterval is not
supported
java.lang.UnsupportedOperationException: Schema for type
org.apache.spark.unsafe.types.CalendarInterval is not supported
at
org.apache.spark.sql.catalyst.ScalaReflection391anonfun.apply(ScalaReflection.scala:781)
## How was this patch tested?
Added test case in ScalaReflectionSuite.scala and
ExpressionEncoderSuite.scala
Also, tested by creating an udf that returns Calendar interval.
jira entry for detail: https://issues.apache.org/jira/browse/SPARK-24695
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
