[
https://issues.apache.org/jira/browse/SPARK-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131119#comment-14131119
]
Cheng Hao commented on SPARK-3501:
----------------------------------
https://github.com/apache/spark/pull/2368
> Hive SimpleUDF will create duplicated type cast which cause exception in
> constant folding
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-3501
> URL: https://issues.apache.org/jira/browse/SPARK-3501
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Cheng Hao
> Priority: Minor
>
> When do the query like:
> select datediff(cast(value as timestamp), cast('2002-03-21 00:00:00' as
> timestamp)) from src;
> SparkSQL will raise exception:
> {panel}
> [info] - Cast Timestamp to Timestamp in UDF *** FAILED ***
> [info] scala.MatchError: TimestampType (of class
> org.apache.spark.sql.catalyst.types.TimestampType$)
> [info] at
> org.apache.spark.sql.catalyst.expressions.Cast.castToTimestamp(Cast.scala:77)
> [info] at
> org.apache.spark.sql.catalyst.expressions.Cast.cast$lzycompute(Cast.scala:251)
> [info] at
> org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:247)
> [info] at
> org.apache.spark.sql.catalyst.expressions.Cast.eval(Cast.scala:263)
> [info] at
> org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$5$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:217)
> [info] at
> org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$5$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:210)
> [info] at
> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
> [info] at
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4$$anonfun$apply$2.apply(TreeNode.scala:180)
> [info] at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> [info] at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
