dgd-contributor opened a new pull request #33293:
URL: https://github.com/apache/spark/pull/33293
…tamp
### What changes were proposed in this pull request?
Casting string to timestamp might throw ArrayIndexOutOfBounds in certain
cases
Code to reproduce:
```
val df = Seq(":8:434421+ 98:38").toDF("c0")
val df2 = df.withColumn("c1", col("c0").cast(DataTypes.TimestampType))
df2.show()
```
Error:
```
java.lang.ArrayIndexOutOfBoundsException: 9
at
org.apache.spark.sql.catalyst.util.DateTimeUtils$.stringToTimestamp(DateTimeUtils.scala:328)
at
org.apache.spark.sql.catalyst.expressions.CastBase.$anonfun$castToTimestamp$2(Cast.scala:455)
at
org.apache.spark.sql.catalyst.expressions.CastBase.buildCast(Cast.scala:295)
at
org.apache.spark.sql.catalyst.expressions.CastBase.$anonfun$castToTimestamp$1(Cast.scala:451)
at
org.apache.spark.sql.catalyst.expressions.CastBase.nullSafeEval(Cast.scala:840)
at
org.apache.spark.sql.catalyst.expressions.UnaryExpression.eval(Expression.scala:476)
```
### Why are the changes needed?
Cast String to timestamp shouldn't throw error, it should return Null
instead.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Add test in DateTimeUtilsSuite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
