[
https://issues.apache.org/jira/browse/SPARK-50560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-50560:
-----------------------------------
Assignee: Daniel
> randstr(-1) causes Java stacktrace and argument types are too restrictive
> -------------------------------------------------------------------------
>
> Key: SPARK-50560
> URL: https://issues.apache.org/jira/browse/SPARK-50560
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.0.0
> Reporter: Serge Rielau
> Assignee: Daniel
> Priority: Major
> Labels: pull-request-available
>
> randstr(-1) result in a runtime stacktrace:
> SparkException: Job aborted due to stage failure: Task 0 in stage 18.0 failed
> 4 times, most recent failure: Lost task 0.3 in stage 18.0 (TID 111)
> (10.68.143.147 executor 0): java.lang.NegativeArraySizeException: -1 at
> org.apache.spark.sql.catalyst.expressions.ExpressionImplUtils.randStr(ExpressionImplUtils.java:327)
> at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown
> Source) at
> org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown
> Source) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)
> Further, it is unusual not to allow static numeric types to down cast or
> upcast to the supported type.
> In this case both length and see must be SMALLINT or INT.
> TINYINT shouls work for sure
> And BIGINT should work as long as teh value itself is not too large.
> In fact compare to substr('hello', 1.0, 1.0) which happily works!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
