Xiangrui Meng created SPARK-25378:
-------------------------------------
Summary: ArrayData.toArray assume UTF8String
Key: SPARK-25378
URL: https://issues.apache.org/jira/browse/SPARK-25378
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.0
Reporter: Xiangrui Meng
The following code works in 2.3.1 but failed in 2.4.0-SNAPSHOT:
{code}
import org.apache.spark.sql.catalyst.util._
import org.apache.spark.sql.types.StringType
ArrayData.toArrayData(Array("a", "b")).toArray[String](StringType)
res0: Array[String] = Array(a, b)
{code}
In 2.4.0-SNAPSHOT, the error is
{code}java.lang.ClassCastException: java.lang.String cannot be cast to
org.apache.spark.unsafe.types.UTF8String
at
org.apache.spark.sql.catalyst.util.GenericArrayData.getUTF8String(GenericArrayData.scala:75)
at
org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136)
at
org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136)
at org.apache.spark.sql.catalyst.util.ArrayData.toArray(ArrayData.scala:178)
... 51 elided
{code}
cc: [~cloud_fan] [~yogeshg]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
