Xiangrui Meng created SPARK-25378: ------------------------------------- Summary: ArrayData.toArray assume UTF8String Key: SPARK-25378 URL: https://issues.apache.org/jira/browse/SPARK-25378 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.0 Reporter: Xiangrui Meng
The following code works in 2.3.1 but failed in 2.4.0-SNAPSHOT: {code} import org.apache.spark.sql.catalyst.util._ import org.apache.spark.sql.types.StringType ArrayData.toArrayData(Array("a", "b")).toArray[String](StringType) res0: Array[String] = Array(a, b) {code} In 2.4.0-SNAPSHOT, the error is {code}java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.spark.unsafe.types.UTF8String at org.apache.spark.sql.catalyst.util.GenericArrayData.getUTF8String(GenericArrayData.scala:75) at org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136) at org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136) at org.apache.spark.sql.catalyst.util.ArrayData.toArray(ArrayData.scala:178) ... 51 elided {code} cc: [~cloud_fan] [~yogeshg] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org