Prabhu Joseph created SPARK-16225: ------------------------------------- Summary: Spark Sql throws ArrayIndexOutOfBoundsException on accessing Null Value Key: SPARK-16225 URL: https://issues.apache.org/jira/browse/SPARK-16225 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.2 Reporter: Prabhu Joseph
Spark Sql throws ArrayIndexOutOfBoundsException on accessing Null value Repro: {code} [root@spark1 spark]# cat repro a,b,c,,, ./bin/spark-shell scala> val pTel = sc.textFile("file:///usr/hdp/2.3.4.7-4/spark/repro") scala> pTel.take(1) res0: Array[String] = Array(a,b,c,,,) scala> val delimiter = "," delimiter: String = , scala> import org.apache.spark.sql._ import org.apache.spark.sql._ scala> val pRowRDD = pTel.map(_.split(delimiter)).map(p => Row(p(0),p(1),p(2),p(3),p(4),p(5))) pRowRDD: org.apache.spark.rdd.RDD[org.apache.spark.sql.Row] = MapPartitionsRDD[3] at map at <console>:28 scala> pRowRDD.take(1) 16/06/27 08:21:40 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1) java.lang.ArrayIndexOutOfBoundsException: 3 {code} It works fine when the data is a,b,c,d,e,f -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org