I believe the problem is that the generated code does not check if the selected item in the array is null.Naïvely, I think changing this line would solve this: https://github.com/apache/spark/blob/4af647c77ded6a0d3087ceafb2e30e01d97e7a06/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeExtractors.scala#L219Replace: if (index >= $eval1.numElements() || index < 0) { By: if (index >= $eval1.numElements() || index < 0 || ctx.getValue(eval1, dataType, "index") == null ) { Does that make sense?
-- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Tungsten-gives-unexpected-results-when-selecting-null-elements-in-array-tp15717p15719.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
