XiDuo You created SPARK-36993:
---------------------------------
Summary: Fix json_tupe throw NPE if fields exist no foldable null
column
Key: SPARK-36993
URL: https://issues.apache.org/jira/browse/SPARK-36993
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.2, 3.2.0, 3.3.0
Reporter: XiDuo You
If json_tuple exists no foldable null field, Spark would throw NPE during eval
field.toString.
e.g. the query `SELECT json_tuple('{"a":"1"}', if(c1 < 1, null, 'a')) FROM (
SELECT rand() AS c1 );` will fail with:
{code:java}
Caused by: java.lang.NullPointerException
at
org.apache.spark.sql.catalyst.expressions.JsonTuple.$anonfun$parseRow$2(jsonExpressions.scala:435)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.map(TraversableLike.scala:286)
at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at
org.apache.spark.sql.catalyst.expressions.JsonTuple.parseRow(jsonExpressions.scala:435)
at
org.apache.spark.sql.catalyst.expressions.JsonTuple.$anonfun$eval$6(jsonExpressions.scala:413)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
