[jira] [Commented] (SPARK-31403) TreeNode asCode function incorrectly handles null literals

[Carl Sverre (Jira)]

[ 
https://issues.apache.org/jira/browse/SPARK-31403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17083576#comment-17083576
 ] 

Carl Sverre commented on SPARK-31403:
-

Thanks for checking this on master [~hyukjin.kwon]!  If you happen to have 
Spark master running, can you send me `plan.asCode` and/or the output of your 
repro?  I am very curious to understand what Spark is generating in this case.  
It may be possible to repro this in master with something as simple as `select 
null from t`.

If you don't have time I will see if I can get spark master running and try to 
repro this myself.  Thanks!

> TreeNode asCode function incorrectly handles null literals
> --
>
> Key: SPARK-31403
> URL: https://issues.apache.org/jira/browse/SPARK-31403
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.4.4
>Reporter: Carl Sverre
>Priority: Minor
>
> In the TreeNode code in Catalyst the asCode function incorrectly handles null 
> literals.  When it tries to render a null literal it will match {{null}} 
> using the third case expression and try to call {{null.toString}} which will 
> raise a NullPointerException.
> I verified this bug exists in Spark 2.4.4 and the same code appears to be in 
> master:
> [https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala#L707]
> The fix seems trivial - add an explicit case for null.
> One way to reproduce this is via:
> {code:java}
>   val plan =
> spark
>   .sql("select if(isnull(id), null, 2) from testdb_jdbc.users")
>   .queryExecution
>   .optimizedPlan
>   println(plan.asInstanceOf[Project].projectList.head.asCode) {code}
> However any other way which generates a Literal with the value null will 
> cause the issue.
> In this case the above SparkSQL will generate the literal: {{Literal(null, 
> IntegerType)}} for the "trueValue" of the if statement.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-31403) TreeNode asCode function incorrectly handles null literals

[Carl Sverre (Jira)]

Carl Sverre created SPARK-31403:
---

 Summary: TreeNode asCode function incorrectly handles null literals
 Key: SPARK-31403
 URL: https://issues.apache.org/jira/browse/SPARK-31403
 Project: Spark
  Issue Type: Bug
  Components: Spark Core
Affects Versions: 2.4.4
Reporter: Carl Sverre


In the TreeNode code in Catalyst the asCode function incorrectly handles null 
literals.  When it tries to render a null literal it will match {{null}} using 
the third case expression and try to call {{null.toString}} which will raise a 
NullPointerException.

I verified this bug exists in Spark 2.4.4 and the same code appears to be in 
master:

[https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala#L707]

The fix seems trivial - add an explicit case for null.

One way to reproduce this is via:
{code:java}

  val plan =
spark
  .sql("select if(isnull(id), null, 2) from testdb_jdbc.users")
  .queryExecution
  .optimizedPlan
  println(plan.asInstanceOf[Project].projectList.head.asCode) {code}
However any other way which generates a Literal with the value null will cause 
the issue.

In this case the above SparkSQL will generate the literal: {{Literal(null, 
IntegerType)}} for the "trueValue" of the if statement.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org