[
https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799388#comment-15799388
]
Nattavut Sutyanyong edited comment on SPARK-19017 at 1/5/17 3:04 AM:
---------------------------------------------------------------------
In the first query, the result does not look right to me.
{code}
{code}
In the second query,
{code}
c1: (1 <> 2) or (2 <> null) => true or unknown => true
c2: (1 <> 1) or (2 <> null) => false or unknown => unknown <-- This is correct,
right?
{code}
In (P1 or P2), if P1 is true, we can short-circuit and conclude the result is
true. In the latter case, if P1 is false, we have to continue evaluating P2 and
since P2 is unknown, the result is unknown.
was (Author: nsyca):
{code}
On c1: (1 <> 2) or (2 <> null) => true or unknown => true
On c2: (1 <> 1) or (2 <> null) => false or unknown => unknown <-- This is
correct, right?
{code}
In (P1 or P2), if P1 is true, we can short-circuit and conclude the result is
true. In the latter case, if P1 is false, we have to continue evaluating P2 and
since P2 is unknown, the result is unknown.
> NOT IN subquery with more than one column may return incorrect results
> ----------------------------------------------------------------------
>
> Key: SPARK-19017
> URL: https://issues.apache.org/jira/browse/SPARK-19017
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0, 2.0.1, 2.0.2, 2.1.0
> Reporter: Nattavut Sutyanyong
>
> When putting more than one column in the NOT IN, the query may not return
> correctly if there is a null data. We can demonstrate the problem with the
> following data set and query:
> {code}
> Seq((2,1)).toDF("a1","b1").createOrReplaceTempView("t1")
> Seq[(java.lang.Integer,java.lang.Integer)]((1,null)).toDF("a2","b2").createOrReplaceTempView("t2")
> sql("select * from t1 where (a1,b1) not in (select a2,b2 from t2)").show
> +---+---+
> | a1| b1|
> +---+---+
> +---+---+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
