[ https://issues.apache.org/jira/browse/SPARK-24007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li resolved SPARK-24007. ----------------------------- Resolution: Fixed Fix Version/s: 2.4.0 2.3.1 2.2.2 > EqualNullSafe for FloatType and DoubleType might generate a wrong result by > codegen. > ------------------------------------------------------------------------------------ > > Key: SPARK-24007 > URL: https://issues.apache.org/jira/browse/SPARK-24007 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.2, 2.1.2, 2.2.1, 2.3.0 > Reporter: Takuya Ueshin > Assignee: Takuya Ueshin > Priority: Major > Labels: correctness > Fix For: 2.2.2, 2.3.1, 2.4.0 > > > {{EqualNullSafe}} for {{FloatType}} and {{DoubleType}} might generate a wrong > result by codegen. > {noformat} > scala> val df = Seq((Some(-1.0d), None), (None, Some(-1.0d))).toDF() > df: org.apache.spark.sql.DataFrame = [_1: double, _2: double] > scala> df.show() > +----+----+ > | _1| _2| > +----+----+ > |-1.0|null| > |null|-1.0| > +----+----+ > scala> df.filter("_1 <=> _2").show() > +----+----+ > | _1| _2| > +----+----+ > |-1.0|null| > |null|-1.0| > +----+----+ > {noformat} > The result should be empty but the result remains two rows. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org