[ https://issues.apache.org/jira/browse/SPARK-10737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai resolved SPARK-10737. ------------------------------ Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request 8854 [https://github.com/apache/spark/pull/8854] > When using UnsafeRows, SortMergeJoin may return wrong results > ------------------------------------------------------------- > > Key: SPARK-10737 > URL: https://issues.apache.org/jira/browse/SPARK-10737 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.5.0 > Reporter: Yin Huai > Assignee: Yin Huai > Priority: Blocker > Fix For: 1.6.0, 1.5.1 > > > {code} > val df1 = (1 to 10).map(i => (s"str_$i", i)).toDF("i", "j") > val df2 = > df1 > .join(df1.select(df1("i")), "i") > .select(df1("i"), df1("j")) > val df3 = df2.withColumnRenamed("i", "i1").withColumnRenamed("j", "j1") > val df4 = > df2 > .join(df3, df2("i") === df3("i1")) > .withColumn("diff", $"j" - $"j1") > df4.show(100, false) > +------+---+------+---+----+ > |i |j |i1 |j1 |diff| > +------+---+------+---+----+ > |str_2 |2 |str_2 |2 |0 | > |str_7 |7 |str_2 |2 |5 | > |str_10|10 |str_10|10 |0 | > |str_3 |3 |str_3 |3 |0 | > |str_8 |8 |str_3 |3 |5 | > |str_4 |4 |str_4 |4 |0 | > |str_9 |9 |str_4 |4 |5 | > |str_5 |5 |str_5 |5 |0 | > |str_1 |1 |str_1 |1 |0 | > |str_6 |6 |str_1 |1 |5 | > +------+---+------+---+----+ > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org