[ https://issues.apache.org/jira/browse/SPARK-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15071695#comment-15071695 ]
Xiao Li edited comment on SPARK-12520 at 12/25/15 6:39 PM: ----------------------------------------------------------- The problem still exists in 1.4 and 1.5, I think. I did not make a try. Should we submit a PR for 1.4 and 1.5? [~davies] My PR to 1.6 is just for description and use case updates. was (Author: smilegator): The problem still exists in 1.4 and 1.5, I think. I did not make a try. Should we submit a PR for 1.4 and 1.5? My PR to 1.6 is just for description and use case updates. > Python API dataframe join returns wrong results on outer join > ------------------------------------------------------------- > > Key: SPARK-12520 > URL: https://issues.apache.org/jira/browse/SPARK-12520 > Project: Spark > Issue Type: Bug > Components: PySpark, SQL > Affects Versions: 1.4.1 > Reporter: Aravind B > > Consider the following dataframes: > """ > left_table: > +------------+------------+---------+--------------+ > |head_id_left|tail_id_left|weight|joining_column| > +------------+------------+---------+--------------+ > | 1| 2| 1| 1~2| > +------------+------------+---------+--------------+ > right_table: > +-------------+-------------+--------------+ > |head_id_right|tail_id_right|joining_column| > +-------------+-------------+--------------+ > +-------------+-------------+--------------+ > """ > The following code returns an empty dataframe: > """ > joined_table = left_table.join(right_table, "joining_column", "outer") > """ > joined_table has zero rows. > However: > """ > joined_table = left_table.join(right_table, left_table.joining_column == > right_table.joining_column, "outer") > """ > returns the correct answer with one row. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org