Best practice to avoid ambiguous columns in DataFrame.join

2015-05-15 Thread Justin Yip
Hello, I would like ask know if there are recommended ways of preventing ambiguous columns when joining dataframes. When we join dataframes, it usually happen we join the column with identical name. I could have rename the columns on the right data frame, as described in the following code. Is

Re: Best practice to avoid ambiguous columns in DataFrame.join

2015-05-15 Thread Michael Armbrust
, 40))) df2: org.apache.spark.sql.DataFrame = [_1: int, _2: int] scala df.join(df2.withColumnRenamed(_1, right_key), $_1 === $right_key).printSchema Thanks. Justin -- View this message in context: Best practice to avoid ambiguous columns in DataFrame.join http

Re: Best practice to avoid ambiguous columns in DataFrame.join

2015-05-15 Thread Justin Yip
=== $right_key).printSchema Thanks. Justin -- View this message in context: Best practice to avoid ambiguous columns in DataFrame.join http://apache-spark-user-list.1001560.n3.nabble.com/Best-practice-to-avoid-ambiguous-columns-in-DataFrame-join-tp22907.html Sent from