Mikel San Vicente created SPARK-31123: -----------------------------------------
Summary: Drop does not work after join with aliases Key: SPARK-31123 URL: https://issues.apache.org/jira/browse/SPARK-31123 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.2 Reporter: Mikel San Vicente Hi, I am seeing a really strange behaviour in drop method after a join with aliases. It doesn't seem to find the column when I reference to it using dataframe("columnName") syntax, but it does work with other combinators like select {code:java} case class Record(a: String, dup: String) case class Record2(b: String, dup: String) val df = Seq(Record("a", "dup")).toDF val joined = df.alias("a").join(df2.alias("b"), df("a") === df2("b")) val dupCol = df("dup") joined.drop(dupCol) // Does not drop anything joined.drop(func.col("a.dup")) // It works! joined.select(dupCol) // It works! {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org