Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22102#discussion_r210036342
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -2300,4 +2300,10 @@ class DataFrameSuite extends QueryTest with
SharedSQLContext {
checkAnswer(aggPlusFilter1, aggPlusFilter2.collect())
}
}
+
+ test("SPARK-25051: fix nullabilities of outer join attributes doesn't
stop on AnalysisBarrier") {
+ val df1 = spark.range(4).selectExpr("id", "cast(id as string) as name")
+ val df2 = spark.range(3).selectExpr("id")
+ assert(df1.join(df2, Seq("id"),
"left_outer").where(df2("id").isNull).collect().length == 1)
--- End diff --
Just general suggestion. We should compare the results
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
