Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23057#discussion_r234521306
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala
---
@@ -119,7 +139,7 @@ object RewritePredicateSubquery extends
Rule[LogicalPlan] with PredicateHelper {
// (A.A1 = B.B1 OR ISNULL(A.A1 = B.B1)) AND (B.B2 = A.A2) AND
B.B3 > 1
val finalJoinCond = (nullAwareJoinConds ++
conditions).reduceLeft(And)
// Deduplicate conflicting attributes if any.
- dedupJoin(Join(outerPlan, sub, LeftAnti, Option(finalJoinCond)))
+ dedupJoin(Join(outerPlan, newSub, LeftAnti,
Option(finalJoinCond)))
case (p, predicate) =>
val (newCond, inputPlan) =
rewriteExistentialExpr(Seq(predicate), p)
Project(p.output, Filter(newCond.get, inputPlan))
--- End diff --
this fails indeed. I'll investigate it, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
