spark-test commented on code in PR #52185:
URL: https://github.com/apache/spark/pull/52185#discussion_r4347547889
##########
sql/core/src/test/scala/org/apache/spark/sql/connector/GroupBasedMergeIntoTableSuite.scala:
##########
@@ -184,6 +185,212 @@ class GroupBasedMergeIntoTableSuite extends
MergeIntoTableSuiteBase {
}
}
+ test("merge pushes conditional NOT MATCHED BY SOURCE predicates to target") {
+ withTempView("source") {
+ createAndInitTable("pk INT NOT NULL, salary INT, dep STRING",
+ """{ "pk": 1, "salary": 100, "dep": "hr" }
+ |{ "pk": 2, "salary": 200, "dep": "software" }
+ |{ "pk": 3, "salary": 300, "dep": "finance" }
+ |""".stripMargin)
+
+ val sourceDF = Seq(1, 3, 6).toDF("pk")
+ sourceDF.createOrReplaceTempView("source")
+
Review Comment:
**Nit (P3):** This says the whole ON condition is pushed together with the
NOT MATCHED BY SOURCE condition. With NMBS clauses present, only the
target-only conjuncts of ON take part in the disjunction; the join predicate
`t.pk = s.pk` is excluded, which is why the assertion below expects `(t.dep IN
('hr', 'finance'))`. Pushing the full ON condition was the bug fixed earlier in
this PR, so the comment currently describes the behavior the test should guard
against. Suggested wording: "every NOT MATCHED BY SOURCE clause is conditional,
so the target-only part of the ON condition and that clause's condition can be
pushed to the target table as a disjunction".
**Verification:**
- **Inspection:** The reworded comment describes the pushed condition as the
target-only part of the ON condition OR the NOT MATCHED BY SOURCE condition and
matches the condition the test asserts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
