andygrove opened a new pull request, #4644:
URL: https://github.com/apache/datafusion-comet/pull/4644
## Summary
- PR #4429 added a native BNLJ rule (`BroadcastNestedLoopJoinExec ->
CometBroadcastNestedLoopJoinExec` in `CometExecRule.scala`) and updated the
Spark 3.5.8 / 4.0.2 / 4.1.2 test diffs, but `dev/diffs/3.4.3.diff` was missed.
As a result, `JoinSuite`'s `SPARK-34593: Preserve broadcast nested loop join
partitioning and ordering` fails on Spark 3.4 with `0 did not equal 1` at
JoinSuite.scala:1346 - the assertion still expects
`BroadcastNestedLoopJoinExec` only.
- Regenerate `dev/diffs/3.4.3.diff` (apply existing diff to v3.4.3, edit
`JoinSuite.scala`, regenerate via `git diff`) so tests 1 and 3 of SPARK-34593 -
the cases where `getSupportLevel` returns `Compatible` - accept either
`BroadcastNestedLoopJoinExec` or `CometBroadcastNestedLoopJoinExec`. Tests 2
and 4 are intentionally left unchanged because their join/build pairs hit the
`Unsupported` branch in `CometBroadcastNestedLoopJoinExec.getSupportLevel`, so
BNLJ is not wrapped (matches what the 3.5.8/4.0.2/4.1.2 diffs do).
## Test plan
- [x] `git apply --check dev/diffs/3.4.3.diff` against Spark v3.4.3 -
applies cleanly.
- [x] Local SBT run with `ENABLE_COMET=true` against the patched Spark
v3.4.3:
```
sql/testOnly org.apache.spark.sql.JoinSuite -- -z "SPARK-34593"
```
Result: `Tests: succeeded 1, failed 0` (3.283s).
- [ ] CI Spark 3.4 lane (add the `run-spark-3.4-tests` label to trigger).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
