kazuyukitanimura commented on code in PR #178:
URL:
https://github.com/apache/arrow-datafusion-comet/pull/178#discussion_r1524318325
##########
spark/src/test/scala/org/apache/comet/exec/CometExecSuite.scala:
##########
@@ -58,6 +58,53 @@ class CometExecSuite extends CometTestBase {
}
}
+ // TODO: Add a test for SortMergeJoin with join filter after new DataFusion
release
+ test("SortMergeJoin without join filter") {
+ withSQLConf(
+ SQLConf.ADAPTIVE_AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1",
+ SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1") {
+ withParquetTable((0 until 10).map(i => (i, i % 5)), "tbl_a") {
+ withParquetTable((0 until 10).map(i => (i % 10, i + 2)), "tbl_b") {
+ val df1 = sql("SELECT * FROM tbl_a JOIN tbl_b ON tbl_a._2 =
tbl_b._1")
+ checkSparkAnswerAndOperator(df1)
Review Comment:
I see. How does `checkSparkAnswerAndOperator` checks whether it is Comet
SMJ?
Unless we provide `includeClasses`, it only checks classes below?
https://github.com/apache/arrow-datafusion-comet/blob/main/spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala#L156
```
case _: CometScanExec | _: CometBatchScanExec => true
case _: CometSinkPlaceHolder | _: CometScanWrapper => false
case _: CometExec | _: CometShuffleExchangeExec => true
case _: CometBroadcastExchangeExec => true
case _: WholeStageCodegenExec | _: ColumnarToRowExec | _: InputAdapter
=> true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
