leanken commented on a change in pull request #29455:
URL: https://github.com/apache/spark/pull/29455#discussion_r472580755
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala
##########
@@ -83,15 +83,18 @@ trait ShuffleExchangeLike extends Exchange {
case class ShuffleExchangeExec(
override val outputPartitioning: Partitioning,
child: SparkPlan,
- noUserSpecifiedNumPartition: Boolean = true) extends ShuffleExchangeLike {
+ noUserSpecifiedNumPartition: Boolean = true,
+ isNullAwareAntiJoin: Boolean = false) extends ShuffleExchangeLike {
Review comment:
> Just personal opinion - I feel NULL-aware anti join is not very
popular query patterns, is there a way to avoid adding parameter here for
popular class `ShuffleExchangeExec`? It's kind of weird to read that shuffle
exchange depending on a config for NULL-aware anti join.
Yes. I found it weird to push extra parameter in ShuffleExchangeExec too.
I've got two option in the beginning
* Count nullKey and put them into ShuffleExchangeExec SQLMetrics for all
Query.
* Count nullKey only for NAAJ Query but add extra parameter.
Maybe you knew about a better way to identify nullKey in Exchange, please
feel free to share your idea.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
