ulysses-you commented on a change in pull request #32328:
URL: https://github.com/apache/spark/pull/32328#discussion_r621825338
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala
##########
@@ -148,7 +148,7 @@ object OptimizeSkewedJoin extends CustomShuffleReaderRule {
/*
* This method aim to optimize the skewed join with the following steps:
Review comment:
In current implementaion, we have already supported optimize skew build
side with inner join. For other join type we cann't optimize it due to the
semantics.
IMO It's better to consider OOM at shj itself instead of fallback to smj
which might make things more complicated. Actually, we might change smj to shj
at `reOptimize` if we disable the `preferSortMerge`, so it's confused to change
join strategy again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
