ulysses-you commented on PR #6048: URL: https://github.com/apache/incubator-gluten/pull/6048#issuecomment-2164287906
> However, because the plan has not changed, the plan will not be replaced it's not sure, AQE will use the new plan if [currentPhysicalPlan != newPhysicalPlan](https://github.com/apache/spark/blob/fd045c9887feabc37c0f15fa41c860847f5fffa0/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala#L381-L389). So I think ShuffledHashJoin should always choose the small table. SortMergeJoin is a special case that Spark always build right side for inner join, etc. If your goal is to optimize the vanilla Spark SortMergeJoin. I think it's better to push to Spark community first. For gluten, we can just optimize SortMergeJoin when do transform. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
