leanken opened a new pull request #29614: URL: https://github.com/apache/spark/pull/29614
### What changes were proposed in this pull request? Currently, EliminateJoinToEmptyRelation Rule will convert Join into EmptyRelation in some cases with AQE on. But if streamedSide of Join is a ShuffleQueryStage(canChangeNumPartitions == false), which means the Exchange produced by repartition Or singlePartition, in this case, if we were to convert it into an EmptyRelation, it will lost user specified number partition information for downstream operator, it's not right. ### Why are the changes needed? NumPartition info incorrect when streamedSide is a repartition plan or SinglePartition plan. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added case in AdaptiveQueryExecSuite. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
