leanken opened a new pull request #29389: URL: https://github.com/apache/spark/pull/29389
### What changes were proposed in this pull request? In [SPARK-32290](https://issues.apache.org/jira/browse/SPARK-32290), we introduced several new types of HashedRelation. * EmptyHashedRelation * EmptyHashedRelationWithAllNullKeys They were all limited to used only in NAAJ scenario. But as for a improvement, EmptyHashedRelation could also be used in Normal AntiJoin for fast stop, and as for in AQE, we can even eliminate anti join when we knew that buildSide is empty. This Patch including two changes. * In Non-AQE, using EmptyHashedRelation to do fast stop for common anti join as well * In AQE, eliminate anti join if buildSide is a EmptyHashedRelation of ShuffleWriteRecord is 0 ### Why are the changes needed? LeftAntiJoin could apply `fast stop` when BuildSide is Empty, While within AQE, we can even eliminate the anti join. Th is should be a performance improvement in AntiJoin. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? * added case in AdaptiveQueryExecSuite. * added case in HashedRelationSuite. * Make sure SubquerySuite JoinSuite SQLQueryTestSuite passed. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
