andygrove opened a new pull request #29134: URL: https://github.com/apache/spark/pull/29134
### What changes were proposed in this pull request? This PR adds support for plugins to be able to provide columnar exchanges when AQE is enabled. The main changes are: - In AdaptiveSparkPlanExec, the newQueryStage method now optimizes the Exchange node rather than just optimizing its children. This allows plugins to participate in optimizing and replacing the Exchange node itself. - New abstract base classes have been introduced for ShuffleExchange and BroadcastExchange so that Spark can work with replaced versions of these operators. ### Why are the changes needed? The changes are needed so that plugins can provide columnar exchage operators when AQE is enabled. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing unit tests have been updated as part of this PR to test replacing broadcast and shuffle exchanges when AQE is enabled. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
