JkSelf commented on pull request #31756: URL: https://github.com/apache/spark/pull/31756#issuecomment-834046989
@tgravescs This PR is mainly to solve the limitations of [PR#31258](https://github.com/apache/spark/pull/31258). When DPP + AQE is supported in [PR#31258](https://github.com/apache/spark/pull/31258), only the broadcast exchange on the build side can be executed first. Then the probe side can reuse the exchange of the build side in the DPP subquery, otherwise DPP will not be supported in AQE. This approach mainly contain two steps. 1. In `PlanAdaptiveDynamicPruningFilters` rule, judge whether the broadcast exchange can be reused, if so, it will insert the DPP subquery filter on the probe side. 2. Create a `AdaptiveSparkPlanExec` with the broadcast exchange and then we can reuse the existing reuse logic to reuse the broadcast exchange in `AdaptiveSparkPlanExec` plan。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
