gaoyajun02 opened a new pull request #33884: URL: https://github.com/apache/spark/pull/33884
### What changes were proposed in this pull request? Add the option to use physicalStats in AQE ### Why are the changes needed? Currently when AQE's queryStage is not materialized, it uses the stats of the logical plan to estimate whether the plan can be converted to BHJ, and in some scenarios the estimated value is several orders of magnitude larger than the actual broadcast data, which can lead to large tables being broadcast ### Does this PR introduce _any_ user-facing change? Yes, a new config spark.sql.adaptive.broadcastJoin.usePhysicalStats.enabled added. ### How was this patch tested? (TODO) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
