[
https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-36630:
---------------------------------
Target Version/s: (was: 3.2.0)
> Add the option to use physical statistics to avoid large tables being
> broadcast
> -------------------------------------------------------------------------------
>
> Key: SPARK-36630
> URL: https://issues.apache.org/jira/browse/SPARK-36630
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: gaoyajun02
> Priority: Major
>
> Currently AQE is turned on, when queryStage is not materialized, it uses the
> stats of the logical plan to estimate whether the plan can be converted to
> BHJ, and in some scenarios the estimated value is several orders of magnitude
> larger than the actual broadcast data, which can lead to large tables being
> broadcast
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
