gaoyajun02 created SPARK-36630:
----------------------------------
Summary: Add the option to use physical statistics to avoid large
tables being broadcast
Key: SPARK-36630
URL: https://issues.apache.org/jira/browse/SPARK-36630
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.2.0
Reporter: gaoyajun02
Currently AQE is turned on, when queryStage is not materialized, it uses the
stats of the logical plan to estimate whether the plan can be converted to BHJ,
and in some scenarios the estimated value is several orders of magnitude larger
than the actual broadcast data, which can lead to large tables being broadcast
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
