ulysses you created SPARK-35282:
-----------------------------------
Summary: Support AQE side shuffled hash join formula
Key: SPARK-35282
URL: https://issues.apache.org/jira/browse/SPARK-35282
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 3.2.0
Reporter: ulysses you
Use AQE runtime statistics to decide if we can use shuffled hash join instead
of sort merge join. Currently, the formula of shuffled hash join selection is
not worked due to the dymanic shuffle partition number.
Add a new config `spark.sql.adaptive.shuffledHashJoinLocalMapThreshold` to
decide if join can be converted to shuffled hash join safely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
