exmy created SPARK-34987:
----------------------------
Summary: AQE improve: change shuffle hash join to sort merge join
when skewed shuffle hash join exists
Key: SPARK-34987
URL: https://issues.apache.org/jira/browse/SPARK-34987
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.1, 3.0.1, 3.2.0
Reporter: exmy
In our production, `spark.sql.join.preferSortMergeJoin` is false by default.
AQE currently can only optimize skewed join for sort merge join, it will be
better if we can change shuffle hash join to sort merge join when skewed
shuffle hash join exists.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
