Zhen Wang created SPARK-44065:
---------------------------------
Summary: Optimize BroadcastHashJoin skew when localShuffleReader
is disabled
Key: SPARK-44065
URL: https://issues.apache.org/jira/browse/SPARK-44065
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.5.0
Reporter: Zhen Wang
In RemoteShuffleService services such as uniffle and celeborn, it is
recommended to disable localShuffleReader by default for better performance.
But it may make BroadcastHashJoin skewed, so I want to optimize
BroadcastHashJoin skew in OptimizeSkewedJoin when localShuffleReader is
disabled.
Refer to:
https://github.com/apache/incubator-celeborn#spark-configuration
https://github.com/apache/incubator-uniffle/blob/master/docs/client_guide.md#support-spark-aqe
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
