Zhongwei Zhu created SPARK-43396:
------------------------------------
Summary: Add config to control max ratio of decommissioning
executors
Key: SPARK-43396
URL: https://issues.apache.org/jira/browse/SPARK-43396
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 3.4.0
Reporter: Zhongwei Zhu
Decommission too many executors at the same time with shuffle or rdd migration
could severely hurt performance of shuffle fetch. Block manager decommissioner
try to migrate shuffle or rdd as soon as possible, this will compete network
and disk IO with shuffle fetch in the target executor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
