Zhongwei Zhu created SPARK-43238:
------------------------------------
Summary: Support only decommission idle workers in standalone
Key: SPARK-43238
URL: https://issues.apache.org/jira/browse/SPARK-43238
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.4.0
Reporter: Zhongwei Zhu
Currently, standalone master web ui supports kill/decommission workers. But
when graceful decommission is enabled, running task, shuffle and rdd migration
could take long time. While waiting for running task, shuffle and rdd
migration, decommissioned workers can't run new executors and decommissioned
executors can't run new tasks. This caused lot of resource waste.
If only idle workers to be decommissioned, these workers could be shutdown and
removed to save cost without waiting long decommissioning process.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
