qinyu created SPARK-33192:
-----------------------------
Summary: on k8s dynamic resource allocation failed
Key: SPARK-33192
URL: https://issues.apache.org/jira/browse/SPARK-33192
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.0.1
Reporter: qinyu
when spark running with following parameters
:spark.dynamicAllocation.enabled=true,spark.shuffle.service.enabled=false,spark.dynamicAllocation.shuffleTracking.enabled=true,spark.dynamicAllocation.minExecutors=0,spark.dynamicAllocation.maxExecutor=3,spark.dynamicAllocation.executorIdleTimeout=60,
it means using dynamic resource allocation without shuffle service. ifÂ
submitting a job with shuffle operation, then executor will not be deleted
forever even if the timeout is arrived.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
