Xuefu Zhang created SPARK-22870:
-----------------------------------
Summary: Dynamic allocation should allow 0 idle time
Key: SPARK-22870
URL: https://issues.apache.org/jira/browse/SPARK-22870
Project: Spark
Issue Type: Improvement
Components: Scheduler
Affects Versions: 1.6.0
Reporter: Xuefu Zhang
Priority: Minor
As discussed in SPARK-22765, with SPARK-21656, an executor will not idle out
when there are pending tasks to run. When there is no task to run, an executor
will die out after {{spark.dynamicAllocation.executorIdleTimeout}}, which is
currently required to be greater than zero. However, for efficiency, a user
should be able to specify that an executor can die out immediately w/o being
required to be idle for at least 1s.
This is to make {{0}} a valid value for
{{spark.dynamicAllocation.executorIdleTimeout}}, and special handling such a
case might be needed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
