[
https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-1453.
----------------------------------
Resolution: Duplicate
duplicate of https://issues.apache.org/jira/browse/SPARK-1946
> Improve the way Spark on Yarn waits for executors before starting
> -----------------------------------------------------------------
>
> Key: SPARK-1453
> URL: https://issues.apache.org/jira/browse/SPARK-1453
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
>
> Currently Spark on Yarn just delays a few seconds between when the spark
> context is initialized and when it allows the job to start. If you are on a
> busy hadoop cluster is might take longer to get the number of executors.
> In the very least we could make this timeout a configurable value. Its
> currently hardcoded to 3 seconds.
> Better yet would be to allow user to give a minimum number of executors it
> wants to wait for, but that looks much more complex.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
