Thomas Graves created SPARK-1453:
------------------------------------
Summary: Improve the way Spark on Yarn waits for executors before
starting
Key: SPARK-1453
URL: https://issues.apache.org/jira/browse/SPARK-1453
Project: Spark
Issue Type: Improvement
Components: YARN
Affects Versions: 1.0.0
Reporter: Thomas Graves
Assignee: Thomas Graves
Currently Spark on Yarn just delays a few seconds between when the spark
context is initialized and when it allows the job to start. If you are on a
busy hadoop cluster is might take longer to get the number of executors.
In the very least we could make this timeout a configurable value. Its
currently hardcoded to 3 seconds.
Better yet would be to allow user to give a minimum number of executors it
wants to wait for, but that looks much more complex.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
