Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/900#issuecomment-48604243
postStartHook seems like a good place to put it. It will only be called
once there and as you said Yarn cluster mode actually waits for the
sparkcontext to be initialized before allocating executors.
It looks like we aren't handling mesos. We should atleast file a jira for
this.
@li-zhihui did you look at mesos at all?
For the yarn side where you added the TODO's about the sleep. I think we
can leave them here as there is another jira to remove them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
