Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/900#issuecomment-45776939
@kayousterhout @mridulm Looking briefly at pr #892 it seems that is
handling locality when executors are added later and I assume some of the
locality wait configs come into affect here, but it doesn't just wait for a
certain number of executors to be there to start (or period of time), does it?
Are the changes good enough where that waiting for a good percentage of the
executors no longer matters?
I seems like depending on your resource manager and how busy of a cluster
you run on it could take a while (minutes) to get a large number of executors.
I think this should just be a configuration and user code should not have to
"sleep" or workaround this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
