Github user lirui-intel commented on the pull request:
https://github.com/apache/spark/pull/1313#issuecomment-49264857
If a TaskSet only contains no-pref tasks, there won't be delay because the
only valid level is ANY, so everything gets scheduled right away.
If a TaskSet contains process-local + no-pref tasks (I suppose it should be
process-local + node-local + no-pref actually since process-local task is also
node-local?), there'll be delay on the no-pref tasks, even when all the
process-local tasks have been launched. This is because valid levels are not
re-computed as task finishes. Not sure if there'll be too much overhead to do
so... (it should also be re-computed when executor is lost)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
