Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/1313#issuecomment-49481250
@mridulm I think in the example you talked about, the idea is the
TaskSetManagers are supposed to be iterated over in the "fair" order -- so it's
fine if we launch rack-local tasks for task set manager A even if there's a
later task set manager that could have launched a node-local task (because the
fact that we tried A first means that scheduling the tasks for A is higher
priority).
In terms of what to do for this patch, it seems like we should do @mateiz's
suggestion where we add an extra, special locality level that gets called by
TaskSchedulerImpl after all of the valid locality levels, that will allow
speculative and no-pref tasks to be launched.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
