Github user lirui-intel commented on the pull request:
https://github.com/apache/spark/pull/892#issuecomment-45176804
Hi @mateiz , I think we should distinguish between tasks that truly have no
preference, and tasks whose preference is unavailable when picking tasks from
pendingTasksWithNoPrefs. If a task really have no preference, it should indeed
be considered as PROCESS_LOCAL because it runs equally well anywhere. And it
should take precedence over NODE_LOCAL tasks. For tasks whose preference is
unavailable, they should wait until we try all the real locality levels.
Maybe we should firstly figure out how to deal with this
pendingTasksWithNoPrefs? Because otherwise it could be difficult to even verify
if the patch works.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
