Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/3816#issuecomment-68798497
My conclusion (sorry if it was unclear above) was that dequeueTask
returning NO_PREF instead of PROCESS_LOCAL should have no effect at all. I
still think it's worth changing for clarity, but it's obviously less important.
In what cases would dequeueTask return NO_PREF tasks when
maxLocality=RACK_LOCAL or ANY? My understanding is that, in a single
resourceOffers call, dequeueTask gets called multiple times in order of
TaskLocality, so any NO_PREF tasks would be returned when it's called with
maxLocality=NO_PREF. And none would remain when ANY and RACK_LOCAL come around.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
