Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/1313#issuecomment-49215361
I am not sure if it is ok to wait ... This is something I never considered
from the beginning when I added process_local ... Maybe it is ok !
If it is not, then we might need to come up with something.
Unlike earlier, the noPrefs list now truely contains tasks which have no
preference (earlier task failure also ended up here) ... So maybe not
common anymore ? And so ok to wait ?
On 17-Jul-2014 12:26 am, "Matei Zaharia" <[email protected]> wrote:
> Ah, I see, so you're saying it's worth to wait 3 seconds for node-local
> ones to be able to go instead of launching no-prefs tasks. That does make
> sense. We just have to make sure that if a task set contains *only*
> process-local and no-prefs tasks, for instance, this wait doesn't happen.
> (I'm pretty sure that right now we don't wait for node-local if there were
> no process-local tasks, right? We only wait when we move up a level.)
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1313#issuecomment-49210568>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
