Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/9154#issuecomment-149292119
Ok, I see. I think I was under the impression that the code was actually
removing tasks from old pending lists, but that doesn't seem to be the case.
That also looks like a bug in itself - what's the point of keeping a pending
list for a dead executor? I can see keeping a pending list for empty hosts
because of dynamic allocation, but even that sounds fishy (the per-host pending
list can be re-created if an executor is ever started again on that host).
So you're right, after reading the code again a few more times it does seem
addPendingTask is redundant.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
