Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/7786#issuecomment-153984512
Hi @mccheah sorry for late response, looks like this message is out of my
radar.
From Yarn's side, Yarn makes this preemption decision depends on status of
that time in RM, the status will be changed (other applications may free the
container), so the preempted containers will be changed time to time, no
guarantee that this container will definitely be preempted.
Here in my solution, I also updates the preempted containers time to time,
so if at this time, container A is preempted, tasks will be avoided to allocate
to container A. But later on when A is not preempted, tasks can be allocated on
container A again.
The preemption information only reflects the status of cluster at that
time, so in the driver side this preemption information will be updated time to
time to reflect this changing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
