Steven Rand created YARN-6956:

             Summary: preemption may only consider resource requests for one 
                 Key: YARN-6956
             Project: Hadoop YARN
          Issue Type: Bug
          Components: fairscheduler
    Affects Versions: 2.9.0, 3.0.0-beta1
         Environment: CDH 5.11.0
            Reporter: Steven Rand

I'm observing the following series of events on a CDH 5.11.0 cluster, which 
seem to be possible after

1. An application is considered to be starved, so {{FSPreemptionThread}} calls 
{{identifyContainersToPreempt}}, and that calls 
{{FSAppAttempt#getStarvedResourceRequests}} to get a list of 
{{ResourceRequest}} instances that are enough to address the app's starvation.

2. The first {{ResourceRequest}} that {{getStarvedResourceRequests}} sees is 
enough to address the app's starvation, so we break out of the loop over 
{{appSchedulingInfo.getAllResourceRequests()}} after only one iteration:
 We return only this one {{ResourceRequest}} back to the 
{{identifyContainersToPreempt}} method.

3. It turns out that this particular {{ResourceRequest}} happens to have a 
value for {{getResourceName}} that identifies a specific node in the cluster. 
This cause preemption to only consider containers on that node, and not the 
rest of the cluster.

[~kasha], does that make sense? I'm happy to submit a patch if I'm 
understanding the problem correctly.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to