[
https://issues.apache.org/jira/browse/YARN-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213403#comment-15213403
]
Arun Suresh commented on YARN-4752:
-----------------------------------
Thanks for collating and aggregating all the outstanding issue [~kasha]..
With respect to your doc, I was wondering if it is really necessary to have a
PriorityQueue of apps based on how starved they are. Two apps A1 and A2 might
be starved over their fair share by different amounts, but it is arguably not
MORE 'fair' to pick the the app with the larger deficit than probably pickling
one at random... Ordering them on starvation time seems better (or maybe track
the number of times each starved app is passed over in a pre-emption run and
bubble up apps will longer preemptions passes)
Also, I don't think the 'demand' is calculated accurately in the FSAppAttempt.
It looks like the {{updateDemand()}} method in {{FSAppAttempt}} actually adds
all RRs in a priority. This means that some requests will be counted 3 times
(node + rack + ANY) and will show up as overly starved compared to requests for
just ANY resource.
> [Umbrella] FairScheduler: Improve preemption
> --------------------------------------------
>
> Key: YARN-4752
> URL: https://issues.apache.org/jira/browse/YARN-4752
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.8.0
> Reporter: Karthik Kambatla
> Attachments: YARN-4752.FairSchedulerPreemptionOverhaul.pdf
>
>
> A number of issues have been reported with respect to preemption in
> FairScheduler along the lines of:
> # FairScheduler preempts resources from nodes even if the resultant free
> resources cannot fit the incoming request.
> # Preemption doesn't preempt from sibling queues
> # Preemption doesn't preempt from sibling apps under the same queue that is
> over its fairshare
> # ...
> Filing this umbrella JIRA to group all the issues together and think of a
> comprehensive solution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
