[
https://issues.apache.org/jira/browse/YARN-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640377#comment-16640377
]
Zoltan Siegl commented on YARN-8707:
------------------------------------
Hi [~uranus],
I have been trying to understand this deeper.
One thing is that to me it seems that this is a bug indeed, but it should not
cause any real problems, as {{Resource fairShareStarvation()}} uses the correct
calculation, so it will go like:
- Are you starved?
- Hell yes!
- And how many resources do you need?
- Amm.. None.
So no preemption is done in favour for the app in question.
However this should still be fixed.
According to the patch:
{code:java}
Resource fairShareStarvation() {
long now = scheduler.getClock().getTime();
Resource threshold = Resources.multiply(
getFairShare(), getQueue().getFairSharePreemptionThreshold());
Resource fairDemand = Resources.componentwiseMin(threshold, demand);
...
{code}
This code path here considers {{getQueue().getFairSharePreemptionThreshold()}},
unlike the current patch, and I believe that should be the correct approach to
this.
Actually the whole code around here is a bit fuzzy.
What we probably should do here is wise up the {{isStarvedForFairShare()}}
method so it takes care of:
- calculating fair demand
- being starved for fair demand
- being in or out of preemption time treshold
while {{Resource fairShareStarvation()}} should call
{{isStarvedForFairShare()}} and if so it should return the amount, else empty
resource.
Also we definitely need some unit tests around this area, as using the word
"obvious" seems a bit brave in the world of fair scheduler. :)
> It's not reasonable to decide whether app is starved by fairShare
> -----------------------------------------------------------------
>
> Key: YARN-8707
> URL: https://issues.apache.org/jira/browse/YARN-8707
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 3.0.0-alpha3
> Reporter: Zhaohui Xin
> Assignee: Zhaohui Xin
> Priority: Minor
> Attachments: YARN-8707.patch
>
>
> When app's usage reached demand, it's still be considered fairShare starved.
> Obviously, that's not reasonable!
> {code:java}
> boolean isStarvedForFairShare() {
> return isUsageBelowShare(getResourceUsage(), getFairShare());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
