[ https://issues.apache.org/jira/browse/YARN-6193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870838#comment-15870838 ]
Daniel Templeton commented on YARN-6193: ---------------------------------------- Thanks, [~kasha]. A couple more comments: * {{SchedulingPolicy.instances}} should be {{INSTANCES}}. Not your fault, but while you're in there... * Why don't the want the comparator to be static? * You've changed the behavior of {{FifoPolicy.checkIfUsageOverFairShare()}}. Are you sure that's the right thing to do? * You should explain in this JIRA thread why you're upping the test iterations from 100 back to 1000. * On a philosophical note, {{checkIfUsageOverFairShare()}} and {{checkIfUsageUnderFairShare()}} don't actually have anything to do with the fair share. They're just resource comparisons. Should the {{fairShare}} arg maybe be replaced with a call to {{getFairShare()}}? * I took some time to stare at the code for a bit, and I don't see where this {code}- Resource currentUsage = getResourceUsage(); - Resource fairshare = getFairShare(); - Resource overFairShareBy = Resources.subtract(currentUsage, fairshare); - - return (Resources.fitsIn(container.getAllocatedResource(), - overFairShareBy)); + Resource usageAfterPreemption = Resources.subtract( + getResourceUsage(), container.getAllocatedResource()); + return !fsQueue.getPolicy().checkIfUsageUnderFairShare( + usageAfterPreemption, getFairShare());{code} actually makes any difference. Seems to be the same operation to me, assuming that the app is over its fair share. > FairScheduler might not trigger preemption when using DRF > --------------------------------------------------------- > > Key: YARN-6193 > URL: https://issues.apache.org/jira/browse/YARN-6193 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler > Affects Versions: 2.8.0 > Reporter: Karthik Kambatla > Assignee: Karthik Kambatla > Attachments: YARN-6193.000.patch, YARN-6193.001.patch, > YARN-6193.002.patch > > > {{FSAppAttempt#canContainerBePreempted}} verifies preempting a container > doesn't lead to new starvation ({{Resources.fitsIn}}). When using DRF, this > leads to verifying both resources instead of just the dominant resource. > Ideally, the check should be based on policy. > Note that current implementation of > {{DominantResourceFairnessPolicy#checkIfUsageOverFairShare}} is broken. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org