Xianyin Xin commented on YARN-4120:

Thanks [~kasha]. How about distinguishing getResourceUsage() (the current gross 
resource usage) and getNetResourceUsage() (the current gross resource usage 
minus preempted)? The latter are used for preemption related calculations and 
the former for others?

> FSAppAttempt.getResourceUsage() should not take preemptedResource into account
> ------------------------------------------------------------------------------
>                 Key: YARN-4120
>                 URL: https://issues.apache.org/jira/browse/YARN-4120
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Xianyin Xin
> When compute resource usage for Schedulables, the following code is envolved,
> {{FSAppAttempt.getResourceUsage}},
> {code}
> public Resource getResourceUsage() {
>   return Resources.subtract(getCurrentConsumption(), getPreemptedResources());
> }
> {code}
> and this value is aggregated to FSLeafQueues and FSParentQueues. In my 
> opinion, taking {{preemptedResource}} into account here is not reasonable, 
> there are two main reasons,
> # it is something in future, i.e., even though these resources are marked as 
> preempted, it is currently used by app, and these resources will be 
> subtracted from {{currentCosumption}} once the preemption is finished. it's 
> not reasonable to make arrange for it ahead of time. 
> # there's another problem here, consider following case,
> {code}
>             root
>            /    \
>       queue1   queue2
>       /    \
> queue1.3, queue1.4
> {code}
> suppose queue1.3 need resource and it can preempt resources from queue1.4, 
> the preemption happens in the interior of queue1. But when compute resource 
> usage of queue1, {{queue1.resourceUsage = it's_current_resource_usage - 
> preemption}} according to the current code, which is unfair to queue2 when 
> doing resource allocating.

This message was sent by Atlassian JIRA

Reply via email to