Xianyin Xin created YARN-4120:
---------------------------------
Summary: FSAppAttempt.getResourceUsage() should not take
preemptedResource into account
Key: YARN-4120
URL: https://issues.apache.org/jira/browse/YARN-4120
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler
Reporter: Xianyin Xin
When compute resource usage for Schedulables, the following code is envolved,
{{FSAppAttempt.getResourceUsage}},
{code}
public Resource getResourceUsage() {
return Resources.subtract(getCurrentConsumption(), getPreemptedResources());
}
{code}
and this value is aggregated to FSLeafQueues and FSParentQueues. In my opinion,
taking {{preemptedResource}} into account here is not reasonable, there are two
main reasons,
# it is something in future, i.e., even though these resources are marked as
preempted, it is currently used by app, and these resources will be subtracted
from {{currentCosumption}} once the preemption is finished. it's not reasonable
to make arrange for it ahead of time.
# there's another problem here, consider following case,
{code}
root
/ \
queue1 queue2
/ \
queue1.3, queue1.4
{code}
suppose queue1.3 need resource and it can preempt resources from queue1.4, the
preemption happens in the interior of queue1. But when compute resource usage
of queue1, {{queue1.resourceUsage = it's_current_resource_usage - preemption}}
according to the current code, which is unfair to queue2 when doing resource
allocating.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
