[
https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15151747#comment-15151747
]
Hudson commented on YARN-4690:
------------------------------
FAILURE: Integrated in Hadoop-trunk-Commit #9319 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9319/])
YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when (sjlee:
rev 7de70680fe44967e2afc92ba4c92f8e7afa7b151)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
* hadoop-yarn-project/CHANGES.txt
> Skip object allocation in FSAppAttempt#getResourceUsage when possible
> ---------------------------------------------------------------------
>
> Key: YARN-4690
> URL: https://issues.apache.org/jira/browse/YARN-4690
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
> Fix For: 2.8.0, 2.7.3, 2.9.0, 2.6.5
>
> Attachments: YARN-4690.patch
>
>
> YARN-2768 addresses an important bottleneck. Here is another similar instance
> where object allocation in Resources#subtract will slow down the fair
> scheduler's event processing thread.
> {noformat}
> org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java)
> org.apache.hadoop.yarn.util.Records.newRecord(Records.java)
>
> org.apache.hadoop.yarn.util.resource.Resources.createResource(Resources.java)
> org.apache.hadoop.yarn.util.resource.Resources.clone(Resources.java)
> org.apache.hadoop.yarn.util.resource.Resources.subtract(Resources.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt.getResourceUsage(FSAppAttempt.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.getResourceUsage(FSLeafQueue.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
> java.util.TimSort.binarySort(TimSort.java)
> java.util.TimSort.sort(TimSort.java)
> java.util.TimSort.sort(TimSort.java)
> java.util.Arrays.sort(Arrays.java)
> java.util.Collections.sort(Collections.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>
> org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
>
> org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
> {noformat}
> One way to fix it is to return {{getCurrentConsumption()}} if there is no
> preemption which is the normal case. This means {{getResourceUsage}} method
> will return reference to {{FSAppAttempt}}'s internal resource object. But
> that should be ok as {{getResourceUsage}} doesn't expect the caller to modify
> the object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
