[
https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145321#comment-15145321
]
Ming Ma commented on YARN-4690:
-------------------------------
The failed tests pass locally and aren't related.
> Skip object allocation in FSAppAttempt#getResourceUsage when possible
> ---------------------------------------------------------------------
>
> Key: YARN-4690
> URL: https://issues.apache.org/jira/browse/YARN-4690
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: YARN-4690.patch
>
>
> YARN-2768 addresses an important bottleneck. Here is another similar instance
> where object allocation in Resources#subtract will slow down the fair
> scheduler's event processing thread.
> {noformat}
> org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java)
> org.apache.hadoop.yarn.util.Records.newRecord(Records.java)
>
> org.apache.hadoop.yarn.util.resource.Resources.createResource(Resources.java)
> org.apache.hadoop.yarn.util.resource.Resources.clone(Resources.java)
> org.apache.hadoop.yarn.util.resource.Resources.subtract(Resources.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt.getResourceUsage(FSAppAttempt.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.getResourceUsage(FSLeafQueue.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
> java.util.TimSort.binarySort(TimSort.java)
> java.util.TimSort.sort(TimSort.java)
> java.util.TimSort.sort(TimSort.java)
> java.util.Arrays.sort(Arrays.java)
> java.util.Collections.sort(Collections.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>
> org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
>
> org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
> {noformat}
> One way to fix it is to return {{getCurrentConsumption()}} if there is no
> preemption which is the normal case. This means {{getResourceUsage}} method
> will return reference to {{FSAppAttempt}}'s internal resource object. But
> that should be ok as {{getResourceUsage}} doesn't expect the caller to modify
> the object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
