[
https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885072#comment-15885072
]
zhangshilong commented on YARN-4090:
------------------------------------
[~yufeigu] I found one problem when doing this issue.
In FSLeafQueue: I think resourceUsage of app should not be changed in
assignContainer because FairShareComparator uses resourceUsage to sort Apps.
{code:java}
private TreeSet<FSAppAttempt> fetchAppsWithDemand() {
TreeSet<FSAppAttempt> pendingForResourceApps =
new TreeSet<>(policy.getComparator());
readLock.lock();
try {
for (FSAppAttempt app : runnableApps) {
Resource pending = app.getAppAttemptResourceUsage().getPending();
if (!pending.equals(none())) {
pendingForResourceApps.add(app);
}
}
} finally {
readLock.unlock();
}
return pendingForResourceApps;
}
{code}
But In FSPreemptionThread
run->preemptContainers->app.trackContainerForPreemption
preemptedResources of app will be changed without FairScheduler Lock.
So getResourceUsage of App will be changed in function: assignContainer in
FSLeafQueue.
{code:java}
@Override
public Resource getResourceUsage() {
/*
* getResourcesToPreempt() returns zero, except when there are containers
* to preempt. Avoid creating an object in the common case.
*/
return getPreemptedResources().equals(Resources.none())
? getCurrentConsumption()
: Resources.subtract(getCurrentConsumption(), getPreemptedResources());
}
{code}
> Make Collections.sort() more efficient in FSParentQueue.java
> ------------------------------------------------------------
>
> Key: YARN-4090
> URL: https://issues.apache.org/jira/browse/YARN-4090
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Reporter: Xianyin Xin
> Assignee: zhangshilong
> Attachments: sampling1.jpg, sampling2.jpg, YARN-4090.001.patch,
> YARN-4090.002.patch, YARN-4090.003.patch, YARN-4090.004.patch,
> YARN-4090.005.patch, YARN-4090.006.patch, YARN-4090-preview.patch,
> YARN-4090-TestResult.pdf
>
>
> Collections.sort() consumes too much time in a scheduling round.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
