Hong Zhiguo created YARN-2768:
---------------------------------
Summary: optimize FSAppAttempt.updateDemand by avoid clone of
Resource which takes 85% of computing time of update thread
Key: YARN-2768
URL: https://issues.apache.org/jira/browse/YARN-2768
Project: Hadoop YARN
Issue Type: Improvement
Components: fairscheduler
Reporter: Hong Zhiguo
Assignee: Hong Zhiguo
Priority: Minor
See the attached picture of profiling result. The clone of Resource object
within Resources.multiply() takes up **85%** (19.2 / 22.6) CPU time of the
function FairScheduler.update().
The code of FSAppAttempt.updateDemand:
{code}
public void updateDemand() {
demand = Resources.createResource(0);
// Demand is current consumption plus outstanding requests
Resources.addTo(demand, app.getCurrentConsumption());
// Add up outstanding resource requests
synchronized (app) {
for (Priority p : app.getPriorities()) {
for (ResourceRequest r : app.getResourceRequests(p).values()) {
Resource total = Resources.**multiply**(r.getCapability(),
r.getNumContainers());
Resources.addTo(demand, total);
}
}
}
}
{code}
The code of Resources.multiply:
{code}
public static Resource multiply(Resource lhs, double by) {
return multiplyTo(**clone**(lhs), by);
}
{code}
The clone could be skipped by directly update the value of this.demand.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
