Jian Fang created YARN-2380:
-------------------------------
Summary: The normalizeRequests method in SchedulerUtils always
resets the vCore to 1
Key: YARN-2380
URL: https://issues.apache.org/jira/browse/YARN-2380
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.4.0
Reporter: Jian Fang
Priority: Critical
I added some log info to the method normalizeRequest() as follows.
public static void normalizeRequest(
ResourceRequest ask,
ResourceCalculator resourceCalculator,
Resource clusterResource,
Resource minimumResource,
Resource maximumResource,
Resource incrementResource) {
LOG.info("Before request normalization, the ask capacity: " +
ask.getCapability());
Resource normalized =
Resources.normalize(
resourceCalculator, ask.getCapability(), minimumResource,
maximumResource, incrementResource);
LOG.info("After request normalization, the ask capacity: " + normalized);
ask.setCapability(normalized);
}
The resulted log showed that the vcore in ask was changed from 2 to 1.
2014-08-01 20:54:15,537 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC
Server handler 4 on 9024): Before request normalization, the ask capacity:
<memory:1536, vCores:2>
2014-08-01 20:54:15,537 INFO
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC
Server handler 4 on 9024): After request normalization, the ask capacity:
<memory:1536, vCores:1>
The root cause is the DefaultResourceCalculator calls
Resources.createResource(normalizedMemory) to regenerate a new resource with
vcore = 1.
This bug is critical and it leads to the mismatch of the request resource and
the container resource and many other potential issues if the user requests
containers with vcore > 1.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
