[
https://issues.apache.org/jira/browse/YARN-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535547#comment-14535547
]
Zhijie Shen commented on YARN-2380:
-----------------------------------
I've two comments:
1. Other than normalize, DefaultResourceCalculator's other methods are still
using Resources.createResource(int), which will generate the resource object
with vcore = 0/1. This change makes the behavior inconsistent in
DefaultResourceCalculator.
2. There're failed test cases. And it bring me another question if users
provide vcore <= 0, do we need to reset them to 0/1 in this case?
> The normalizeRequests method in SchedulerUtils always resets the vCore to 1
> ---------------------------------------------------------------------------
>
> Key: YARN-2380
> URL: https://issues.apache.org/jira/browse/YARN-2380
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Jian Fang
> Assignee: Jian Fang
> Priority: Critical
> Attachments: YARN-2380.patch
>
>
> I added some log info to the method normalizeRequest() as follows.
> public static void normalizeRequest(
> ResourceRequest ask,
> ResourceCalculator resourceCalculator,
> Resource clusterResource,
> Resource minimumResource,
> Resource maximumResource,
> Resource incrementResource) {
> LOG.info("Before request normalization, the ask capacity: " +
> ask.getCapability());
> Resource normalized =
> Resources.normalize(
> resourceCalculator, ask.getCapability(), minimumResource,
> maximumResource, incrementResource);
> LOG.info("After request normalization, the ask capacity: " + normalized);
> ask.setCapability(normalized);
> }
> The resulted log showed that the vcore in ask was changed from 2 to 1.
> 2014-08-01 20:54:15,537 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC
> Server handler 4 on 9024): Before request normalization, the ask capacity:
> <memory:1536, vCores:2>
> 2014-08-01 20:54:15,537 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC
> Server handler 4 on 9024): After request normalization, the ask capacity:
> <memory:1536, vCores:1>
> The root cause is the DefaultResourceCalculator calls
> Resources.createResource(normalizedMemory) to regenerate a new resource with
> vcore = 1.
> This bug is critical and it leads to the mismatch of the request resource and
> the container resource and many other potential issues if the user requests
> containers with vcore > 1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
