[ https://issues.apache.org/jira/browse/YARN-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535547#comment-14535547 ]
Zhijie Shen commented on YARN-2380: ----------------------------------- I've two comments: 1. Other than normalize, DefaultResourceCalculator's other methods are still using Resources.createResource(int), which will generate the resource object with vcore = 0/1. This change makes the behavior inconsistent in DefaultResourceCalculator. 2. There're failed test cases. And it bring me another question if users provide vcore <= 0, do we need to reset them to 0/1 in this case? > The normalizeRequests method in SchedulerUtils always resets the vCore to 1 > --------------------------------------------------------------------------- > > Key: YARN-2380 > URL: https://issues.apache.org/jira/browse/YARN-2380 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager > Affects Versions: 2.4.0 > Reporter: Jian Fang > Assignee: Jian Fang > Priority: Critical > Attachments: YARN-2380.patch > > > I added some log info to the method normalizeRequest() as follows. > public static void normalizeRequest( > ResourceRequest ask, > ResourceCalculator resourceCalculator, > Resource clusterResource, > Resource minimumResource, > Resource maximumResource, > Resource incrementResource) { > LOG.info("Before request normalization, the ask capacity: " + > ask.getCapability()); > Resource normalized = > Resources.normalize( > resourceCalculator, ask.getCapability(), minimumResource, > maximumResource, incrementResource); > LOG.info("After request normalization, the ask capacity: " + normalized); > ask.setCapability(normalized); > } > The resulted log showed that the vcore in ask was changed from 2 to 1. > 2014-08-01 20:54:15,537 INFO > org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC > Server handler 4 on 9024): Before request normalization, the ask capacity: > <memory:1536, vCores:2> > 2014-08-01 20:54:15,537 INFO > org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC > Server handler 4 on 9024): After request normalization, the ask capacity: > <memory:1536, vCores:1> > The root cause is the DefaultResourceCalculator calls > Resources.createResource(normalizedMemory) to regenerate a new resource with > vcore = 1. > This bug is critical and it leads to the mismatch of the request resource and > the container resource and many other potential issues if the user requests > containers with vcore > 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)