[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14663100#comment-14663100 ]
Karthik Kambatla commented on YARN-3534: ---------------------------------------- Hopefully, last set of comments: # We should also not deprecate NM_CONTAINER_MON_RESOURCE_CALCULATOR. Should have been more explicit in my previous comment. # ContainersMonitorImpl should prioritize container-specific configs and fallback if they are not present. Something along these lines: {code} this.monitoringInterval = conf.getLong(YarnConfiguration.NM_CONTAINER_MON_INTERVAL_MS, conf.getLong(YarnConfiguration.NM_RESOURCE_MON_INTERVAL_MS, YarnConfiguration.DEFAULT_NM_RESOURCE_MON_INTERVAL_MS)); Class<? extends ResourceCalculatorPlugin> clazz = conf.getClass(YarnConfiguration.NM_CONTAINER_MON_RESOURCE_CALCULATOR, conf.getClass( YarnConfiguration.NM_MON_RESOURCE_CALCULATOR, null, ResourceCalculatorPlugin.class), ResourceCalculatorPlugin.class); {code} # yarn-default.xml needs to be updated for the CONTAINER_MON configs we are changing the behavior of, and any default values there should be removed. > Collect memory/cpu usage on the node > ------------------------------------ > > Key: YARN-3534 > URL: https://issues.apache.org/jira/browse/YARN-3534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager > Affects Versions: 2.7.0 > Reporter: Inigo Goiri > Assignee: Inigo Goiri > Attachments: YARN-3534-1.patch, YARN-3534-10.patch, > YARN-3534-11.patch, YARN-3534-12.patch, YARN-3534-14.patch, > YARN-3534-15.patch, YARN-3534-16.patch, YARN-3534-16.patch, > YARN-3534-2.patch, YARN-3534-3.patch, YARN-3534-3.patch, YARN-3534-4.patch, > YARN-3534-5.patch, YARN-3534-6.patch, YARN-3534-7.patch, YARN-3534-8.patch, > YARN-3534-9.patch > > Original Estimate: 336h > Remaining Estimate: 336h > > YARN should be aware of the resource utilization of the nodes when scheduling > containers. For this, this task will implement the collection of memory/cpu > usage on the node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)