[
https://issues.apache.org/jira/browse/YARN-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382613#comment-15382613
]
Nathan Roberts commented on YARN-5356:
--------------------------------------
Thanks [~elgoiri] for the patch! Some quick comments
{noformat}
+ int nodeMemoryMb = (int) rcp.getPhysicalMemorySize() / (1024 * 1024);
+ int nodeVirtualCores = rcp.getNumProcessors();
+ this.nodeResource = Resource.newInstance(nodeMemoryMb, nodeVirtualCores);
{noformat}
- Wondering if we should remove "Virtual" from variable names since these are
real cores and the ratio of VCores to Cores isn't always 1. Another option
might be "nodePhysicalCores"?
{noformat}
+
+ /**
+ * Get the physical resources in the node to properly estimate resource
+ * utilization.
+ * @return Physical resources in the node.
+ */
+ public abstract Resource getNodeResource();
+
+ /**
+ * Set the physical resources in the node to properly estimate resource
+ * utilization.
+ * @param nodeResource Physical resources in the node.
+ */
+ public abstract void setNodeResource(Resource nodeResource);
{noformat}
- Difference between getResource() and getNodeResource() might lead to
confusion. Maybe getPhysicalResource()?
- Should we change RMNodeImpl so that it contains the physicalNodeResource?
> NodeManager should communicate physical resource capability to ResourceManager
> ------------------------------------------------------------------------------
>
> Key: YARN-5356
> URL: https://issues.apache.org/jira/browse/YARN-5356
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
> Affects Versions: 3.0.0-alpha1
> Reporter: Nathan Roberts
> Assignee: Inigo Goiri
> Attachments: YARN-5356.000.patch
>
>
> Currently ResourceUtilization contains absolute quantities of resource used
> (e.g. 4096MB memory used). It would be good if the NM also communicated the
> actual physical resource capabilities of the node so that the RM can use this
> data to schedule more effectively (overcommit, etc)
> Currently the only available information is the Resource the node registered
> with (or later updated using updateNodeResource). However, these aren't
> really sufficient to get a good view of how utilized a resource is. For
> example, if a node reports 400% CPU utilization, does that mean it's
> completely full, or barely utilized? Today there is no reliable way to figure
> this out.
> [~elgoiri] - Lots of good work is happening in YARN-2965 so curious if you
> have thoughts/opinions on this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
