[
https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096324#comment-16096324
]
Sunil G commented on YARN-3409:
-------------------------------
Hi [~templedf]
Lemme take the liberty of pointing out some thoughts here.
Currently we have CommonNodeLabelManager which handles node label and other
common tasks such as
# Api Handling
# Storage and mappings
# Recovery
For internal scheduler access point, a derivative version of
RMNodeLabelManager is used. All together, as you mentioned we might need to
reinvent whole wheel here. Ideally we would be looking a Common base class to
handle recovery, storage and basic mappings. NodeLabel specific manager could
derive from this base class to handle internal node label specific apis etc.
Similarly for attributes, we can have a RMNodeAttribute manage derived from the
common class. In a nutshell, there are some pieces in node label and attributes
which are different and it could be handled respectively.
About the point +"value for the label could change between when the allocation
is made and when the AM launches the container"+ , I think it is kind of
handled now in node label. We could see something like containers which are
scheduler based on old attribute and still running, and new set of containers
which are scheduled based on new attribute. If container demand for new
attribute is more on that node, all old style containers could be marked for
preemption. In fact, we do same now for node label (when partition of node is
changed runtime). Is this a possible way of handling same. Thoughts?
I think [[email protected]] also has more thoughts on same. Please
correct me if I am wrong. Thank You.
> Add constraint node labels
> --------------------------
>
> Key: YARN-3409
> URL: https://issues.apache.org/jira/browse/YARN-3409
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, capacityscheduler, client
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf,
> YARN-3409.WIP.001.patch
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to
> determinate how resources of a special set of nodes could be shared by a
> group of entities (like teams, departments, etc.). Partitions of a cluster
> has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has
> priority to use the partition).
> - Percentage of capacities can apply on partition (Market team has 40%
> minimum capacity and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of
> node’s hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >=
> 2.20 && JDK.version >= 8u20 && x86_64).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
