[
https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099292#comment-16099292
]
Wangda Tan commented on YARN-3409:
----------------------------------
[~kkaranasos],
Add my thoughts to your questions, [[email protected]] please add
yours if you think differently.
bq. It was not clear to me how the newly added node attributes are going to
play with existing node labels. Is the plan to share some code or will it be
completely separate?
There're still many differences between partition and attribute, for example,
we don't need queue-acl for node-attribute, and after revisit existing
implementation, we may not need to support multiple NM launched on the same
host with different ports as well.
It may share some basic implementations (like node-label-manager), however for
the API level it might be better to have a separate node-attribute-protocol,
since adding them to NodeLabelProto looks too complex.
The main part of "2. API proto changes" should be read as proposal#1, and
"Alternate Proposal 1"/"Alternate Proposal 2" should be read as proposal #2 and
#3.
bq. Re: how applications will be specifying node attribute constraints.
I think so, right? +Naga.
bq. In the CLI API the replace and update seem a bit confusing to me ...
Regarding to semantics of node attribute CLI, I think we all agree with
{{update}} (adding new constraints or replacing the value of existing ones).
Instead of calling it {{update}} or {{set}}, how about call it {{add}} (which
overwrite value if key presents)?
I suggest to keep {{replace}} as a single operation, since it is a superset of
{{remove}} and {{remove+add}}, which we can provide atomic op as well. Also, I
think it might be more frequently used by end users instead of a plain
{{remove}} (what is the scenario we need to clean up all attribute on a node?).
{{node1:att1=val1}} looks better than {{node1=att1:val1}}.
> Support Node Attribute functionality
> ------------------------------------
>
> Key: YARN-3409
> URL: https://issues.apache.org/jira/browse/YARN-3409
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: api, capacityscheduler, client
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: 3409-apiChanges_v2.pdf (4).pdf,
> Constraint-Node-Labels-Requirements-Design-doc_v1.pdf, YARN-3409.WIP.001.patch
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to
> determinate how resources of a special set of nodes could be shared by a
> group of entities (like teams, departments, etc.). Partitions of a cluster
> has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has
> priority to use the partition).
> - Percentage of capacities can apply on partition (Market team has 40%
> minimum capacity and Dev team has 60% of minimum capacity of the partition).
> Attributes are orthogonal to partition, they’re describing features of node’s
> hardware/software just for affinity. Some example of attributes:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >=
> 2.20 && JDK.version >= 8u20 && x86_64).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
