Naganarasimha G R commented on YARN-3214:

Hi [~wangda], 
I had a query (not sure whether this is the jira i need to discuss this though)
IIUC when Labels Requirement started we were trying to cater to 2 kinds of 
# Similar to the current jira, label the nodes and try to partition the cluster 
and ensure few queues/users get particular partition of nodes with high 
priority.(Multi Tenant scenario)
# Tagging the node with particular labels (like high MEM Nodes, More CPU cores, 
has more or particular kind of GPU's,  has particular library version , java 
version etc...) and trying to launch apps based on these tags.

Currently it seems like we are only focusing on first kind, and almost not 
supporting second one at all (we are not even accepting more than a label for a 
node), So i was thinking like we can support 2 kinds of labels; First kind of 
Labels which we will be able to support capacity  and second kind of label for 
tagging. And any given node can have at most one label of the first kind (one 
on which capacity can be specified ) and multiple tag kind of labels. App can 
specify label expression on tag kind of labels.
Correct me if my understanding is wrong or if there can be still limitations 
with the above said approach too.

> Add non-exclusive node labels 
> ------------------------------
>                 Key: YARN-3214
>                 URL: https://issues.apache.org/jira/browse/YARN-3214
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: Non-exclusive-Node-Partition-Design.pdf
> Currently node labels partition the cluster to some sub-clusters so resources 
> cannot be shared between partitioned cluster. 
> With the current implementation of node labels we cannot use the cluster 
> optimally and the throughput of the cluster will suffer.
> We are proposing adding non-exclusive node labels:
> 1. Labeled apps get the preference on Labeled nodes 
> 2. If there is no ask for labeled resources we can assign those nodes to non 
> labeled apps
> 3. If there is any future ask for those resources , we will preempt the non 
> labeled apps and give them back to labeled apps.

This message was sent by Atlassian JIRA

Reply via email to