Lohit Vijayarenu commented on YARN-3214:

Thanks [~wangda] for reply. I feel partitions and constraints as two separate 
entities will cause more confusion. If allocation is challenge (as you 
described in example for multiple labels), then it is something which should be 
solved in scheduler, no? This is same problem one would have even without 
labels. For a given node which advertises 10G of memory, and app/queue with X 
and Y, how would you divide resource among X and Y? 
PS: Mesos Scheduler for example uses term called constraints which is similar 
to labels. In that sense I agree with [~vinodkv] that we should probably call 
this feature as partition or something related? 

> Add non-exclusive node labels 
> ------------------------------
>                 Key: YARN-3214
>                 URL: https://issues.apache.org/jira/browse/YARN-3214
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: Non-exclusive-Node-Partition-Design.pdf
> Currently node labels partition the cluster to some sub-clusters so resources 
> cannot be shared between partitioned cluster. 
> With the current implementation of node labels we cannot use the cluster 
> optimally and the throughput of the cluster will suffer.
> We are proposing adding non-exclusive node labels:
> 1. Labeled apps get the preference on Labeled nodes 
> 2. If there is no ask for labeled resources we can assign those nodes to non 
> labeled apps
> 3. If there is any future ask for those resources , we will preempt the non 
> labeled apps and give them back to labeled apps.

This message was sent by Atlassian JIRA

Reply via email to