Wangda Tan commented on YARN-3214:

Hi Naga,
Thanks for review,
First I need to day, what we plan to do for this JIRA and already committed 
patches of YARN-2492 is not only partitioning, it's partition and tagging nodes 
(but each node can have at most one partition). This is to make scheduler part 
consistent, and each queue can have a dedicated portion of capacities on 
different partitions.

Regarding {{And any given node can have at most one label of the first kind 
(one on which capacity can be specified ) and multiple tag kind of labels. App 
can specify label expression on tag kind of labels.}}.
We're working on a design for this -- support multiple labels in each node -- 
we have some discussions internally, there're some workable ways to do it, but 
they're not perfect (some limitations with these approach like you said). Will 
post design doc once the proposal get polished.


> Add non-exclusive node labels 
> ------------------------------
>                 Key: YARN-3214
>                 URL: https://issues.apache.org/jira/browse/YARN-3214
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: Non-exclusive-Node-Partition-Design.pdf
> Currently node labels partition the cluster to some sub-clusters so resources 
> cannot be shared between partitioned cluster. 
> With the current implementation of node labels we cannot use the cluster 
> optimally and the throughput of the cluster will suffer.
> We are proposing adding non-exclusive node labels:
> 1. Labeled apps get the preference on Labeled nodes 
> 2. If there is no ask for labeled resources we can assign those nodes to non 
> labeled apps
> 3. If there is any future ask for those resources , we will preempt the non 
> labeled apps and give them back to labeled apps.

This message was sent by Atlassian JIRA

Reply via email to