Wangda Tan commented on YARN-2496:

Craig, still about #2, I think what you commented is make sense to me, AM can 
get a more precise headroom to plan its following resource usage, but I think:
1) It may not enough as what you said:
bq. For this reason, headroom should reflect the labels in the last resource 
request from the application, not the queue's labels.
It is possible an AM sent resource requests with different label expression, so 
what we will response headroom back to AM?
I think maybe we need a new field in AllocateRequest to request different 
headrooms under different label expression.

2) Even with 1), I cannot think of a good way to fast computing random label 
expression in an acceptable time complexity, it is possible thousands of 
different label expression existed in a big cluster at the same time. Our 
current implementation can make sure resource of labels of a queue will 
up-to-date whenever resource change happened.

With 1) and 2). I suggest to make it as a pending task, and we can deal it in 
the future.

bq. -re 5, I though * could be in requests, if no, then should not be an issue.
Yes, we doesn't support specify * in requests, because it may cause some 
possible resource wastage. AM should clearly know what resource it needed.


> [YARN-796] Changes for capacity scheduler to support allocate resource 
> respect labels
> -------------------------------------------------------------------------------------
>                 Key: YARN-2496
>                 URL: https://issues.apache.org/jira/browse/YARN-2496
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2496.patch, YARN-2496.patch, YARN-2496.patch, 
> YARN-2496.patch
> This JIRA Includes:
> - Add/parse labels option to {{capacity-scheduler.xml}} similar to other 
> options of queue like capacity/maximum-capacity, etc.
> - Include a "default-label-expression" option in queue config, if an app 
> doesn't specify label-expression, "default-label-expression" of queue will be 
> used.
> - Check if labels can be accessed by the queue when submit an app with 
> labels-expression to queue or update ResourceRequest with label-expression
> - Check labels on NM when trying to allocate ResourceRequest on the NM with 
> label-expression
> - Respect  labels when calculate headroom/user-limit

This message was sent by Atlassian JIRA

Reply via email to