Wangda Tan commented on YARN-2495:

Hi [~cwelch],
Thanks for jumping in and providing your thoughts, and really sorry for the 
late response.
I think biggest concern of you is about DECENTRALIZED_CONFIGURATION_ENABLED, 
let me talk about my thinkings :)

IMHO, mixing decentralized/centralized is dangerous and will cause 
non-determinated result. You may think about merging them together, such as 
some labels set by admin using RMAdminCLI, and some others are set by NM. But I 
can give you an example shows it is still non-determinated even if we have +/- 
for ResourceTracker protocol:
- Assume a node has label x,y (reported +x,+y)
- RMAdmin remove y from the node (-y)
- NM failure then restart, and report it has x,y (+x, +y). What should labels 
on the node be?

I also don't like adding too much switches in configuration, but it seems a 
good way that we can support both with determinated behavior.

For your other suggestions,
- Name changes is->are,
- Make RegisterNodeManagerRequest consist wiht NodeHeartbeatRequest
I all agree with

One more suggestion (as per suggested by [~vinodkv]), when there's anything 
wrong with node label reported from NM, we should fail NM (ask it to shutdown 
and give it proper diagnostic message). This is because if NM report a label 
but rejected, even if RM tell NM this, NM cannot handle it properly except 
print some error messages (we don't have smart logic now). Which will lead to 
problems in debugging (A NM reported some label to RM but scheduler failed 
allocating containers on the NM). To avoid it, a simple way is to shutdown the 
NM and admin can take a look at what happened.


> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch, 
> YARN-2495.20141030-1.patch, YARN-2495.20141031-1.patch, 
> YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch, 
> YARN-2495.20141204-1.patch, YARN-2495.20141208-1.patch, 
> YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or 
> using script suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

Reply via email to