[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317048#comment-14317048
]
Wangda Tan commented on YARN-2495:
----------------------------------
Hi [~cwelch],
Thanks for jumping in and providing your thoughts, and really sorry for the
late response.
I think biggest concern of you is about DECENTRALIZED_CONFIGURATION_ENABLED,
let me talk about my thinkings :)
IMHO, mixing decentralized/centralized is dangerous and will cause
non-determinated result. You may think about merging them together, such as
some labels set by admin using RMAdminCLI, and some others are set by NM. But I
can give you an example shows it is still non-determinated even if we have +/-
for ResourceTracker protocol:
- Assume a node has label x,y (reported +x,+y)
- RMAdmin remove y from the node (-y)
- NM failure then restart, and report it has x,y (+x, +y). What should labels
on the node be?
I also don't like adding too much switches in configuration, but it seems a
good way that we can support both with determinated behavior.
For your other suggestions,
- Name changes is->are,
- Make RegisterNodeManagerRequest consist wiht NodeHeartbeatRequest
I all agree with
One more suggestion (as per suggested by [~vinodkv]), when there's anything
wrong with node label reported from NM, we should fail NM (ask it to shutdown
and give it proper diagnostic message). This is because if NM report a label
but rejected, even if RM tell NM this, NM cannot handle it properly except
print some error messages (we don't have smart logic now). Which will lead to
problems in debugging (A NM reported some label to RM but scheduler failed
allocating containers on the NM). To avoid it, a simple way is to shutdown the
NM and admin can take a look at what happened.
Thoughts?
Wangda
> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch,
> YARN-2495.20141030-1.patch, YARN-2495.20141031-1.patch,
> YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch,
> YARN-2495.20141204-1.patch, YARN-2495.20141208-1.patch,
> YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or
> using script suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
