Vinod Kumar Vavilapalli updated YARN-3399:
    Summary: Consider having a Default cluster ID  (was: Default cluster ID for 

Editing title to be appropriate.

Others commented on YARN-3040. So I'll try to summarize the discussion from 
YARN-1029 and YARN-3040.
 - We should have a generic {{yarn.cluster-id}} and deprecate the current RM 
only configuration
 - We need to have a reasonable default cluster-id
    -- This is needed for the Timeline service functionality - we want to 
gather insights per cluster
    -- Forcing admins to set the ID explicitly is one more hurdle w.r.t 
    -- For single node non-HA clusters, forcing the dev/admin to set it is 
 - But there are concerns too
    -- Default cluster-id can potentially cause hard-to-debug issues in HA mode.
 - Other constraints while picking a default cluster ID
    -- Restarting RM on the same node shouldn't change the cluster-id

So, I propose that we set the default cluster-ID to be something like 
"default-$(RM-host-name)-cluster". This way
 - by default, single node clusters are good across RM restarts, unless you are 
running active/standby RMs on the same machine (dev environments)
 - HA RMs have to be setup explicitly to be part of the same cluster - thereby 
avoiding debuggability issues.
 - For real life use, in order to facilitate RM migrations, administrators will 
set their own cluster-id.

> Consider having a Default cluster ID
> ------------------------------------
>                 Key: YARN-3399
>                 URL: https://issues.apache.org/jira/browse/YARN-3399
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Zhijie Shen
>            Assignee: Brahma Reddy Battula
> In YARN-3040, timeline service will set the default cluster ID if users don't 
> provide one. RM HA's current behavior is a bit different when users don't 
> provide cluster ID. IllegalArgumentException will throw instead. Let's 
> continue the discussion if RM HA needs the default cluster ID or not here, 
> and what's the proper default cluster ID.

This message was sent by Atlassian JIRA

Reply via email to