Allen Wittenauer commented on YARN-2424:

This fix is all about EOU and operability.  I can certainly understand the 
desire to run cgroups without needing local users. But transitioning to 
security is not a binary process for most users (or, at least, it doesn't have 
to be...)

The problem with the current code base is that someone moving to a secure mode 
now has to either enable cgroups (which, as pointed out in YARN-1253 is 
irrelevant for security) or cut everything over at once.  Enabling LCE prior to 
enabling security allows for a two step transition and eases problem 
determination when doing the security upgrade.  Is that user missing from the 
system or is Kerberos failing?  Clearly the issues stemming from the former can 
be sorted out without security.  This makes the operations side of the house 
much easier.

It's also worth pointing out that one of the key benefits of running tasks as 
the user who submitted them is that it makes troubleshooting much easier.  When 
one hops on a node, it is evident as to which user's tasks one is looking at 
it, even if those tasks aren't validated as "that" user.  This is especially 
important in heavy multi-tenant  scenarios.

But, again, the fix in YARN-1253 caused a regression.  LCE w/out security was 
supported prior to Hadoop 2.3 and was definitely used by people.    This change 
still sets the default to be LCE w/either one user or security, but now for 
folks who want the prior behavior, they can flip a flag and get it.

> LCE should support non-cgroups, non-secure mode
> -----------------------------------------------
>                 Key: YARN-2424
>                 URL: https://issues.apache.org/jira/browse/YARN-2424
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.3.0, 2.4.0, 2.5.0, 2.4.1
>            Reporter: Allen Wittenauer
>            Priority: Blocker
>              Labels: regression
>         Attachments: YARN-2424.patch
> After YARN-1253, LCE no longer works for non-secure, non-cgroup scenarios.  
> This is a fairly serious regression, as turning on LCE prior to turning on 
> full-blown security is a fairly standard procedure.

This message was sent by Atlassian JIRA

Reply via email to