[jira] [Commented] (YARN-4650) The AM should be launched with its own set of configs instead of using the NM's configs

2016-03-02 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176322#comment-15176322
 ] 

Vinod Kumar Vavilapalli commented on YARN-4650:
---

Trying to understand the problem and solution being addressed here. May be I am 
missing something, but I actually don't see a major change from what we already 
have today.

>From the beginning of YARN, we've been very careful about apps not relying on 
>server configuration. In theory it is still possible for an app to hard-code 
>and depend on server configuration (via 
>{{ApplicationConstants.Environment.HADOOP_CONF_DIR}} / 
>{{YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH}}), but things like 
>rolling-upgrades (YARN-666) further forced our users to not play such tricks.

bq. The AM should be launched with its own set of configs instead of using the 
NM's configs
For most of our apps (MapReduce, Tez, Spark etc), this already doesn't happen 
by default. MR for example depends on job-configuration 
{{mapreduce.application.classpath}}. In all these cases, all the configuration 
needed by AMs is usually supposed to come from the client itself. Only 
DistributedShell is the corner-case that by default depends on NM Configuration 
via {{DEFAULT_YARN_APPLICATION_CLASSPATH}}.

bq. There are cases, such as a secure LDAP configuration where the NM may need 
access to credentials that should not be exposed to the user. As long as the NM 
and AM share the same configuration files, anything exposed to the NM is also 
exposed to the AM and hence the users.
This is  already possible to do right now, *without* breaking most of our 
well-behave apps: an admin can simply (a) remove HADOOP_CONF_DIR from NM 
white-list and/or (b) change the permissions of the NMs configs to be very 
restrictive.

> The AM should be launched with its own set of configs instead of using the 
> NM's configs
> ---
>
> Key: YARN-4650
> URL: https://issues.apache.org/jira/browse/YARN-4650
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: nodemanager
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>
> There are cases, such as a secure LDAP configuration where the NM may need 
> access to credentials that should not be exposed to the user.  As long as the 
> NM and AM share the same configuration files, anything exposed to the NM is 
> also exposed to the AM and hence the users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (YARN-4650) The AM should be launched with its own set of configs instead of using the NM's configs

2016-01-28 Thread Arun Suresh (JIRA)

[ 
https://issues.apache.org/jira/browse/YARN-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122000#comment-15122000
 ] 

Arun Suresh commented on YARN-4650:
---

[~templedf], looks like this would be related to YARN-4083. While this seeks to 
provide different set of configs (from the NM) to the AM, YARN-4083 is more 
interested in the scheduler address.

I believe [~subru]/[~kishorch] had proposed to replace the HADOOP_CONF_DIR with 
a HADOOP_CLIENT_CONF_DIR with different set of *-site files before launching 
the AM (v9 patch of YARN-2884)

> The AM should be launched with its own set of configs instead of using the 
> NM's configs
> ---
>
> Key: YARN-4650
> URL: https://issues.apache.org/jira/browse/YARN-4650
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: nodemanager
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
>
> There are cases, such as a secure LDAP configuration where the NM may need 
> access to credentials that should not be exposed to the user.  As long as the 
> NM and AM share the same configuration files, anything exposed to the NM is 
> also exposed to the AM and hence the users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)