[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119892#comment-16119892
 ] 

Doroszlai, Attila commented on AMBARI-21116:
--------------------------------------------

[~aonishuk], found one more problem with [this 
commit|https://github.com/apache/ambari/commit/4405d4982842a729ebfbd4c000c5ffb023abbbb5].
  In HDP 2.X recursive permissions were eliminated for the wrong directory: 
{{mapreduce_jobhistory_done_dir}} instead of {{node_labels_dir}}.

Compare:

{noformat:title=YARN/2.1.0.2.0/package/scripts/yarn.py}
   params.HdfsResource(params.mapreduce_jobhistory_done_dir,
                        type="directory",
                         action="create_on_execute",
                         owner=params.mapred_user,
                         group=params.user_group,
 -                       change_permissions_for_parents=True,
                         mode=0777
    )
{noformat}

with:

{noformat:title=YARN/3.0.0.3.0/package/scripts/yarn.py}
     params.HdfsResource(params.node_labels_dir,
                           type="directory",
                           action="create_on_execute",
 -                         change_permissions_for_parents=True,
                           owner=params.yarn_user,
                           group=params.user_group,
                           mode=0700
     )
{noformat}

This causes MapReduce History Server to stop after startup, because it cannot 
create {{/mr-history/tmp}} due to wrong permissions on {{/mr-history}}:

{noformat:title=trunk}
drwxr-xr-x   - hdfs   hdfs            0 2017-08-09 11:54 /mr-history
drwxrwxrwx   - mapred hadoop          0 2017-08-09 11:54 /mr-history/done
{noformat}

{noformat:title=branch-2.5}
drwxrwxrwx   - mapred hadoop          0 2017-08-09 12:26 /mr-history
drwxrwxrwx   - mapred hadoop          0 2017-08-09 12:26 /mr-history/done
drwxrwxrwt   - mapred hadoop          0 2017-08-09 12:26 /mr-history/tmp
{noformat}

{noformat:title=mapred-mapred-historyserver.log}
2017-08-09 11:54:20,957 INFO  service.AbstractService 
(AbstractService.java:noteFailure(272)) - Service 
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state INITED; 
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating 
intermediate done directory
...
Caused by: org.apache.hadoop.security.AccessControlException: Permission 
denied: user=mapred, access=WRITE, inode="/mr-history/tmp":hdfs:hdfs:drwxr-xr-x
...
2017-08-09 11:54:20,971 INFO  util.ExitUtil (ExitUtil.java:terminate(124)) - 
Exiting with status -1
{noformat}

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> ------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-21116
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21116
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Sumana Sathish
>            Assignee: Andrew Onischuk
>            Priority: Critical
>             Fix For: 3.0.0
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>                       'yarn.node-labels.enabled' : "True",
>                       'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>                       'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx------   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to