[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-08-09 Thread Doroszlai, Attila (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119892#comment-16119892
 ] 

Doroszlai, Attila commented on AMBARI-21116:


[~aonishuk], found one more problem with [this 
commit|https://github.com/apache/ambari/commit/4405d4982842a729ebfbd4c000c5ffb023a5].
  In HDP 2.X recursive permissions were eliminated for the wrong directory: 
{{mapreduce_jobhistory_done_dir}} instead of {{node_labels_dir}}.

Compare:

{noformat:title=YARN/2.1.0.2.0/package/scripts/yarn.py}
   params.HdfsResource(params.mapreduce_jobhistory_done_dir,
type="directory",
 action="create_on_execute",
 owner=params.mapred_user,
 group=params.user_group,
 -   change_permissions_for_parents=True,
 mode=0777
)
{noformat}

with:

{noformat:title=YARN/3.0.0.3.0/package/scripts/yarn.py}
 params.HdfsResource(params.node_labels_dir,
   type="directory",
   action="create_on_execute",
 - change_permissions_for_parents=True,
   owner=params.yarn_user,
   group=params.user_group,
   mode=0700
 )
{noformat}

This causes MapReduce History Server to stop after startup, because it cannot 
create {{/mr-history/tmp}} due to wrong permissions on {{/mr-history}}:

{noformat:title=trunk}
drwxr-xr-x   - hdfs   hdfs0 2017-08-09 11:54 /mr-history
drwxrwxrwx   - mapred hadoop  0 2017-08-09 11:54 /mr-history/done
{noformat}

{noformat:title=branch-2.5}
drwxrwxrwx   - mapred hadoop  0 2017-08-09 12:26 /mr-history
drwxrwxrwx   - mapred hadoop  0 2017-08-09 12:26 /mr-history/done
drwxrwxrwt   - mapred hadoop  0 2017-08-09 12:26 /mr-history/tmp
{noformat}

{noformat:title=mapred-mapred-historyserver.log}
2017-08-09 11:54:20,957 INFO  service.AbstractService 
(AbstractService.java:noteFailure(272)) - Service 
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state INITED; 
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating 
intermediate done directory
...
Caused by: org.apache.hadoop.security.AccessControlException: Permission 
denied: user=mapred, access=WRITE, inode="/mr-history/tmp":hdfs:hdfs:drwxr-xr-x
...
2017-08-09 11:54:20,971 INFO  util.ExitUtil (ExitUtil.java:terminate(124)) - 
Exiting with status -1
{noformat}

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 3.0.0
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-05-25 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024707#comment-16024707
 ] 

Hudson commented on AMBARI-21116:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7538 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/7538/])
AMBARI-21116. Setting yarn.node-labels.fs-store.root-dir to a "path" (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=9dbf8f15e659a19e3b4d8a6cdd7530160089c1dd])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py


> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.5.2
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-05-25 Thread Doroszlai, Attila (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024627#comment-16024627
 ] 

Doroszlai, Attila commented on AMBARI-21116:


Thanks!

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.5.2
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-05-25 Thread Andrew Onischuk (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024622#comment-16024622
 ] 

Andrew Onischuk commented on AMBARI-21116:
--

thanks [~adoroszlai]. Committed the fix.

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.5.2
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-05-25 Thread Doroszlai, Attila (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024613#comment-16024613
 ] 

Doroszlai, Attila commented on AMBARI-21116:


[~aonishuk], can you please change python unit test as well, it's 
[failing|https://builds.apache.org/job/Ambari-trunk-Commit/7531/consoleText] 
since this comit?

{noformat}
FAIL: test_configure_default (test_historyserver.TestHistoryServer)
FAIL: test_configure_secured (test_historyserver.TestHistoryServer)
FAIL: test_start_default (test_historyserver.TestHistoryServer)
FAIL: test_start_secured (test_historyserver.TestHistoryServer)
{noformat}

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.5.2
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (AMBARI-21116) Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"

2017-05-24 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023648#comment-16023648
 ] 

Hudson commented on AMBARI-21116:
-

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7531 (See 
[https://builds.apache.org/job/Ambari-trunk-Commit/7531/])
AMBARI-21116. Setting yarn.node-labels.fs-store.root-dir to a "path" (aonishuk: 
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=4405d4982842a729ebfbd4c000c5ffb023a5])
* (edit) 
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
* (edit) 
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/yarn.py


> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission 
> of the "root path"
> 
>
> Key: AMBARI-21116
> URL: https://issues.apache.org/jira/browse/AMBARI-21116
> Project: Ambari
>  Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Andrew Onischuk
>Priority: Critical
> Fix For: 2.5.2
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>   'yarn.node-labels.enabled' : "True",
>   'yarn.node-labels.fs-store.root-dir': 
> NODE_LABEL_STORE_DIR,
>   'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx--   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing 
> other tests to fail since it cannot access /tmp  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)