Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/#review143969
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 26, 2016, 7:49 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50395/
> ---
> 
> (Updated July 26, 2016, 7:49 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17888
> https://issues.apache.org/jira/browse/AMBARI-17888
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A common method of addressing failed disks, which is to set unwritable 
> permissions on a directory such as /hadoop10 within the root filesystem.
> 
> Once mounted, the mounted filesystem will overlay the correct permissions 
> structure, allowing hdfs to write to the mounted disk.
> 
> In the case of a failed mount operation, the associated /hadoop10 directory 
> will be unwritable by hdfs.
> 
> However, Ambari will see the visible directory with incorrect permissions, 
> and "fix" the permissions back to a writable directory-- resulting in the 
> datanode filling the root partition of a machine.
> 
> We need to prevent ambari from changing permissions on YARN local and HDFS 
> data dirs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> 9fa986b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  ba59a92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
> 934007b 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
> daf6e80 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
> d1365ac 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
> 7a68a07 
> 
> Diff: https://reviews.apache.org/r/50395/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/
---

(Updated July 26, 2016, 7:49 p.m.)


Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-17888
https://issues.apache.org/jira/browse/AMBARI-17888


Repository: ambari


Description
---

A common method of addressing failed disks, which is to set unwritable 
permissions on a directory such as /hadoop10 within the root filesystem.

Once mounted, the mounted filesystem will overlay the correct permissions 
structure, allowing hdfs to write to the mounted disk.

In the case of a failed mount operation, the associated /hadoop10 directory 
will be unwritable by hdfs.

However, Ambari will see the visible directory with incorrect permissions, and 
"fix" the permissions back to a writable directory-- resulting in the datanode 
filling the root partition of a machine.

We need to prevent ambari from changing permissions on YARN local and HDFS data 
dirs.


Diffs (updated)
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
9fa986b 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 ba59a92 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
934007b 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
daf6e80 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
d1365ac 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
7a68a07 

Diff: https://reviews.apache.org/r/50395/diff/


Testing
---


Thanks,

Sid Wagle

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/#review143414
---


Ship it!




Ship It!

- Sumit Mohanty


On July 25, 2016, 6:17 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50395/
> ---
> 
> (Updated July 25, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17888
> https://issues.apache.org/jira/browse/AMBARI-17888
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A common method of addressing failed disks, which is to set unwritable 
> permissions on a directory such as /hadoop10 within the root filesystem.
> 
> Once mounted, the mounted filesystem will overlay the correct permissions 
> structure, allowing hdfs to write to the mounted disk.
> 
> In the case of a failed mount operation, the associated /hadoop10 directory 
> will be unwritable by hdfs.
> 
> However, Ambari will see the visible directory with incorrect permissions, 
> and "fix" the permissions back to a writable directory-- resulting in the 
> datanode filling the root partition of a machine.
> 
> We need to prevent ambari from changing permissions on YARN local and HDFS 
> data dirs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> 9fa986b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  ba59a92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
> 934007b 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
> daf6e80 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
> d1365ac 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
> 7a68a07 
> 
> Diff: https://reviews.apache.org/r/50395/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/
---

(Updated July 25, 2016, 6:17 p.m.)


Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-17888
https://issues.apache.org/jira/browse/AMBARI-17888


Repository: ambari


Description
---

A common method of addressing failed disks, which is to set unwritable 
permissions on a directory such as /hadoop10 within the root filesystem.

Once mounted, the mounted filesystem will overlay the correct permissions 
structure, allowing hdfs to write to the mounted disk.

In the case of a failed mount operation, the associated /hadoop10 directory 
will be unwritable by hdfs.

However, Ambari will see the visible directory with incorrect permissions, and 
"fix" the permissions back to a writable directory-- resulting in the datanode 
filling the root partition of a machine.

We need to prevent ambari from changing permissions on YARN local and HDFS data 
dirs.


Diffs (updated)
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
9fa986b 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 ba59a92 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
934007b 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
daf6e80 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
d1365ac 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
7a68a07 

Diff: https://reviews.apache.org/r/50395/diff/


Testing
---


Thanks,

Sid Wagle

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/#review143395
---




ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 (line 131)


Can this throw a KeyError ?



ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 (line 216)


typo: point


- Sid Wagle


On July 25, 2016, 5:06 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50395/
> ---
> 
> (Updated July 25, 2016, 5:06 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17888
> https://issues.apache.org/jira/browse/AMBARI-17888
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A common method of addressing failed disks, which is to set unwritable 
> permissions on a directory such as /hadoop10 within the root filesystem.
> 
> Once mounted, the mounted filesystem will overlay the correct permissions 
> structure, allowing hdfs to write to the mounted disk.
> 
> In the case of a failed mount operation, the associated /hadoop10 directory 
> will be unwritable by hdfs.
> 
> However, Ambari will see the visible directory with incorrect permissions, 
> and "fix" the permissions back to a writable directory-- resulting in the 
> datanode filling the root partition of a machine.
> 
> We need to prevent ambari from changing permissions on YARN local and HDFS 
> data dirs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> 9fa986b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  ba59a92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
> 934007b 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
> daf6e80 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
> d1365ac 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
> 7a68a07 
> 
> Diff: https://reviews.apache.org/r/50395/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/
---

Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-17888
https://issues.apache.org/jira/browse/AMBARI-17888


Repository: ambari


Description
---

A common method of addressing failed disks, which is to set unwritable 
permissions on a directory such as /hadoop10 within the root filesystem.

Once mounted, the mounted filesystem will overlay the correct permissions 
structure, allowing hdfs to write to the mounted disk.

In the case of a failed mount operation, the associated /hadoop10 directory 
will be unwritable by hdfs.

However, Ambari will see the visible directory with incorrect permissions, and 
"fix" the permissions back to a writable directory-- resulting in the datanode 
filling the root partition of a machine.

We need to prevent ambari from changing permissions on YARN local and HDFS data 
dirs.


Diffs
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
9fa986b 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 ba59a92 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
934007b 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
daf6e80 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
d1365ac 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
7a68a07 

Diff: https://reviews.apache.org/r/50395/diff/


Testing
---


Thanks,

Sid Wagle