Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-06 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review141037
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 6, 2016, 1:43 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 6, 2016, 1:43 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> c33a295 
>   ambari-agent/src/test/python/resource_management/TestFileSystem.py 925758c 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  2a859ed 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  9574ce5 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4dfb8af 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-06 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140993
---


Ship it!




Ship It!

- Dmytro Sen


On Июль 6, 2016, 1:43 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated Июль 6, 2016, 1:43 п.п.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> c33a295 
>   ambari-agent/src/test/python/resource_management/TestFileSystem.py 925758c 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  2a859ed 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  9574ce5 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4dfb8af 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-06 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/
---

(Updated July 6, 2016, 1:43 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.


Bugs: AMBARI-17550
https://issues.apache.org/jira/browse/AMBARI-17550


Repository: ambari


Description
---

While changing NN and DN directories from ambari, for example:  
changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**

The values are changed without being validated  
This leads to datanodes start failing




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 174, in 
DataNode().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 709, in restart
self.start(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 60, in start
self.configure(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 55, in configure
datanode(action="configure")
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
 line 53, in datanode
data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
params.dfs_data_dirs, params.data_dir_mount_file, params)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
 line 158, in handle_mounted_dirs
raise Fail(message + " . Please turn off 
cluster-env/one_dir_per_partition or handle the situation manually.")
resource_management.core.exceptions.Fail: Trying to create another 
directory on the following mount: /grid/0 . Please turn off 
cluster-env/one_dir_per_partition or handle the situation manually.


The test fails because it set invalid value, and the directory was not created  
A warning message informing why the new directory name is invalid could be
useful


Diffs (updated)
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
c33a295 
  ambari-agent/src/test/python/resource_management/TestFileSystem.py 925758c 
  
ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
 2a859ed 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 9574ce5 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
06f7cfe 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
7a092fc 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 08b9554 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 4dfb8af 

Diff: https://reviews.apache.org/r/49590/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-06 Thread Andrew Onischuk 


> On July 5, 2016, 7:39 p.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py,
> >  line 1476
> > 
> >
> > Based on this impl, we would get 1 warning at a time vs getting all dir 
> > with issues. Is this pattern consistent with other validations?

We are validating property 'dfs.datanode.data.dir'. Usually stack_advisor 
returns 1 warning for a property. Per dir warnings can flood UI. I think it 
should be 1 warning with all issues described in it's message.

Host mount configurations can differ, so basicly some dirs can be be valid on 
some hosts and invalid on other. The detailed message should contain something 
like:
warnings.append("Host: " + hostName + "; Mount: " + mountPoint + "; Data 
directories: " + ", ".join(dirList))
for each mount of each host. But it could be too long on clusters with ~1000 
nodes.

So currently the message displays only an affected host list.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140859
---


On July 6, 2016, 12:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 6, 2016, 12:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  2a859ed 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  9574ce5 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4dfb8af 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-06 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/
---

(Updated July 6, 2016, 12:40 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.


Bugs: AMBARI-17550
https://issues.apache.org/jira/browse/AMBARI-17550


Repository: ambari


Description
---

While changing NN and DN directories from ambari, for example:  
changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**

The values are changed without being validated  
This leads to datanodes start failing




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 174, in 
DataNode().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 709, in restart
self.start(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 60, in start
self.configure(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
 line 55, in configure
datanode(action="configure")
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
 line 53, in datanode
data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
params.dfs_data_dirs, params.data_dir_mount_file, params)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
 line 158, in handle_mounted_dirs
raise Fail(message + " . Please turn off 
cluster-env/one_dir_per_partition or handle the situation manually.")
resource_management.core.exceptions.Fail: Trying to create another 
directory on the following mount: /grid/0 . Please turn off 
cluster-env/one_dir_per_partition or handle the situation manually.


The test fails because it set invalid value, and the directory was not created  
A warning message informing why the new directory name is invalid could be
useful


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
 2a859ed 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 9574ce5 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
06f7cfe 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
7a092fc 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 08b9554 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 4dfb8af 

Diff: https://reviews.apache.org/r/49590/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-05 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140859
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(line 1476)


Based on this impl, we would get 1 warning at a time vs getting all dir 
with issues. Is this pattern consistent with other validations?


- Sid Wagle


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-05 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140801
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(line 1453)


Can we move this out of stack advisor and into common services? We already 
have a file called mounted_dirs_helper.py


- Alejandro Fernandez


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-05 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140798
---


Ship it!




Ship It!

- Sid Wagle


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>