> On July 5, 2016, 7:39 p.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py,
> >  line 1476
> > <https://reviews.apache.org/r/49590/diff/1/?file=1436300#file1436300line1476>
> >
> >     Based on this impl, we would get 1 warning at a time vs getting all dir 
> > with issues. Is this pattern consistent with other validations?

We are validating property 'dfs.datanode.data.dir'. Usually stack_advisor 
returns 1 warning for a property. Per dir warnings can flood UI. I think it 
should be 1 warning with all issues described in it's message.

Host mount configurations can differ, so basicly some dirs can be be valid on 
some hosts and invalid on other. The detailed message should contain something 
like:
warnings.append("Host: " + hostName + "; Mount: " + mountPoint + "; Data 
directories: " + ", ".join(dirList))
for each mount of each host. But it could be too long on clusters with ~1000 
nodes.

So currently the message displays only an affected host list.


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140859
-----------------------------------------------------------


On July 6, 2016, 12:40 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> -----------------------------------------------------------
> 
> (Updated July 6, 2016, 12:40 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
>     https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
>     
>     
>     
>     Traceback (most recent call last):
>       File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in <module>
>         DataNode().execute()
>       File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
>         method(env)
>       File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
>         self.start(env, upgrade_type=upgrade_type)
>       File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
>         self.configure(env)
>       File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
>         datanode(action="configure")
>       File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
>         return fn(*args, **kwargs)
>       File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
>         data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>       File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
>         raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
>     resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
>     
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  2a859ed 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  9574ce5 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4dfb8af 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Reply via email to