-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review193820
-----------------------------------------------------------




ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
Lines 24 (patched)
<https://reviews.apache.org/r/64594/#comment272454>

    STACK_NAME_SETTING_KEY or STACK_NAME_PROPERTY
    
    Lets try not to use CONFIG to avoid confusion.


- Jayush Luniya


On Dec. 14, 2017, 9:02 a.m., Swapan Shridhar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> -----------------------------------------------------------
> 
> (Updated Dec. 14, 2017, 9:02 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22649
>     https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==========================================================================================**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==========================================================================================**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **------------------------------**
>   - **get_cluster_setting_entries(setting_names)** : 
>     -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>        If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>        If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
>     -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
>     -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> **----------------------------**
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>    - **get_stack_setting_entries(setting_names)** : 
>       --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
>             If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
>             If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>    - **get_stack_setting_value(setting_name)**:
>     -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
>     -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>    -- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-----------------------------------------------------------------------------**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  92823b0 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b741a33 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  d9233a3 
> 
> 
> Diff: https://reviews.apache.org/r/64594/diff/1/
> 
> 
> Testing
> -------
> 
> Python UT passes.
> 
> 
> **Testing:**
> 
> Tested on live cluster
> 
> 
> **=================**
> **clusterSettings:**
> **=================**
> 
> **A.** get_cluster_setting_entries():
> **----------------------------------**
> 
>   - 1. Retrieve **single** setting : 'recovery_enabled'
>     -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['recovery_enabled'])
>       **o/p**:   {'recovery_enabled': True}
> 
>    - 2. Retrieve **two** settings : 'recovery_enabled', 
> 'sysprep_skip_setup_jce'
>     -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['recovery_enabled', 'sysprep_skip_setup_jce'])
>         **o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}
> 
> 
> - 3. Retrieve settings where passed in empty set -> Expected nothing returned
>  -- In get_cluster_setting_entries(). Passed-in setting(s) : set([])
>     **o/p**:   None
> 
> 
> - 4. Retrieve **three** settings : 'smokeuser', 'recovery_enabled', 
> 'sysprep_skip_setup_jce'
>    -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce'])
>       **o/p**:  {'recovery_enabled': True, 'sysprep_skip_setup_jce': False, 
> 'smokeuser': 'ambari-qa'}
> 
> 
> - 5. Retrieve **three** settings where **middle setting is non-existent**
>  -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['recovery_enabled', 'abc', 'sysprep_skip_setup_jce'])
>      **o/p** :  {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}
> 
> 
> - 6. Retrieve **three** settings where **1st setting is non-existent**
>    -- In get_cluster_setting_entries(). Passed-in setting(s) : set(['abc', 
> 'recovery_enabled', 'sysprep_skip_setup_jce'])
>        **o/p** :   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}
> 
> 
> - 7. Retrieve **three** settings where **last setting is non-existent**
>   -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['recovery_enabled', 'sysprep_skip_setup_jce', 'abc'])
>      **o/p**:  {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}
> 
> 
>  - 8. Retrieve passed in setting which is **non-existent**
>    -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['non-existent1'])
>        **o/p** :  None
> 
> 
>  - 9. Retrieve **two** passed in settings and both are non-existent
>   -- In get_cluster_setting_entries(). Passed-in setting(s) : 
> set(['non-existent1', 'non-existent2'])
>      **o/p** :  None
> 
>      
> - 10. Retrieve settings where set passed in is None.  -> **returns all 
> settings.**
>       --  In get_cluster_setting_entries(). Passed-in setting(s) : None
>           **o/p**:   {'security_enabled': 'false', 
> 'namenode_rolling_restart_timeout': '4200', 'enable_external_ranger': 
> 'false', 'override_uid': 'true', 'kerberos_domain': 'EXAMPLE.COM', 
> 'one_dir_per_partition': 'false', 'agent_mounts_ignore_list': '', 
> 'repo_ubuntu_template': '{{package_type}} {{base_url}} {{components}}', 
> 'ignore_groupsusers_create': 'false', 'alerts_repeat_tolerance': '1', 
> 'hide_yarn_memory_widget': 'false', 'fetch_nonlocal_groups': 'true', 
> 'manage_dirs_on_root': 'true', 'recovery_lifetime_max_count': '1024', 
> 'recovery_type': 'AUTO_START', 'ignore_bad_mounts': 'false', 
> 'recovery_window_in_minutes': '60', 'sysprep_skip_copy_tarballs_hdfs': 
> 'false', 'user_group': 'hadoop', 
> 'namenode_rolling_restart_safemode_exit_timeout': '3600', 
> 'recovery_retry_interval': '5', 'sysprep_skip_copy_oozie_share_lib_to_hdfs': 
> 'false', 'sysprep_skip_setup_jce': 'false', 'manage_hive_fsroot': 'true', 
> 'service_check_type': 'full', 'recovery_enabled': 'true', 
> 'recovery_max_count': '6', 's
 ysprep_skip_create_users_and_groups': 'false', 'smokeuser_keytab': 
'/etc/security/keytabs/smokeuser.headless.keytab', 
'managed_hdfs_resource_property_names': 'false', 'smokeuser': 'ambari-qa', 
'sysprep_skip_copy_fast_jar_hdfs': 'false'}
>      
> 
> **B.** get_cluster_setting_value():
> **--------------------------------**
> 
>    - 1. Retrieve cluster_setting : 'hide_yarn_memory_widget'  
>      -- In get_cluster_setting_value(). Passed-in setting : 
> hide_yarn_memory_widget
>         **o/p** : false
> 
>    - 2. Retrieve cluster_setting : 'recovery_max_count'  
>       -- In get_cluster_setting_value(). Passed-in setting : 
> recovery_max_count
>          **o/p** : 6
> 
>    - 3. Retrieve cluster_setting where passed in setting is 'non_existing' 
> --> Empty string
>         -- In get_cluster_setting_value(). Passed-in setting : non_existing
>            **o/p** : None
> 
>     - 4. Retrieve cluster_setting setting passed in is "None"  
>          -- In get_cluster_setting_value(). Passed-in setting : None
>             **o/p** : None   
> 
> 
> **C**. is_security_enabled():
> **-------------------------**
> 
>     - 1. Retrieve security state of cluster by calling 
> 'is_security_enabled()' which in turn calls **get_cluster_setting_value()**
>          -- In get_cluster_setting_value(). Passed-in setting : 
> security_enabled
>             **o/p** : false  
> 
> 
> **=================**
> **stackSettings**:
> **=================**
> 
> 
> **A.** Tests for **get_stack_setting_entries()**
> **--------------------------------------------**
> 
> - 1. Retrieve : **stack_name**
> 
>   --  In get_stack_setting_entries(). Passed-in setting(s) : 
> set(['stack_name'])
>       o/p:  {'stack_name': 'HDP'}
> 
> - 2. Retrieve : **stack_root**
>   -- In get_stack_setting_entries(). Passed-in setting(s) : 
> set(['stack_root'])
>      **o/p**:   {'stack_root': '{"HDP":"/usr/hdp"}'}
> 
> 
> - 3. Retrieve **three** stack settings:  **'stack_name', 'stack_root', 
> 'stack_tools'**
>   --  In get_stack_setting_entries(). Passed-in setting(s) : 
> set(['stack_name', 'stack_root', 'stack_tools'])
>       **o/p** :   {'stack_name': 'HDP', 'stack_root': '{"HDP":"/usr/hdp"}', 
> 'stack_tools': '{\n  "HDP": {\n    "stack_selector": [\n      "hdp-select",\n 
>      "/usr/bin/hdp-select",\n      "hdp-select"\n    ],\n    "conf_selector": 
> [\n      "conf-select",\n      "/usr/bin/conf-select",\n      "conf-select"\n 
>    ]\n  }\n}'}
> 
> - 4. Retrieve **four** stack settings : **'BAD_VALUE', 'stack_name', 
> 'stack_root', 'stack_tools'** ---> (**BAD_VALUE** should get ignored)
>   --  In get_stack_setting_entries(). Passed-in setting(s) : 
> set(['stack_name', 'stack_root', 'stack_tools'])
>       **o/p** :   {'stack_name': 'HDP', 'stack_root': '{"HDP":"/usr/hdp"}', 
> 'stack_tools': '{\n  "HDP": {\n    "stack_selector": [\n      "hdp-select",\n 
>      "/usr/bin/hdp-select",\n      "hdp-select"\n    ],\n    "conf_selector": 
> [\n      "conf-select",\n      "/usr/bin/conf-select",\n      "conf-select"\n 
>    ]\n  }\n}'}
> 
> 
> **B.** Tests for existing and modified : **stack_select.get_packages()** 
> which internally now calls **stack_settings.get_stack_setting_value()**
> **------------------------------------------------------------------------------------------**
> 
> - 1. Retrieve **specific component package** with call -> 
> stack_select.get_packages(orchestration, "HDFS", "DATANODE")
>   -- In get_stack_setting_value(). Passed-in setting : stack_packages
>      **o/p** : ['hadoop-hdfs-datanode'] 
> 
> - 2. Retrieve **specific component package** with call -> 
> stack_select.get_packages(orchestration, "ZOOKEEPER", "ZOOKEEPER_CLIENT")
>   -- In get_stack_setting_value(). Passed-in setting : stack_packages
>      **o/p** : ['zookeeper-client']
> 
> - 3. Retrieve **non-existing component package** with call -> 
> stack_select.get_packages(orchestration, "ZOOKEEPER", "NON_EXISTING")
>   -- In get_stack_setting_value(). Passed-in setting : stack_packages
>      Skipping stack-select on ABC because it does not exist in the 
> stack-select package structure.
>      **o/p** : None
> 
> 
> **C**. Tests for existing and modified : 
> **stack_features.check_stack_feature()** which internally now calls 
> **stack_settings.get_stack_setting_value()**
> **------------------------------------------------------------------------------------------**
> 
> - 1. Retrieve  **specific component stack feature** with call -> 
> stack_features.check_stack_feature("hive_server_interactive", "2.6")    
>   -- In get_stack_setting_value(). Passed-in setting : stack_features
>      **o/p** : True
> 
> - 2. Retrieve  **specific component stack feature** with call -> 
> stack_features.check_stack_feature("hive_server_interactive", "2.2")    
>   -- In get_stack_setting_value(). Passed-in setting : stack_features
>      **o/p** : False      
> 
> **D**. Test for existing and modified : **stack_tools.get_stack_tool()** 
> which internally now calls **stack_settings.get_stack_setting_value()**
> **-------------------------------------------------------------------------------------------**
> 
>    - 1. Retrieve **stack relevant tools** with call -> 
> stack_tools.get_stack_tool("stack_selector")   
>     -- In get_stack_setting_value(). Passed-in setting : stack_tools
>        **o/p**: ('hdp-select', '/usr/bin/hdp-select', 'hdp-select')
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Reply via email to