Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-19 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/
---

(Updated Dec. 19, 2017, 2:10 p.m.)


Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.


Changes
---

Added "Refer **TestSettings.py** for usage."


Bugs: AMBARI-22649
https://issues.apache.org/jira/browse/AMBARI-22649


Repository: ambari


Description
---

Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
"cluster settings" in Ambari.

**==**
**Library for querying _clusterSettings_ and _stackSettings_ for its contents 
in command*.json.**
**==**

One should be able to query for a given **clusterSettings** or 
**stackSettings**:
 -  by passing in the setting name(one or more) in order to get it back as 
key-value map, or
 -  just get the value back for a passed-in setting.


**Functions for clusterSettings:**
**--**
  - **get_cluster_setting_entries(setting_names)** : 
-- Retrieves the passed-in cluster setting entr(y/ies) and their values as 
a map.
   If 'setting_names' is passed-in as None : all the settings names and 
their corresponding values will be returned as map.
   If 'setting_names' is passed-in as empty set : None will be returned.

  - **get_cluster_setting_value(setting_name)** :
-- Retrieves the passed-in cluster setting entry's value.

  - **is_security_enabled()** : 
-- Retrieves the cluster's security status.


**Functions for stackSettings:** 
****

Stack settings as of now has 5 settings : stack_name, stack_root, 
stack_features, stack_tools, stack_packages. stack_name, stack_root have string 
as values, whereas stack_features, stack_tools, stack_packages have values as 
JSON. Further there already exists python functions in files : 
**stack_features.py**, **stack_tools.py** and **stack_select.py**.

   - **get_stack_setting_entries(setting_names)** : 
  --   Retrieves the passed-in stack setting entr(y/ies) and their values 
as a map.
If 'setting_names' is passed-in as None, all the settings names and 
their corresponding values will be returned as map.
If 'setting_names' is passed-in as empty set : None will be 
returned.

   - **get_stack_setting_value(setting_name)**:
-- Retrieves the passed-in stack setting entry's value.

- **get_stack_name()**:
-- Retrieves the stack name.

- **get_stack_root()**:
   -- Retrieves the stack root.

 

**Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
files:**
**-**

- Given that these already exist and as of now they read the relevant stack 
setting from *configurations/cluster_env*. 
- Thus, code has been added to try reading from /stackSettings first by calling 
the new fn.() get_stack_setting_value(). if setting not found, go for the fall 
back  *configurations/cluster_env* (which would be removed soon, when we remove 
cluster_env).


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 92823b0 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 b741a33 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 d9233a3 
  ambari-server/src/test/python/TestSettings.py PRE-CREATION 


Diff: https://reviews.apache.org/r/64594/diff/4/


Testing (updated)
---

Python UT passes.


  Refer **TestSettings.py** for usage.


**Testing:**

Tested on live cluster


**=**
**clusterSettings:**
**=**

**A.** get_cluster_setting_entries():
**--**

  - 1. Retrieve **single** setting : 'recovery_enabled'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled'])
  **o/p**:   {'recovery_enabled': True}

   - 2. Retrieve **two** settings : 'recovery_enabled', 'sysprep_skip_setup_jce'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled', 'sysprep_skip_setup_jce'])
**o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}


- 3. Retrieve settings where passed in empty set -> Expected nothing returned
 -- In get_cluster_setting_entries(). Passed-in setting(s) :

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-19 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/
---

(Updated Dec. 19, 2017, 2:09 p.m.)


Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.


Changes
---

Updated review as per Attila's comments. Furhher, Attila had provided UT code, 
which I improvized further and added to this review request. Thanks Attila for 
that.


Bugs: AMBARI-22649
https://issues.apache.org/jira/browse/AMBARI-22649


Repository: ambari


Description
---

Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
"cluster settings" in Ambari.

**==**
**Library for querying _clusterSettings_ and _stackSettings_ for its contents 
in command*.json.**
**==**

One should be able to query for a given **clusterSettings** or 
**stackSettings**:
 -  by passing in the setting name(one or more) in order to get it back as 
key-value map, or
 -  just get the value back for a passed-in setting.


**Functions for clusterSettings:**
**--**
  - **get_cluster_setting_entries(setting_names)** : 
-- Retrieves the passed-in cluster setting entr(y/ies) and their values as 
a map.
   If 'setting_names' is passed-in as None : all the settings names and 
their corresponding values will be returned as map.
   If 'setting_names' is passed-in as empty set : None will be returned.

  - **get_cluster_setting_value(setting_name)** :
-- Retrieves the passed-in cluster setting entry's value.

  - **is_security_enabled()** : 
-- Retrieves the cluster's security status.


**Functions for stackSettings:** 
****

Stack settings as of now has 5 settings : stack_name, stack_root, 
stack_features, stack_tools, stack_packages. stack_name, stack_root have string 
as values, whereas stack_features, stack_tools, stack_packages have values as 
JSON. Further there already exists python functions in files : 
**stack_features.py**, **stack_tools.py** and **stack_select.py**.

   - **get_stack_setting_entries(setting_names)** : 
  --   Retrieves the passed-in stack setting entr(y/ies) and their values 
as a map.
If 'setting_names' is passed-in as None, all the settings names and 
their corresponding values will be returned as map.
If 'setting_names' is passed-in as empty set : None will be 
returned.

   - **get_stack_setting_value(setting_name)**:
-- Retrieves the passed-in stack setting entry's value.

- **get_stack_name()**:
-- Retrieves the stack name.

- **get_stack_root()**:
   -- Retrieves the stack root.

 

**Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
files:**
**-**

- Given that these already exist and as of now they read the relevant stack 
setting from *configurations/cluster_env*. 
- Thus, code has been added to try reading from /stackSettings first by calling 
the new fn.() get_stack_setting_value(). if setting not found, go for the fall 
back  *configurations/cluster_env* (which would be removed soon, when we remove 
cluster_env).


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 92823b0 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 b741a33 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 d9233a3 
  ambari-server/src/test/python/TestSettings.py PRE-CREATION 


Diff: https://reviews.apache.org/r/64594/diff/4/

Changes: https://reviews.apache.org/r/64594/diff/3-4/


Testing
---

Python UT passes.


**Testing:**

Tested on live cluster


**=**
**clusterSettings:**
**=**

**A.** get_cluster_setting_entries():
**--**

  - 1. Retrieve **single** setting : 'recovery_enabled'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled'])
  **o/p**:   {'recovery_enabled': True}

   - 2. Retrieve **two** settings : 'recovery_enabled', 'sysprep_skip_setup_jce'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled', 'sysprep_skip_setup_jce'])
**o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}


- 3.

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-19 Thread Swapan Shridhar 


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > I've [attached a small unit 
> > test](https://issues.apache.org/jira/secure/attachment/12902568/AMBARI-22649-test.patch)
> >  for `settings.py` to 
> > [AMBARI-22649](https://issues.apache.org/jira/browse/AMBARI-22649), please 
> > feel free to use it.  I think it simplifies testing behavior of this 
> > library.

Thanks Attila. This helps.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 28 (patched)
> > 
> >
> > Default value of `None` for `setting_names` would make it easier to get 
> > all settings.

Done and UT added.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 47-49 (patched)
> > 
> >
> > Shouldn't this also return `None`?  Now it says `/agentConfigParams` is 
> > not supported, but goes on to return the requested values.

Fixed.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 56 (patched)
> > 
> >
> > Why is a `list`, `frozenset` or `tuple` not acceptable?

Added support for them.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 60 (patched)
> > 
> >
> > This is unreachable, since `None` is not an instance of `set`, hence it 
> > already returns `None` a bit earlier, instead of returning all settings 
> > here.

Fixed.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 69 (patched)
> > 
> >
> > I think if the specified keys are not present, instead of `None` it 
> > should just return the empty `dict` that `result` already contains.
> > 
> > For example:
> > 
> > ```
> > get_setting_type_entries('/clusterSettings', None)['key']
> > ```
> > 
> > and 
> > 
> > ```
> > get_setting_type_entries('/clusterSettings', set(['key']))['key']
> > ```
> > 
> > should have the same result (actual value or `KeyError`), but returning 
> > `None` means the second one may result in `TypeError: 'NoneType' object has 
> > no attribute '__getitem__'` instead.

Fixed.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 89 (patched)
> > 
> >
> > Should `return None` early if `setting_name is None`

Fixed.


> On Dec. 17, 2017, 8:01 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/settings.py
> > Lines 105-108 (patched)
> > 
> >
> > Could be simplified to
> > 
> > ```
> > return setting_type in (STACK_SETTINGS_TYPE, CLUSTER_SETTINGS_TYPE)
> > ```

Fixed.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review194031
---


On Dec. 17, 2017, 8:11 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 17, 2017, 8:11 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
> Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-17 Thread Attila Doroszlai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review194031
---


Fix it, then Ship it!




I've [attached a small unit 
test](https://issues.apache.org/jira/secure/attachment/12902568/AMBARI-22649-test.patch)
 for `settings.py` to 
[AMBARI-22649](https://issues.apache.org/jira/browse/AMBARI-22649), please feel 
free to use it.  I think it simplifies testing behavior of this library.


ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 28 (patched)


Default value of `None` for `setting_names` would make it easier to get all 
settings.



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 47-49 (patched)


Shouldn't this also return `None`?  Now it says `/agentConfigParams` is not 
supported, but goes on to return the requested values.



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 56 (patched)


Why is a `list`, `frozenset` or `tuple` not acceptable?



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 60 (patched)


This is unreachable, since `None` is not an instance of `set`, hence it 
already returns `None` a bit earlier, instead of returning all settings here.



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 69 (patched)


I think if the specified keys are not present, instead of `None` it should 
just return the empty `dict` that `result` already contains.

For example:

```
get_setting_type_entries('/clusterSettings', None)['key']
```

and 

```
get_setting_type_entries('/clusterSettings', set(['key']))['key']
```

should have the same result (actual value or `KeyError`), but returning 
`None` means the second one may result in `TypeError: 'NoneType' object has no 
attribute '__getitem__'` instead.



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 89 (patched)


Should `return None` early if `setting_name is None`



ambari-common/src/main/python/resource_management/libraries/functions/settings.py
Lines 105-108 (patched)


Could be simplified to

```
return setting_type in (STACK_SETTINGS_TYPE, CLUSTER_SETTINGS_TYPE)
```


- Attila Doroszlai


On Dec. 17, 2017, 9:11 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 17, 2017, 9:11 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
> Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values,

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-17 Thread Swapan Shridhar 


> On Dec. 16, 2017, 12:49 p.m., Attila Doroszlai wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
> > Lines 32-88 (patched)
> > 
> >
> > Most of these 2 functions are the same as those in `cluster_settings`.  
> > Can you please try to reduce duplication?

Done.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review194013
---


On Dec. 17, 2017, 8:11 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 17, 2017, 8:11 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
> Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  92823b0 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b741a33 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  d9233a3 
> 
> 
> Diff: https://reviews.apache.org/r/64594/diff/3/
> 
> 
> Testing
> ---
> 
> Python UT passes.
> 
> 
> **Testing:**
> 
> Tested on live

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-17 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/
---

(Updated Dec. 17, 2017, 8:11 a.m.)


Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.


Changes
---

Updated cluster_settings.py and stack_settings.py code to common location 
settings.py


Bugs: AMBARI-22649
https://issues.apache.org/jira/browse/AMBARI-22649


Repository: ambari


Description
---

Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
"cluster settings" in Ambari.

**==**
**Library for querying _clusterSettings_ and _stackSettings_ for its contents 
in command*.json.**
**==**

One should be able to query for a given **clusterSettings** or 
**stackSettings**:
 -  by passing in the setting name(one or more) in order to get it back as 
key-value map, or
 -  just get the value back for a passed-in setting.


**Functions for clusterSettings:**
**--**
  - **get_cluster_setting_entries(setting_names)** : 
-- Retrieves the passed-in cluster setting entr(y/ies) and their values as 
a map.
   If 'setting_names' is passed-in as None : all the settings names and 
their corresponding values will be returned as map.
   If 'setting_names' is passed-in as empty set : None will be returned.

  - **get_cluster_setting_value(setting_name)** :
-- Retrieves the passed-in cluster setting entry's value.

  - **is_security_enabled()** : 
-- Retrieves the cluster's security status.


**Functions for stackSettings:** 
****

Stack settings as of now has 5 settings : stack_name, stack_root, 
stack_features, stack_tools, stack_packages. stack_name, stack_root have string 
as values, whereas stack_features, stack_tools, stack_packages have values as 
JSON. Further there already exists python functions in files : 
**stack_features.py**, **stack_tools.py** and **stack_select.py**.

   - **get_stack_setting_entries(setting_names)** : 
  --   Retrieves the passed-in stack setting entr(y/ies) and their values 
as a map.
If 'setting_names' is passed-in as None, all the settings names and 
their corresponding values will be returned as map.
If 'setting_names' is passed-in as empty set : None will be 
returned.

   - **get_stack_setting_value(setting_name)**:
-- Retrieves the passed-in stack setting entry's value.

- **get_stack_name()**:
-- Retrieves the stack name.

- **get_stack_root()**:
   -- Retrieves the stack root.

 

**Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
files:**
**-**

- Given that these already exist and as of now they read the relevant stack 
setting from *configurations/cluster_env*. 
- Thus, code has been added to try reading from /stackSettings first by calling 
the new fn.() get_stack_setting_value(). if setting not found, go for the fall 
back  *configurations/cluster_env* (which would be removed soon, when we remove 
cluster_env).


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 92823b0 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 b741a33 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 d9233a3 


Diff: https://reviews.apache.org/r/64594/diff/3/

Changes: https://reviews.apache.org/r/64594/diff/2-3/


Testing
---

Python UT passes.


**Testing:**

Tested on live cluster


**=**
**clusterSettings:**
**=**

**A.** get_cluster_setting_entries():
**--**

  - 1. Retrieve **single** setting : 'recovery_enabled'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled'])
  **o/p**:   {'recovery_enabled': True}

   - 2. Retrieve **two** settings : 'recovery_enabled', 'sysprep_skip_setup_jce'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled', 'sysprep_skip_setup_jce'])
**o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}


- 3. Retrieve settings where passed in empty set -> Expected nothing returned
 -- In get_cluster_setting_entries(). Passed-in setting(s) : set([])

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-16 Thread Attila Doroszlai 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review194013
---




ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
Lines 32-88 (patched)


Most of these 2 functions are the same as those in `cluster_settings`.  Can 
you please try to reduce duplication?


- Attila Doroszlai


On Dec. 14, 2017, 10:05 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 14, 2017, 10:05 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Sen, Jayush Luniya, 
> Madhuvanthi Radhakrishnan, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  92823b0 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b741a33 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  d9233a3 
> 
> 
> Diff: https://reviews.apache.org/r/64594/diff/2/
> 
> 
> Testing
> ---
> 
> Python UT passes.
> 
> 
> **Testing:**
> 
> Tested on live cluster
> 
> 
> **=**
> **clusterSettings:**
> **=**
> 
> **A.** get_cluster_setting_entries():
> **--**
> 
>   - 1. Retrieve **single**

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/
---

(Updated Dec. 14, 2017, 9:02 p.m.)


Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.


Changes
---

Updated code as per sugegstions.


Bugs: AMBARI-22649
https://issues.apache.org/jira/browse/AMBARI-22649


Repository: ambari


Description
---

Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
"cluster settings" in Ambari.

**==**
**Library for querying _clusterSettings_ and _stackSettings_ for its contents 
in command*.json.**
**==**

One should be able to query for a given **clusterSettings** or 
**stackSettings**:
 -  by passing in the setting name(one or more) in order to get it back as 
key-value map, or
 -  just get the value back for a passed-in setting.


**Functions for clusterSettings:**
**--**
  - **get_cluster_setting_entries(setting_names)** : 
-- Retrieves the passed-in cluster setting entr(y/ies) and their values as 
a map.
   If 'setting_names' is passed-in as None : all the settings names and 
their corresponding values will be returned as map.
   If 'setting_names' is passed-in as empty set : None will be returned.

  - **get_cluster_setting_value(setting_name)** :
-- Retrieves the passed-in cluster setting entry's value.

  - **is_security_enabled()** : 
-- Retrieves the cluster's security status.


**Functions for stackSettings:** 
****

Stack settings as of now has 5 settings : stack_name, stack_root, 
stack_features, stack_tools, stack_packages. stack_name, stack_root have string 
as values, whereas stack_features, stack_tools, stack_packages have values as 
JSON. Further there already exists python functions in files : 
**stack_features.py**, **stack_tools.py** and **stack_select.py**.

   - **get_stack_setting_entries(setting_names)** : 
  --   Retrieves the passed-in stack setting entr(y/ies) and their values 
as a map.
If 'setting_names' is passed-in as None, all the settings names and 
their corresponding values will be returned as map.
If 'setting_names' is passed-in as empty set : None will be 
returned.

   - **get_stack_setting_value(setting_name)**:
-- Retrieves the passed-in stack setting entry's value.

- **get_stack_name()**:
-- Retrieves the stack name.

- **get_stack_root()**:
   -- Retrieves the stack root.

 

**Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
files:**
**-**

- Given that these already exist and as of now they read the relevant stack 
setting from *configurations/cluster_env*. 
- Thus, code has been added to try reading from /stackSettings first by calling 
the new fn.() get_stack_setting_value(). if setting not found, go for the fall 
back  *configurations/cluster_env* (which would be removed soon, when we remove 
cluster_env).


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 92823b0 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 b741a33 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 d9233a3 


Diff: https://reviews.apache.org/r/64594/diff/2/

Changes: https://reviews.apache.org/r/64594/diff/1-2/


Testing (updated)
---

Python UT passes.


**Testing:**

Tested on live cluster


**=**
**clusterSettings:**
**=**

**A.** get_cluster_setting_entries():
**--**

  - 1. Retrieve **single** setting : 'recovery_enabled'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled'])
  **o/p**:   {'recovery_enabled': True}

   - 2. Retrieve **two** settings : 'recovery_enabled', 'sysprep_skip_setup_jce'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled', 'sysprep_skip_setup_jce'])
**o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}


- 3. Retrieve settings where passed in empty set -> Expected nothing returned
 -- In get_cluster_setting_entries(). Passed-in setting(s) : set([])
**o/p**:   None


- 4. Retrieve **three** settings : 'smokeuser', 'recovery_enabled', 
'sysprep_skip_setup_jce'
   -- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['smokeuser',

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Swapan Shridhar 


> On Dec. 14, 2017, 7:01 p.m., Madhuvanthi Radhakrishnan wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
> > Lines 45 (patched)
> > 
> >
> > Hi,
> > Eventually it might never happen that cluster_settings is None, but if 
> > it does then this would be a Logger.error and return None instead of 
> > Logger.info because it will throw an error at Line 54 if we continue with 
> > the function. Looks like we have this in Line 78 so need to have the same 
> > behaviour here. Thanks

Added return None after line 45. Just a safeguard as of now.


> On Dec. 14, 2017, 7:01 p.m., Madhuvanthi Radhakrishnan wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
> > Lines 53 (patched)
> > 
> >
> > What is the format that the calling function is expected to pass 
> > "setting_names" as? Do we need to do set(setting_names) if it is already 
> > being passed as a set?

Expected to be set. I was seeing in case someone has passed list, and we can 
still make it work.
Fixed it explicityuly by checking the type and returning None if not a set.


> On Dec. 14, 2017, 7:01 p.m., Madhuvanthi Radhakrishnan wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
> > Lines 50 (patched)
> > 
> >
> > return None

Added.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review193825
---


On Dec. 14, 2017, 9:02 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 14, 2017, 9:02 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Swapan Shridhar 


> On Dec. 14, 2017, 5:37 p.m., Jayush Luniya wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
> > Lines 24 (patched)
> > 
> >
> > STACK_NAME_SETTING_KEY or STACK_NAME_PROPERTY
> > 
> > Lets try not to use CONFIG to avoid confusion.

Fixed.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review193820
---


On Dec. 14, 2017, 9:02 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 14, 2017, 9:02 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  92823b0 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b741a33 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  d9233a3 
> 
> 
> Diff: https://reviews.apache.org/r/64594/diff/1/
> 
> 
> Testing
> ---
> 
> Python UT passes.
> 
> 
> **Testing:**
> 
> Tested on live cluster
> 
> 
> **=**
> **clusterSettings:**
> **=**
> 
> **A.** get_cluster_setting_entries():
> **--**
> 
>

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Madhuvanthi Radhakrishnan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review193825
---


Fix it, then Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
Lines 45 (patched)


Hi,
Eventually it might never happen that cluster_settings is None, but if it 
does then this would be a Logger.error and return None instead of Logger.info 
because it will throw an error at Line 54 if we continue with the function. 
Looks like we have this in Line 78 so need to have the same behaviour here. 
Thanks



ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
Lines 53 (patched)


What is the format that the calling function is expected to pass 
"setting_names" as? Do we need to do set(setting_names) if it is already being 
passed as a set?



ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
Lines 50 (patched)


return None



ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
Lines 58 (patched)


Remove set()


- Madhuvanthi Radhakrishnan


On Dec. 14, 2017, 9:02 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 14, 2017, 9:02 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -
> 
>   
>

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/#review193820
---




ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
Lines 24 (patched)


STACK_NAME_SETTING_KEY or STACK_NAME_PROPERTY

Lets try not to use CONFIG to avoid confusion.


- Jayush Luniya


On Dec. 14, 2017, 9:02 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64594/
> ---
> 
> (Updated Dec. 14, 2017, 9:02 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22649
> https://issues.apache.org/jira/browse/AMBARI-22649
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
> "cluster settings" in Ambari.
> 
> **==**
> **Library for querying _clusterSettings_ and _stackSettings_ for its contents 
> in command*.json.**
> **==**
> 
> One should be able to query for a given **clusterSettings** or 
> **stackSettings**:
>  -  by passing in the setting name(one or more) in order to get it back as 
> key-value map, or
>  -  just get the value back for a passed-in setting.
> 
> 
> **Functions for clusterSettings:**
> **--**
>   - **get_cluster_setting_entries(setting_names)** : 
> -- Retrieves the passed-in cluster setting entr(y/ies) and their values 
> as a map.
>If 'setting_names' is passed-in as None : all the settings names and 
> their corresponding values will be returned as map.
>If 'setting_names' is passed-in as empty set : None will be returned.
> 
>   - **get_cluster_setting_value(setting_name)** :
> -- Retrieves the passed-in cluster setting entry's value.
> 
>   - **is_security_enabled()** : 
> -- Retrieves the cluster's security status.
> 
> 
> **Functions for stackSettings:** 
> ****
> 
> Stack settings as of now has 5 settings : stack_name, stack_root, 
> stack_features, stack_tools, stack_packages. stack_name, stack_root have 
> string as values, whereas stack_features, stack_tools, stack_packages have 
> values as JSON. Further there already exists python functions in files : 
> **stack_features.py**, **stack_tools.py** and **stack_select.py**.
> 
>- **get_stack_setting_entries(setting_names)** : 
>   --   Retrieves the passed-in stack setting entr(y/ies) and their values 
> as a map.
> If 'setting_names' is passed-in as None, all the settings names 
> and their corresponding values will be returned as map.
> If 'setting_names' is passed-in as empty set : None will be 
> returned.
> 
>- **get_stack_setting_value(setting_name)**:
> -- Retrieves the passed-in stack setting entry's value.
> 
> - **get_stack_name()**:
> -- Retrieves the stack name.
> 
> - **get_stack_root()**:
>-- Retrieves the stack root.
> 
>  
> 
> **Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
> files:**
> **-**
> 
> - Given that these already exist and as of now they read the relevant stack 
> setting from *configurations/cluster_env*. 
> - Thus, code has been added to try reading from /stackSettings first by 
> calling the new fn.() get_stack_setting_value(). if setting not found, go for 
> the fall back  *configurations/cluster_env* (which would be removed soon, 
> when we remove cluster_env).
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  92823b0 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  b741a33 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  d9233a3 
> 
> 
> Diff: https://reviews.apache.org/r/64594/diff/1/
> 
> 
> Testing
> ---
> 
> Python UT passes.
> 
> 
> **Testing:**
> 
> Tested on live cluster
> 
> 
> **=**
> **clusterSettings:**
> **=**
> 
> **A.** get_cluster_setting_entries():
> **--**
> 
>   - 1. Retrieve **single** setting : 'recovery_enabled'
> -- In get_cluster_setting_entries().

Re: Review Request 64594: AMBARI-22649. Library for querying clusterSettings and stackSettings for its contents in command*.json

2017-12-14 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64594/
---

(Updated Dec. 14, 2017, 9:02 a.m.)


Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.


Bugs: AMBARI-22649
https://issues.apache.org/jira/browse/AMBARI-22649


Repository: ambari


Description
---

Background : AMBARI-22198 added "stack settings", and AMBARI-22196 introduced 
"cluster settings" in Ambari.

**==**
**Library for querying _clusterSettings_ and _stackSettings_ for its contents 
in command*.json.**
**==**

One should be able to query for a given **clusterSettings** or 
**stackSettings**:
 -  by passing in the setting name(one or more) in order to get it back as 
key-value map, or
 -  just get the value back for a passed-in setting.


**Functions for clusterSettings:**
**--**
  - **get_cluster_setting_entries(setting_names)** : 
-- Retrieves the passed-in cluster setting entr(y/ies) and their values as 
a map.
   If 'setting_names' is passed-in as None : all the settings names and 
their corresponding values will be returned as map.
   If 'setting_names' is passed-in as empty set : None will be returned.

  - **get_cluster_setting_value(setting_name)** :
-- Retrieves the passed-in cluster setting entry's value.

  - **is_security_enabled()** : 
-- Retrieves the cluster's security status.


**Functions for stackSettings:** 
****

Stack settings as of now has 5 settings : stack_name, stack_root, 
stack_features, stack_tools, stack_packages. stack_name, stack_root have string 
as values, whereas stack_features, stack_tools, stack_packages have values as 
JSON. Further there already exists python functions in files : 
**stack_features.py**, **stack_tools.py** and **stack_select.py**.

   - **get_stack_setting_entries(setting_names)** : 
  --   Retrieves the passed-in stack setting entr(y/ies) and their values 
as a map.
If 'setting_names' is passed-in as None, all the settings names and 
their corresponding values will be returned as map.
If 'setting_names' is passed-in as empty set : None will be 
returned.

   - **get_stack_setting_value(setting_name)**:
-- Retrieves the passed-in stack setting entry's value.

- **get_stack_name()**:
-- Retrieves the stack name.

- **get_stack_root()**:
   -- Retrieves the stack root.

 

**Modifications in  _stack_features.py, stack_tools.py and stack_select.py_ 
files:**
**-**

- Given that these already exist and as of now they read the relevant stack 
setting from *configurations/cluster_env*. 
- Thus, code has been added to try reading from /stackSettings first by calling 
the new fn.() get_stack_setting_value(). if setting not found, go for the fall 
back  *configurations/cluster_env* (which would be removed soon, when we remove 
cluster_env).


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/cluster_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 92823b0 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 b741a33 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_settings.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 d9233a3 


Diff: https://reviews.apache.org/r/64594/diff/1/


Testing (updated)
---

Python UT passes.


**Testing:**

Tested on live cluster


**=**
**clusterSettings:**
**=**

**A.** get_cluster_setting_entries():
**--**

  - 1. Retrieve **single** setting : 'recovery_enabled'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled'])
  **o/p**:   {'recovery_enabled': True}

   - 2. Retrieve **two** settings : 'recovery_enabled', 'sysprep_skip_setup_jce'
-- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['recovery_enabled', 'sysprep_skip_setup_jce'])
**o/p**:   {'recovery_enabled': True, 'sysprep_skip_setup_jce': False}


- 3. Retrieve settings where passed in empty set -> Expected nothing returned
 -- In get_cluster_setting_entries(). Passed-in setting(s) : set([])
**o/p**:   None


- 4. Retrieve **three** settings : 'smokeuser', 'recovery_enabled', 
'sysprep_skip_setup_jce'
   -- In get_cluster_setting_entries(). Passed-in setting(s) : 
set(['smokeuser', 'recovery_enabled', 'sysprep_skip_setup_jce'])
  **o/p**:  {'recovery_enabled': True, 'sysprep_skip_setup_jce':