Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and removing LLAP queue when Interactive query is turned on and off respectively.

2016-04-15 Thread Swapan Shridhar 


> On April 14, 2016, 6:03 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml,
> >  line 360
> > 
> >
> > Is this correct, cluster_name?

Yes. Suggested by Sid.


> On April 14, 2016, 6:03 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml,
> >  line 28
> > 
> >
> > This means that ambari upgrade will not add this property, but because 
> > it is new, then stack upgrade from HDP 2.4 to 2.5 will add it. Is that 
> > correct?
> > DOes it need a value?

It doesn't need a value. Used for the reaosn that initially capacity-scheduler 
reads props from its xml and later in lifecycle Ambari UI and backend talks in 
terms of all properties coalesced as a bag of string 
(key=value\nkey=value\nkey=value\n). So, this becomes a placeholder for 
setting values at backend and be passed back to UI.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review128936
---


On April 14, 2016, 9:01 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 14, 2016, 9:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - When Hive Server Interactive is turned ON, Ambari will create 'llap' named 
> YARN queue if only 'default' queue exists at root level and is consuming 100% 
> capacity.
> - When Hive Server Interactive is turned OFF, Ambari will delete 'llap' queue 
> only if 2 queues exist at root level namely, 'llap' and 'default'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and deletion logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and removing LLAP queue when Interactive query is turned on and off respectively.

2016-04-14 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review128936
---


Fix it, then Ship it!





ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 (line 360)


Is this correct, cluster_name?



ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
 (line 28)


This means that ambari upgrade will not add this property, but because it 
is new, then stack upgrade from HDP 2.4 to 2.5 will add it. Is that correct?
DOes it need a value?


- Alejandro Fernandez


On April 14, 2016, 9:01 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 14, 2016, 9:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - When Hive Server Interactive is turned ON, Ambari will create 'llap' named 
> YARN queue if only 'default' queue exists at root level and is consuming 100% 
> capacity.
> - When Hive Server Interactive is turned OFF, Ambari will delete 'llap' queue 
> only if 2 queues exist at root level namely, 'llap' and 'default'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and deletion logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and removing LLAP queue when Interactive query is turned on and off respectively.

2016-04-14 Thread Swapan Shridhar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/
---

(Updated April 14, 2016, 9:01 a.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit Mohanty.


Changes
---

- Suggestions incorporated as per Sumit's review.
- Testing : Python UT passes.


Bugs: AMBARI-15876
https://issues.apache.org/jira/browse/AMBARI-15876


Repository: ambari


Description
---

- When Hive Server Interactive is turned ON, Ambari will create 'llap' named 
YARN queue if only 'default' queue exists at root level and is consuming 100% 
capacity.
- When Hive Server Interactive is turned OFF, Ambari will delete 'llap' queue 
only if 2 queues exist at root level namely, 'llap' and 'default'.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 d518c8b 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 ba46f00 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
7407da3 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py c21bad7 

Diff: https://reviews.apache.org/r/46183/diff/


Testing
---

- Python UT passes.
- Manual testing done. Various scenarios tested.
- Test cases added to test the creation and deletion logic.

- Screenshots attached.


File Attachments


LLAP queue created
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
HSI On and llap and default queues shown in dropdown
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
YARn queue manager with llap and default queues 
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png


Thanks,

Swapan Shridhar

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and removing LLAP queue when Interactive query is turned on and off respectively.

2016-04-13 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review128828
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
 (line 21)


Anyway to avoid it? This is not a valid property for capacity-scheduler.



ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
124)


Lets protect the whole logic based on if HIVE server exists and HSI is 
mapped to a host. Its possible to add hive-interactive-site without adding 
service/component.



ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
140)


Similar to above - check that service exists, component exists and then 
configuration exists. Perhaps a helper method that can be used in multiple 
places.



ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
149)


This may change based on what gets decided about RM delete. For now, you 
can keep the code as is.


- Sumit Mohanty


On April 14, 2016, 3:06 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 14, 2016, 3:06 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - When Hive Server Interactive is turned ON, Ambari will create 'llap' named 
> YARN queue if only 'default' queue exists at root level and is consuming 100% 
> capacity.
> - When Hive Server Interactive is turned OFF, Ambari will delete 'llap' queue 
> only if 2 queues exist at root level namely, 'llap' and 'default'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and deletion logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>