Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/
---

(Updated Oct. 3, 2016, 8:16 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Updated Description.


Bugs: AMBARI-18508
https://issues.apache.org/jira/browse/AMBARI-18508


Repository: ambari


Description (updated)
---

**Issue:** LLAP daemons can be killed by the YARN Memory Monitor

The following messages in the AM log of LLAP YARN Application.
{quote}
is running beyond physical memory limits. Current usage:  of  
GB physical memory used.
{quote}

**Fix:** Added a configurable option for 'headroom space' with default value as 
6G. This option gets used while calculating heap size.

- Screenshot attached.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 b14cc8c 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
2a4728a 

Diff: https://reviews.apache.org/r/52451/diff/


Testing
---

- Python UT passes.
- Tested on live cluster.
- Ambari QA pending.


File Attachments


"llap headroom space" on UI.
  
https://reviews.apache.org/media/uploaded/files/2016/10/03/08a70951-a3e9-4327-84a2-579bb40d5d2a__Screen_Shot_2016-10-03_at_12.29.31_PM.png


Thanks,

Swapan Shridhar

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/#review151221
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 3, 2016, 7:32 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52451/
> ---
> 
> (Updated Oct. 3, 2016, 7:32 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18508
> https://issues.apache.org/jira/browse/AMBARI-18508
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:** LLAP daemons can be killed by the YARN Memory Monitor
> 
> The following messages in the AM log of LLAP YARN Application.
> {quote}
> is running beyond physical memory limits. Current usage:  of 
>  GB physical memory used.
> {quote}
> 
> **Fix:** Added a configurable option for 'headroom space' with default value 
> as 6G.
> 
> - Screenshot attached.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  b14cc8c 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 2a4728a 
> 
> Diff: https://reviews.apache.org/r/52451/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Tested on live cluster.
> - Ambari QA pending.
> 
> 
> File Attachments
> 
> 
> "llap headroom space" on UI.
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/03/08a70951-a3e9-4327-84a2-579bb40d5d2a__Screen_Shot_2016-10-03_at_12.29.31_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/
---

(Updated Oct. 3, 2016, 7:32 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Updated description.


Bugs: AMBARI-18508
https://issues.apache.org/jira/browse/AMBARI-18508


Repository: ambari


Description (updated)
---

**Issue:** LLAP daemons can be killed by the YARN Memory Monitor

The following messages in the AM log of LLAP YARN Application.
{quote}
is running beyond physical memory limits. Current usage:  of  
GB physical memory used.
{quote}

**Fix:** Added a configurable option for 'headroom space' with default value as 
6G.

- Screenshot attached.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 b14cc8c 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
2a4728a 

Diff: https://reviews.apache.org/r/52451/diff/


Testing
---

- Python UT passes.
- Tested on live cluster.
- Ambari QA pending.


File Attachments


"llap headroom space" on UI.
  
https://reviews.apache.org/media/uploaded/files/2016/10/03/08a70951-a3e9-4327-84a2-579bb40d5d2a__Screen_Shot_2016-10-03_at_12.29.31_PM.png


Thanks,

Swapan Shridhar

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Swapan Shridhar]

> On Oct. 1, 2016, 3:21 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py, 
> > line 956
> > 
> >
> > Should we instead introduce a variable in hive_interactive_env called - 
> > llap_headroom_space and use that?

Added 'llap headroom space' as configurable option in hive-interactive-env.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/#review151112
---


On Oct. 3, 2016, 7:30 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52451/
> ---
> 
> (Updated Oct. 3, 2016, 7:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18508
> https://issues.apache.org/jira/browse/AMBARI-18508
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:** LLAP daemons can be killed by the YARN Memory Monitor
> 
> The following messages in the AM log of LLAP YARN Application.
> {quote}
> is running beyond physical memory limits. Current usage:  of 
>  GB physical memory used.
> {quote}
> 
> **Fix:** Increase the headroom space for heap from 1G (existing) to 6G.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  b14cc8c 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 2a4728a 
> 
> Diff: https://reviews.apache.org/r/52451/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Tested on live cluster.
> - Ambari QA pending.
> 
> 
> File Attachments
> 
> 
> "llap headroom space" on UI.
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/03/08a70951-a3e9-4327-84a2-579bb40d5d2a__Screen_Shot_2016-10-03_at_12.29.31_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/
---

(Updated Oct. 3, 2016, 7:30 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Added 'llap headroom space' as configurable option in hive-interactive-env.


Bugs: AMBARI-18508
https://issues.apache.org/jira/browse/AMBARI-18508


Repository: ambari


Description
---

**Issue:** LLAP daemons can be killed by the YARN Memory Monitor

The following messages in the AM log of LLAP YARN Application.
{quote}
is running beyond physical memory limits. Current usage:  of  
GB physical memory used.
{quote}

**Fix:** Increase the headroom space for heap from 1G (existing) to 6G.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 b14cc8c 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
2a4728a 

Diff: https://reviews.apache.org/r/52451/diff/


Testing
---

- Python UT passes.
- Tested on live cluster.
- Ambari QA pending.


File Attachments (updated)


"llap headroom space" on UI.
  
https://reviews.apache.org/media/uploaded/files/2016/10/03/08a70951-a3e9-4327-84a2-579bb40d5d2a__Screen_Shot_2016-10-03_at_12.29.31_PM.png


Thanks,

Swapan Shridhar

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/#review151112
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
956)


Should we instead introduce a variable in hive_interactive_env called - 
llap_headroom_space and use that?


- Sumit Mohanty


On Sept. 30, 2016, 9:41 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52451/
> ---
> 
> (Updated Sept. 30, 2016, 9:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18508
> https://issues.apache.org/jira/browse/AMBARI-18508
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:** LLAP daemons can be killed by the YARN Memory Monitor
> 
> The following messages in the AM log of LLAP YARN Application.
> {quote}
> is running beyond physical memory limits. Current usage:  of 
>  GB physical memory used.
> {quote}
> 
> **Fix:** Increase the headroom space for heap from 1G (existing) to 6G.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 2a4728a 
> 
> Diff: https://reviews.apache.org/r/52451/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Tested on live cluster.
> - Ambari QA pending.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/#review151095
---


Ship it!




Ship It!

- Alejandro Fernandez


On Sept. 30, 2016, 9:41 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52451/
> ---
> 
> (Updated Sept. 30, 2016, 9:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18508
> https://issues.apache.org/jira/browse/AMBARI-18508
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:** LLAP daemons can be killed by the YARN Memory Monitor
> 
> The following messages in the AM log of LLAP YARN Application.
> {quote}
> is running beyond physical memory limits. Current usage:  of 
>  GB physical memory used.
> {quote}
> 
> **Fix:** Increase the headroom space for heap from 1G (existing) to 6G.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 2a4728a 
> 
> Diff: https://reviews.apache.org/r/52451/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Tested on live cluster.
> - Ambari QA pending.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Review Request 52451: AMBARI-18508. Increasing headroom on heap size is from 1G to 6G for LLAP.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52451/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-18508
https://issues.apache.org/jira/browse/AMBARI-18508


Repository: ambari


Description
---

**Issue:** LLAP daemons can be killed by the YARN Memory Monitor

The following messages in the AM log of LLAP YARN Application.
{quote}
is running beyond physical memory limits. Current usage:  of  
GB physical memory used.
{quote}

**Fix:** Increase the headroom space for heap from 1G (existing) to 6G.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
2a4728a 

Diff: https://reviews.apache.org/r/52451/diff/


Testing
---

- Python UT passes.
- Tested on live cluster.
- Ambari QA pending.


Thanks,

Swapan Shridhar