Re: Review Request 50752: Tune HDFS opts parameters to trigger GC more predicitably

2016-08-03 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50752/#review144615
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Aug. 3, 2016, 5:34 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50752/
> ---
> 
> (Updated Aug. 3, 2016, 5:34 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18006
> https://issues.apache.org/jira/browse/AMBARI-18006
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> DN Heap usage alert is set to 80% warning and 90% critical. This alert is
> causing a lot of alerts to be fired off and needs to be tuned.
> 
> One thing we need to make sure is that DN gc is happening. I am not sure at
> what % of heap usage will the gc be fired but we need to make sure our alerts
> are for a % higher than that as when i manually ran a GC the heap usage went
> below the threshold.
> 
> Based on discussions with hdfs dev's it was determined we should be adding the
> following
> 
> 
> 
> 
> XX:+UseCMSInitiatingOccupancyOnly and 
> -XX:CMSInitiatingOccupancyFraction=
> 
> 
> to the namenode and datanode opts. Based on this setting hopefully we can also
> determine what % to set the alerts to.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml
>  40483b8 
> 
> Diff: https://reviews.apache.org/r/50752/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>



Review Request 50752: Tune HDFS opts parameters to trigger GC more predicitably

2016-08-03 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50752/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18006
https://issues.apache.org/jira/browse/AMBARI-18006


Repository: ambari


Description
---

DN Heap usage alert is set to 80% warning and 90% critical. This alert is
causing a lot of alerts to be fired off and needs to be tuned.

One thing we need to make sure is that DN gc is happening. I am not sure at
what % of heap usage will the gc be fired but we need to make sure our alerts
are for a % higher than that as when i manually ran a GC the heap usage went
below the threshold.

Based on discussions with hdfs dev's it was determined we should be adding the
following




XX:+UseCMSInitiatingOccupancyOnly and 
-XX:CMSInitiatingOccupancyFraction=


to the namenode and datanode opts. Based on this setting hopefully we can also
determine what % to set the alerts to.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml
 40483b8 

Diff: https://reviews.apache.org/r/50752/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk