Re: Review Request 45447: AMBARI-15621 : Cluster Second aggregator taking more than 2 mins to execute on large clusters, thereby causing lag.

2016-03-30 Thread Aravindan Vijayan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45447/
---

(Updated March 30, 2016, 5:51 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Changes
---

Addressed review comments.


Bugs: AMBARI-15621
https://issues.apache.org/jira/browse/AMBARI-15621


Repository: ambari


Description
---

Fix issues in aggregator scheduling implementation. 

Recommend value of "timeline.metrics.service.resultset.fetchSize" through stack 
advisor based on cluster size.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 2f080e3 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 f8ec516 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregatorTest.java
 21b9839 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
b8e2c45 

Diff: https://reviews.apache.org/r/45447/diff/


Testing
---

Manually tested on a large cluster.

Unit tests pass.


Thanks,

Aravindan Vijayan

Re: Review Request 45447: AMBARI-15621 : Cluster Second aggregator taking more than 2 mins to execute on large clusters, thereby causing lag.

2016-03-30 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45447/#review126140
---


Fix it, then Ship it!





ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 (line 105)


Should be consistent time use for cal and logged.


- Sid Wagle


On March 29, 2016, 9:15 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45447/
> ---
> 
> (Updated March 29, 2016, 9:15 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15621
> https://issues.apache.org/jira/browse/AMBARI-15621
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix issues in aggregator scheduling implementation. 
> 
> Recommend value of "timeline.metrics.service.resultset.fetchSize" through 
> stack advisor based on cluster size.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  2f080e3 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  f8ec516 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregatorTest.java
>  21b9839 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> b8e2c45 
> 
> Diff: https://reviews.apache.org/r/45447/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on a large cluster.
> 
> Unit tests pass.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Review Request 45447: AMBARI-15621 : Cluster Second aggregator taking more than 2 mins to execute on large clusters, thereby causing lag.

2016-03-29 Thread Aravindan Vijayan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45447/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-15621
https://issues.apache.org/jira/browse/AMBARI-15621


Repository: ambari


Description
---

Fix issues in aggregator scheduling implementation. 

Recommend value of "timeline.metrics.service.resultset.fetchSize" through stack 
advisor based on cluster size.


Diffs
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 2f080e3 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 f8ec516 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregatorTest.java
 21b9839 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
b8e2c45 

Diff: https://reviews.apache.org/r/45447/diff/


Testing
---

Manually tested on a large cluster.

Unit tests pass.


Thanks,

Aravindan Vijayan