> On May 26, 2016, 3:10 p.m., Dmytro Sen wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java,
> >  line 279
> > <https://reviews.apache.org/r/47856/diff/1/?file=1394002#file1394002line279>
> >
> >     I didn't get why there is a special handling for 
> > regionserver.Server.Get_num_ops ?

Thanks DS. I added that INFO log line for a specific metric that I was trying 
to track and test interpolation. I will remove it.


- Aravindan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/#review134988
-----------------------------------------------------------


On May 25, 2016, 10:04 p.m., Aravindan Vijayan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47856/
> -----------------------------------------------------------
> 
> (Updated May 25, 2016, 10:04 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16887
>     https://issues.apache.org/jira/browse/AMBARI-16887
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM
> 
> For counter based metrics, the timestamp tracking is lost and end up 
> aggregated into 30sec data for a 2-minute interval to have the same value. 
> Then when we want to see this value as a rate metric, during these 
> multi-minute intervals, the rate is reported as 0, since it did not change. 
> However, looking at the raw data, it is always increasing within 5-10 second 
> intervals. So it is not the raw data that is not changing, it is our tracking 
> of values that is wrong.
> 
> 
> BUG
> Within the 2 min window, we do only interpolation (Required timestamp value 
> is within range of the set of values used for interpolation) and not 
> extrapolation (Required timestamp outside range of the set of values used for 
> interpolation).
> 
> FIX
> 
> Use all values within the 2 minute window for constructing the linear 
> function and then use that for interpolation / extrapolation. In the case 
> that only 1 value is present in the 2 min window, we use that value for 
> interpolating all required 30sec timestamps in the window.
> 
> Before fix
> (),2,3,(),5,() became -> (2),2,3,(4),5,(5)
> 
> After fix,
> (),2,3,(),5,() became -> (1),2,3,(4),5,(6)
> 
> where (x) -> an interpolated value
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-common/pom.xml e043404 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
>  ab890ec 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
>  1ec71d0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
>  a8d3086 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
>  d2d478c 
> 
> Diff: https://reviews.apache.org/r/47856/diff/
> 
> 
> Testing
> -------
> 
> Manually tested on cluster.
> 
> Added unit tests.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Reply via email to