----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47856/#review134988 -----------------------------------------------------------
Fix it, then Ship it! ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 278) <https://reviews.apache.org/r/47856/#comment199978> I didn't get why there is a special handling for regionserver.Server.Get_num_ops ? - Dmytro Sen On Май 25, 2016, 10:04 п.п., Aravindan Vijayan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47856/ > ----------------------------------------------------------- > > (Updated Май 25, 2016, 10:04 п.п.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16887 > https://issues.apache.org/jira/browse/AMBARI-16887 > > > Repository: ambari > > > Description > ------- > > PROBLEM > > For counter based metrics, the timestamp tracking is lost and end up > aggregated into 30sec data for a 2-minute interval to have the same value. > Then when we want to see this value as a rate metric, during these > multi-minute intervals, the rate is reported as 0, since it did not change. > However, looking at the raw data, it is always increasing within 5-10 second > intervals. So it is not the raw data that is not changing, it is our tracking > of values that is wrong. > > > BUG > Within the 2 min window, we do only interpolation (Required timestamp value > is within range of the set of values used for interpolation) and not > extrapolation (Required timestamp outside range of the set of values used for > interpolation). > > FIX > > Use all values within the 2 minute window for constructing the linear > function and then use that for interpolation / extrapolation. In the case > that only 1 value is present in the 2 min window, we use that value for > interpolating all required 30sec timestamps in the window. > > Before fix > (),2,3,(),5,() became -> (2),2,3,(4),5,(5) > > After fix, > (),2,3,(),5,() became -> (1),2,3,(4),5,(6) > > where (x) -> an interpolated value > > > Diffs > ----- > > ambari-metrics/ambari-metrics-common/pom.xml e043404 > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java > ab890ec > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java > 1ec71d0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > a8d3086 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java > d2d478c > > Diff: https://reviews.apache.org/r/47856/diff/ > > > Testing > ------- > > Manually tested on cluster. > > Added unit tests. > > > Thanks, > > Aravindan Vijayan > >
