-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/
-----------------------------------------------------------

(Updated May 26, 2016, 7:16 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-16887
    https://issues.apache.org/jira/browse/AMBARI-16887


Repository: ambari


Description
-------

PROBLEM

For counter based metrics, the timestamp tracking is lost and end up aggregated 
into 30sec data for a 2-minute interval to have the same value. Then when we 
want to see this value as a rate metric, during these multi-minute intervals, 
the rate is reported as 0, since it did not change. However, looking at the raw 
data, it is always increasing within 5-10 second intervals. So it is not the 
raw data that is not changing, it is our tracking of values that is wrong.


BUG
Within the 2 min window, we do only interpolation (Required timestamp value is 
within range of the set of values used for interpolation) and not extrapolation 
(Required timestamp outside range of the set of values used for interpolation).

FIX

Use all values within the 2 minute window for constructing the linear function 
and then use that for interpolation / extrapolation. In the case that only 1 
value is present in the 2 min window, we use that value for interpolating all 
required 30sec timestamps in the window.

Before fix
(),2,3,(),5,() became -> (2),2,3,(4),5,(5)

After fix,
(),2,3,(),5,() became -> (1),2,3,(4),5,(6)

where (x) -> an interpolated value


Diffs (updated)
-----

  ambari-metrics/ambari-metrics-common/pom.xml e043404 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
 ab890ec 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
 1ec71d0 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 a8d3086 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
 d2d478c 

Diff: https://reviews.apache.org/r/47856/diff/


Testing
-------

Manually tested on cluster.

Added unit tests.


Thanks,

Aravindan Vijayan

Reply via email to