-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/
-----------------------------------------------------------
Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-16887
https://issues.apache.org/jira/browse/AMBARI-16887
Repository: ambari
Description
-------
PROBLEM
For counter based metrics, the timestamp tracking is lost and end up aggregated
into 30sec data for a 2-minute interval to have the same value. Then when we
want to see this value as a rate metric, during these multi-minute intervals,
the rate is reported as 0, since it did not change. However, looking at the raw
data, it is always increasing within 5-10 second intervals. So it is not the
raw data that is not changing, it is our tracking of values that is wrong.
BUG
Within the 2 min window, we do only interpolation (Required timestamp value is
within range of the set of values used for interpolation) and not extrapolation
(Required timestamp outside range of the set of values used for interpolation).
FIX
Use all values within the 2 minute window for constructing the linear function
and then use that for interpolation / extrapolation. In the case that only 1
value is present in the 2 min window, we use that value for interpolating all
required 30sec timestamps in the window.
Before fix
(),2,3,(),5,() became -> (2),2,3,(4),5,(5)
After fix,
(),2,3,(),5,() became -> (1),2,3,(4),5,(6)
where (x) -> an interpolated value
Diffs
-----
ambari-metrics/ambari-metrics-common/pom.xml e043404
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
ab890ec
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
1ec71d0
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
a8d3086
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
d2d478c
Diff: https://reviews.apache.org/r/47856/diff/
Testing
-------
Manually tested on cluster.
Added unit tests.
Thanks,
Aravindan Vijayan
