Re: Review Request 47856: AMBARI-16887 : [AMS / Grafana] Metrics are staying flat for 1 minutes, causing rate calculations to be 0

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/#review135195
---


Ship it!




Ship It!

- Dmytro Sen


On Май 26, 2016, 7:16 п.п., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47856/
> ---
> 
> (Updated Май 26, 2016, 7:16 п.п.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16887
> https://issues.apache.org/jira/browse/AMBARI-16887
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> 
> For counter based metrics, the timestamp tracking is lost and end up 
> aggregated into 30sec data for a 2-minute interval to have the same value. 
> Then when we want to see this value as a rate metric, during these 
> multi-minute intervals, the rate is reported as 0, since it did not change. 
> However, looking at the raw data, it is always increasing within 5-10 second 
> intervals. So it is not the raw data that is not changing, it is our tracking 
> of values that is wrong.
> 
> 
> BUG
> Within the 2 min window, we do only interpolation (Required timestamp value 
> is within range of the set of values used for interpolation) and not 
> extrapolation (Required timestamp outside range of the set of values used for 
> interpolation).
> 
> FIX
> 
> Use all values within the 2 minute window for constructing the linear 
> function and then use that for interpolation / extrapolation. In the case 
> that only 1 value is present in the 2 min window, we use that value for 
> interpolating all required 30sec timestamps in the window.
> 
> Before fix
> (),2,3,(),5,() became -> (2),2,3,(4),5,(5)
> 
> After fix,
> (),2,3,(),5,() became -> (1),2,3,(4),5,(6)
> 
> where (x) -> an interpolated value
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml e043404 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
>  ab890ec 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
>  1ec71d0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
>  a8d3086 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
>  d2d478c 
> 
> Diff: https://reviews.apache.org/r/47856/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on cluster.
> 
> Added unit tests.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 47856: AMBARI-16887 : [AMS / Grafana] Metrics are staying flat for 1 minutes, causing rate calculations to be 0

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/
---

(Updated May 26, 2016, 7:16 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-16887
https://issues.apache.org/jira/browse/AMBARI-16887


Repository: ambari


Description
---

PROBLEM

For counter based metrics, the timestamp tracking is lost and end up aggregated 
into 30sec data for a 2-minute interval to have the same value. Then when we 
want to see this value as a rate metric, during these multi-minute intervals, 
the rate is reported as 0, since it did not change. However, looking at the raw 
data, it is always increasing within 5-10 second intervals. So it is not the 
raw data that is not changing, it is our tracking of values that is wrong.


BUG
Within the 2 min window, we do only interpolation (Required timestamp value is 
within range of the set of values used for interpolation) and not extrapolation 
(Required timestamp outside range of the set of values used for interpolation).

FIX

Use all values within the 2 minute window for constructing the linear function 
and then use that for interpolation / extrapolation. In the case that only 1 
value is present in the 2 min window, we use that value for interpolating all 
required 30sec timestamps in the window.

Before fix
(),2,3,(),5,() became -> (2),2,3,(4),5,(5)

After fix,
(),2,3,(),5,() became -> (1),2,3,(4),5,(6)

where (x) -> an interpolated value


Diffs (updated)
-

  ambari-metrics/ambari-metrics-common/pom.xml e043404 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
 ab890ec 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
 1ec71d0 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 a8d3086 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
 d2d478c 

Diff: https://reviews.apache.org/r/47856/diff/


Testing
---

Manually tested on cluster.

Added unit tests.


Thanks,

Aravindan Vijayan

Re: Review Request 47856: AMBARI-16887 : [AMS / Grafana] Metrics are staying flat for 1 minutes, causing rate calculations to be 0

[Aravindan Vijayan]

> On May 26, 2016, 3:10 p.m., Dmytro Sen wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java,
> >  line 279
> > 
> >
> > I didn't get why there is a special handling for 
> > regionserver.Server.Get_num_ops ?

Thanks DS. I added that INFO log line for a specific metric that I was trying 
to track and test interpolation. I will remove it.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/#review134988
---


On May 25, 2016, 10:04 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47856/
> ---
> 
> (Updated May 25, 2016, 10:04 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16887
> https://issues.apache.org/jira/browse/AMBARI-16887
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> 
> For counter based metrics, the timestamp tracking is lost and end up 
> aggregated into 30sec data for a 2-minute interval to have the same value. 
> Then when we want to see this value as a rate metric, during these 
> multi-minute intervals, the rate is reported as 0, since it did not change. 
> However, looking at the raw data, it is always increasing within 5-10 second 
> intervals. So it is not the raw data that is not changing, it is our tracking 
> of values that is wrong.
> 
> 
> BUG
> Within the 2 min window, we do only interpolation (Required timestamp value 
> is within range of the set of values used for interpolation) and not 
> extrapolation (Required timestamp outside range of the set of values used for 
> interpolation).
> 
> FIX
> 
> Use all values within the 2 minute window for constructing the linear 
> function and then use that for interpolation / extrapolation. In the case 
> that only 1 value is present in the 2 min window, we use that value for 
> interpolating all required 30sec timestamps in the window.
> 
> Before fix
> (),2,3,(),5,() became -> (2),2,3,(4),5,(5)
> 
> After fix,
> (),2,3,(),5,() became -> (1),2,3,(4),5,(6)
> 
> where (x) -> an interpolated value
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml e043404 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
>  ab890ec 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
>  1ec71d0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
>  a8d3086 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
>  d2d478c 
> 
> Diff: https://reviews.apache.org/r/47856/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on cluster.
> 
> Added unit tests.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 47856: AMBARI-16887 : [AMS / Grafana] Metrics are staying flat for 1 minutes, causing rate calculations to be 0

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/#review134988
---


Fix it, then Ship it!





ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 (line 278)


I didn't get why there is a special handling for 
regionserver.Server.Get_num_ops ?


- Dmytro Sen


On Май 25, 2016, 10:04 п.п., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47856/
> ---
> 
> (Updated Май 25, 2016, 10:04 п.п.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16887
> https://issues.apache.org/jira/browse/AMBARI-16887
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> 
> For counter based metrics, the timestamp tracking is lost and end up 
> aggregated into 30sec data for a 2-minute interval to have the same value. 
> Then when we want to see this value as a rate metric, during these 
> multi-minute intervals, the rate is reported as 0, since it did not change. 
> However, looking at the raw data, it is always increasing within 5-10 second 
> intervals. So it is not the raw data that is not changing, it is our tracking 
> of values that is wrong.
> 
> 
> BUG
> Within the 2 min window, we do only interpolation (Required timestamp value 
> is within range of the set of values used for interpolation) and not 
> extrapolation (Required timestamp outside range of the set of values used for 
> interpolation).
> 
> FIX
> 
> Use all values within the 2 minute window for constructing the linear 
> function and then use that for interpolation / extrapolation. In the case 
> that only 1 value is present in the 2 min window, we use that value for 
> interpolating all required 30sec timestamps in the window.
> 
> Before fix
> (),2,3,(),5,() became -> (2),2,3,(4),5,(5)
> 
> After fix,
> (),2,3,(),5,() became -> (1),2,3,(4),5,(6)
> 
> where (x) -> an interpolated value
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml e043404 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
>  ab890ec 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
>  1ec71d0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
>  a8d3086 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
>  d2d478c 
> 
> Diff: https://reviews.apache.org/r/47856/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on cluster.
> 
> Added unit tests.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Review Request 47856: AMBARI-16887 : [AMS / Grafana] Metrics are staying flat for 1 minutes, causing rate calculations to be 0

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47856/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-16887
https://issues.apache.org/jira/browse/AMBARI-16887


Repository: ambari


Description
---

PROBLEM

For counter based metrics, the timestamp tracking is lost and end up aggregated 
into 30sec data for a 2-minute interval to have the same value. Then when we 
want to see this value as a rate metric, during these multi-minute intervals, 
the rate is reported as 0, since it did not change. However, looking at the raw 
data, it is always increasing within 5-10 second intervals. So it is not the 
raw data that is not changing, it is our tracking of values that is wrong.


BUG
Within the 2 min window, we do only interpolation (Required timestamp value is 
within range of the set of values used for interpolation) and not extrapolation 
(Required timestamp outside range of the set of values used for interpolation).

FIX

Use all values within the 2 minute window for constructing the linear function 
and then use that for interpolation / extrapolation. In the case that only 1 
value is present in the 2 min window, we use that value for interpolating all 
required 30sec timestamps in the window.

Before fix
(),2,3,(),5,() became -> (2),2,3,(4),5,(5)

After fix,
(),2,3,(),5,() became -> (1),2,3,(4),5,(6)

where (x) -> an interpolated value


Diffs
-

  ambari-metrics/ambari-metrics-common/pom.xml e043404 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java
 ab890ec 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java
 1ec71d0 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
 a8d3086 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java
 d2d478c 

Diff: https://reviews.apache.org/r/47856/diff/


Testing
---

Manually tested on cluster.

Added unit tests.


Thanks,

Aravindan Vijayan