> On 6 10, 2016, 11:56 오후, Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java,
> >  line 257
> > <https://reviews.apache.org/r/48395/diff/1/?file=1410878#file1410878line257>
> >
> >     Just throwing out a question here. 
> >     
> >     Does it make any difference to do SUM(M1,M2) before or after applying 
> > post processing (rate/diff)? 
> >     
> >     Ideally, these operations should be commutative (Order should not 
> > matter). 
> >     
> >     Rate of Sum and Sum or Rate should be the same.

If it is timestamp aligned, it is commutative for SUM.

Let's say value of (M1, t1) to x1 and value of (M2, t1) to y1.

t1  x1  y1
t2  x2  y2
t3  x3  y3
t4  x4  y4

If we calculate sum first, and rate, 
t1 = 0.0
t2 = ((x2 + y2) - (x1 + y1)) / (t2 - t1) = (x2 + y2 - x1 - y1) / (t2 - t1)
t3 = ((x3 + y3) - (x2 + y2)) / (t3 - t2) = (x3 + y3 - x2 - y2) / (t3 - t2)
t4 = ((x4 + y4) - (x3 + y3)) / (t4 - t3) = (x4 + y4 - x3 - y3) / (t4 - t3)

If we calculate rate first, and sum, 
t1 = 0.0
t2 = ((x2 - x1) + (y2 - y1)) / (t2 - t1) = (x2 + y2 - x1 - y1) / (t2 - t1)
t3 = ((x3 - x2) + (y3 - y2)) / (t3 - t2) = (x3 + y3 - x2 - y2) / (t3 - t2)
t4 = ((x4 - x3) + (y4 - y3)) / (t4 - t3) = (x4 + y4 - x3 - y3) / (t4 - t3)

So they're same.

But if it is not timestamp aligned, it will be messed up. For example, 

t1  x1  y1
t2      y2
t3  x3 
t4  x4  y4

If we calculate sum first, and rate, 
t1 = 0.0
t2 = (y2 - (x1 + y1)) / (t2 - t1) = (y2 - x1 - y1) / (t2 - t1)
t3 = (x3 - y2) / (t3 - t2) = (x3 - y2) / (t3 - t2)
t4 = (x4 + y4 - x3) / (t4 - t3)

If we calculate rate first, and sum, 
t1 = 0.0
t2 = N/A + (y2 - y1) / (t2 - t1) = (y2 - y1) / (t2 - t1)
t3 = (x3 - x1) / (t3 - t1) + N/A = (x3 - x1) / (t3 - t1)
t4 = ((x4 - x3) / (t4 - t3)) + ((y4 - t2) / (t4 - t2))

So they're not same.

I guess 'interpolation' seems important for this side.


- Jungtaek


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48395/#review137086
-----------------------------------------------------------


On 6 10, 2016, 12:50 오전, Jungtaek Lim wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48395/
> -----------------------------------------------------------
> 
> (Updated 6 10, 2016, 12:50 오전)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, 
> Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17027
>     https://issues.apache.org/jira/browse/AMBARI-17027
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> AMS doesn't provide tag so metric is identified by appId, metric name, 
> hostname, instanceId. In this situation metric name is normally consist of 
> origin metric name and tag values, like graphite, but unlike Graphite, AMS 
> also doesn't provide series aggregation functions so aggregation should be 
> done from caller side.
> 
> It would be great if Ambari Metrics Collector provides series aggregation 
> functions, like sumSeries / 
> averageSeries / minSeries / maxSeries on Graphite.
> 
> Query outputs: 
> https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f
> 
> Attached Grafana screenshots to AMBARI-17027. Please refer 
> https://issues.apache.org/jira/browse/AMBARI-17027 for details.
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 
>   
> ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html
>  b034c03 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  1b2d02f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
>  e37bc4d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
>  7d49070 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionFactory.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAvgAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMaxAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMinAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesSumAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java
>  ee3a097 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java
>  cfd1f58 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java
>  a94f4c5 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48395/diff/
> 
> 
> Testing
> -------
> 
> > mvn clean install
> 
> > cd ambari-metrics
> 
> > mvn test
> 
> ```
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] ambari-metrics ..................................... SUCCESS [  1.064 
> s]
> [INFO] Ambari Metrics Common .............................. SUCCESS [ 13.266 
> s]
> [INFO] Ambari Metrics Hadoop Sink ......................... SUCCESS [  4.746 
> s]
> [INFO] Ambari Metrics Flume Sink .......................... SUCCESS [  6.594 
> s]
> [INFO] Ambari Metrics Kafka Sink .......................... SUCCESS [  4.064 
> s]
> [INFO] Ambari Metrics Storm Sink .......................... SUCCESS [  1.289 
> s]
> [INFO] Ambari Metrics Collector ........................... SUCCESS [04:12 
> min]
> [INFO] Ambari Metrics Monitor ............................. SUCCESS [  3.587 
> s]
> [INFO] Ambari Metrics Grafana ............................. FAILURE [01:15 
> min]
> [INFO] Ambari Metrics Assembly ............................ SKIPPED
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 06:02 min
> [INFO] Finished at: 2016-06-08T14:48:58+09:00
> [INFO] Final Memory: 79M/1149M
> [INFO] 
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project 
> ambari-metrics-grafana: An Ant BuildException has occured: 
> javax.net.ssl.SSLHandshakeException: Remote host closed connection during 
> handshake
> [ERROR] around Ant part ...<get usetimestamp="true" 
> src="https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz";
>  
> dest="/Users/jlim/WorkArea/JavaProjects/ambari/ambari-metrics/ambari-metrics-grafana/target/grafana/grafana.tgz"/>...
>  @ 5:222 in 
> /Users/jlim/WorkArea/JavaProjects/ambari/ambari-metrics/ambari-metrics-grafana/target/antrun/build-Download
>  Ambari Grafana.xml: SSL peer shut down incorrectly
> ```
> 
> > mvn test -rf :ambari-metrics-grafana
> 
> ```
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Metrics Grafana ............................. SUCCESS [ 38.966 
> s]
> [INFO] Ambari Metrics Assembly ............................ SUCCESS [  3.855 
> s]
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] 
> ------------------------------------------------------------------------
> [INFO] Total time: 43.037 s
> [INFO] Finished at: 2016-06-08T14:50:33+09:00
> [INFO] Final Memory: 24M/300M
> [INFO] 
> ------------------------------------------------------------------------
> ```
> 
> 
> Thanks,
> 
> Jungtaek Lim
> 
>

Reply via email to