> On 6 10, 2016, 11:56 오후, Aravindan Vijayan wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java, > > line 73 > > <https://reviews.apache.org/r/48395/diff/1/?file=1410881#file1410881line73> > > > > When the data is not timestamp aligned, this will lead to "difficult to > > comprehend" graphs. > > > > For example, let's take Sum of Metrics M1,M2,M3 where > > M1 - t1 - 5.0 > > M1 - t2 - 5.0 > > M2 - t2 - 5.0 > > M3 - t3 - 5.0 > > > > The graph will show up like > > (t1, 5.0) -> (t2,10.0) -> (t3,5.0) > > > > That is why we do some "interpolation" while we aggregate data. > > Jungtaek Lim wrote: > Yeah, I agree that output could be odd when data is not timestamp > aligned. > I didn't think about this side while working, and also I'm not expert on > this. > How AMS interpolate points when aggregating? Does it apply linear > interpolation?
I've talked with Aravindan regarding this. There's no issue unless hostname is not empty & timeRange <= 2hrs since other cases are referencing aggregated tables which are already timestamp aligned and interpolated. For that case it refers raw record table so need to apply interpolation. Need to discuss and resolve this case before or later this patch. - Jungtaek ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48395/#review137086 ----------------------------------------------------------- On 6 10, 2016, 12:50 오전, Jungtaek Lim wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48395/ > ----------------------------------------------------------- > > (Updated 6 10, 2016, 12:50 오전) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, > Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-17027 > https://issues.apache.org/jira/browse/AMBARI-17027 > > > Repository: ambari > > > Description > ------- > > AMS doesn't provide tag so metric is identified by appId, metric name, > hostname, instanceId. In this situation metric name is normally consist of > origin metric name and tag values, like graphite, but unlike Graphite, AMS > also doesn't provide series aggregation functions so aggregation should be > done from caller side. > > It would be great if Ambari Metrics Collector provides series aggregation > functions, like sumSeries / > averageSeries / minSeries / maxSeries on Graphite. > > Query outputs: > https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f > > Attached Grafana screenshots to AMBARI-17027. Please refer > https://issues.apache.org/jira/browse/AMBARI-17027 for details. > > > Diffs > ----- > > ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 > > ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html > b034c03 > ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 1b2d02f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > e37bc4d > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 7d49070 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionFactory.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAvgAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMaxAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMinAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesSumAggregateFunction.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > ee3a097 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > cfd1f58 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java > a94f4c5 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/48395/diff/ > > > Testing > ------- > > > mvn clean install > > > cd ambari-metrics > > > mvn test > > ``` > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] ambari-metrics ..................................... SUCCESS [ 1.064 > s] > [INFO] Ambari Metrics Common .............................. SUCCESS [ 13.266 > s] > [INFO] Ambari Metrics Hadoop Sink ......................... SUCCESS [ 4.746 > s] > [INFO] Ambari Metrics Flume Sink .......................... SUCCESS [ 6.594 > s] > [INFO] Ambari Metrics Kafka Sink .......................... SUCCESS [ 4.064 > s] > [INFO] Ambari Metrics Storm Sink .......................... SUCCESS [ 1.289 > s] > [INFO] Ambari Metrics Collector ........................... SUCCESS [04:12 > min] > [INFO] Ambari Metrics Monitor ............................. SUCCESS [ 3.587 > s] > [INFO] Ambari Metrics Grafana ............................. FAILURE [01:15 > min] > [INFO] Ambari Metrics Assembly ............................ SKIPPED > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 06:02 min > [INFO] Finished at: 2016-06-08T14:48:58+09:00 > [INFO] Final Memory: 79M/1149M > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project > ambari-metrics-grafana: An Ant BuildException has occured: > javax.net.ssl.SSLHandshakeException: Remote host closed connection during > handshake > [ERROR] around Ant part ...<get usetimestamp="true" > src="https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz" > > dest="/Users/jlim/WorkArea/JavaProjects/ambari/ambari-metrics/ambari-metrics-grafana/target/grafana/grafana.tgz"/>... > @ 5:222 in > /Users/jlim/WorkArea/JavaProjects/ambari/ambari-metrics/ambari-metrics-grafana/target/antrun/build-Download > Ambari Grafana.xml: SSL peer shut down incorrectly > ``` > > > mvn test -rf :ambari-metrics-grafana > > ``` > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Metrics Grafana ............................. SUCCESS [ 38.966 > s] > [INFO] Ambari Metrics Assembly ............................ SUCCESS [ 3.855 > s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 43.037 s > [INFO] Finished at: 2016-06-08T14:50:33+09:00 > [INFO] Final Memory: 24M/300M > [INFO] > ------------------------------------------------------------------------ > ``` > > > Thanks, > > Jungtaek Lim > >