> On Oct. 20, 2016, 2:01 a.m., Sid Wagle wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java, > > line 96 > > <https://reviews.apache.org/r/52842/diff/2/?file=1540789#file1540789line96> > > > > The only problem I see here is we loose context of what topN funtion > > was applied if we copy same value everywhere. What about only for the > > column = function and rest as NULL?
I am ok with setting "NULL" values for columns that are not computed through the downsampling. But, I feel that the downsampled metric value should be set in the AVG column, which is returned by default when the metric is requested. Even though we do a Top N by Max value for the downsampling function, the computed value will be in the default value column, and not the MAX column. The user's query will be like "Get me the Top 3 op-user combination for every 5 min slice the last 6 hours". The query need not explicitly request "MAX" of the metric, since that is basically what the topn downsampling configs dictated. - Aravindan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52842/#review153330 ----------------------------------------------------------- On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52842/ > ----------------------------------------------------------- > > (Updated Oct. 18, 2016, 6:35 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, > and Sid Wagle. > > > Bugs: AMBARI-18593 > https://issues.apache.org/jira/browse/AMBARI-18593 > > > Repository: ambari > > > Description > ------- > > HDFS exposes top user activity broken down by operations in jmx (nntop). > These metrics should be captured in AMS and exposed in Grafana's HDFS > dashboards. > > Downsampling should likely be a function like MIN, MAX, AVG, SUM of > underlying timeseries specified from the client. > > > Diffs > ----- > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java > ba7807b > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java > c056d79 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java > 118c695 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 177e444 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java > f7060e0 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java > PRE-CREATION > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > b6e5da9 > > Diff: https://reviews.apache.org/r/52842/diff/ > > > Testing > ------- > > Manually tested. > Unit tests added. > mvn clean test on ambari-metrics pending. > > > Thanks, > > Aravindan Vijayan > >
