----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50966/#review145609 -----------------------------------------------------------
Ship it! Ship It! - Dmytro Sen On Авг. 11, 2016, 10:55 п.п., Aravindan Vijayan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50966/ > ----------------------------------------------------------- > > (Updated Авг. 11, 2016, 10:55 п.п.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-18125 > https://issues.apache.org/jira/browse/AMBARI-18125 > > > Repository: ambari > > > Description > ------- > > Any hadoop metric that need not be aggregated across hosts can specify a > skipAggregation = true flag in its Tags (Metadata). This metadata is captured > by the AMS metadata and used to make a decision whether to aggregate a metric > across hosts or not. > > Went with a "skip aggregation of metric" technique rather than a "don't even > read metrics for aggregation that have skipAggregation = true" since the > number of metrics can become quite large, and a METRIC_NAME NOT IN clause > might become an issue. > > As an example, used this 'skipAggregation' flag in the AMS controlled "per > disk" metrics like disk_1_read_bytes, disk_2_read_count etc. > > List of issues fixed > > 1. Skip Aggregation of certain metrics specified through metadata or configs. > 2. Fix issue in METRIC_NAME where clause in DefaultCondition that causes > mismatched metricNames to positions during PreparedStatement generation. > > Sample 2min aggregator query before patch. > SELECT /+ NATIVE_TIME_RANGE(1470942120000) / METRIC_NAME, HOSTNAME, APP_ID, > INSTANCE_ID, SERVER_TIME, START_TIME, UNITS, METRIC_SUM, METRIC_MAX, > METRIC_MIN, METRIC_COUNT, > METRICS FROM METRIC_RECORD WHERE (METRIC_NAME NOT IN ('?') AND METRIC_NAME > NOT LIKE '?' AND METRIC_NAME NOT LIKE '?') AND SERVER_TIME >= 1470942150000 > AND SERVER_TIME < 1470942360000 ORDER BY METRIC_NAME, HOSTNAME, APP_ID, > SERVER_TIME > > 0 = {String@6887} "disk_ _%" > 1 = {String@6888} "disk_ _ _%" > 2 = {String@6889} "boottime" > 3 = {Long@6890} "1470942150000" > 4 = {Long@6891} "1470942360000" > > Note that IN Clause will be given the value "disk_ _%" and LIKE clause will > be given values - "disk_ _ _%" and "boottime" incorrectly. > > 3. Specify actual disk names in the metric name, instead of a number for > individual disk metrics. > 4. Fix issue in AMS embedded mode startup. > > > Diffs > ----- > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java > 98f4978 > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > c534121 > > ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java > ea7f72d > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py > 845b270 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 0f98db2 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 1da68ba > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > 6731eb3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java > fd471fb > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java > 4873c24 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 0851e8f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java > 34174e2 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java > b8ca599 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java > a95655d > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java > 0f93bab > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultConditionTest.java > PRE-CREATION > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > 871e571 > > Diff: https://reviews.apache.org/r/50966/diff/ > > > Testing > ------- > > Manually tested. > > > Thanks, > > Aravindan Vijayan > >
