[
https://issues.apache.org/jira/browse/AMBARI-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729601#comment-14729601
]
Hudson commented on AMBARI-12983:
---------------------------------
FAILURE: Integrated in Ambari-trunk-Commit #3383 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/3383/])
AMBARI-12983. Optimize aggregator queries by performing GROUP BY on server.
(swagle) (swagle:
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3e0b8f07c388ad2bd67ef23b6dc313f3fef4d117)
*
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
*
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryClientService.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/EmptyCondition.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
*
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
*
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
*
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/SplitByMetricNamesCondition.java
*
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
* ambari-metrics/ambari-metrics-timelineservice/pom.xml
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java
*
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
*
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/Condition.java
> Optimize aggregator queries by performing GROUP BY on server
> ------------------------------------------------------------
>
> Key: AMBARI-12983
> URL: https://issues.apache.org/jira/browse/AMBARI-12983
> Project: Ambari
> Issue Type: Task
> Components: ambari-metrics
> Affects Versions: 2.1.1
> Reporter: Siddharth Wagle
> Assignee: Siddharth Wagle
> Priority: Critical
> Fix For: 2.1.2
>
>
> Query to do calculate host level time aggregates.
> *Sample*
> {code}
> select SUM(METRIC_SUM), SUM(METRIC_COUNT), MAX(METRIC_MAX), MIN(METRIC_MIN),
> METRIC_NAME, HOSTNAME, APP_ID from METRIC_RECORD WHERE METRIC_NAME IN
> ('bytes_in','rpc.rpc.RpcAuthorizationSuccesses') SERVER_TIME > 1440106830000
> AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME, HOSTNAME, APP_ID limit
> 50;
> 0: jdbc:phoenix:localhost:61181:/hbase> select SUM(METRIC_SUM),
> SUM(METRIC_COUNT), MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME, HOSTNAME,
> APP_ID from METRIC_RECORD WHERE METRIC_NAME IN
> ('bytes_in','rpc.rpc.RpcAuthorizationSuccesses') AND SERVER_TIME >
> 1440106830000 AND SERVER_TIME < 1440106950000 GROUP BY METRIC_NAME, HOSTNAME,
> APP_ID limit 50;
> +-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
> | SUM(METRIC_SUM) | SUM(METRIC_COUNT) | MAX(METRIC_MAX) | MIN(METRIC_MIN) |
> METRIC_NAME | HOSTNAME | APP_ID |
> +-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
> | 1546242.5521201263 | 24 | 564689.2744108599 |
> 616.7147803365921 | bytes_in | ams-test-1.c.pramod-thangali.internal |
> HOST |
> | 1521023.3303401857 | 24 | 825825.3247578086 |
> 220.66636815664341 | bytes_in | ams-test-2.c.pramod-thangali.internal |
> HOST |
> | 153685.92452883872 | 24 | 25389.11630078307 |
> 448.15683323834327 | bytes_in | ams-test-3.c.pramod-thangali.internal |
> HOST |
> | 0.0 | 14 | 0.0 | 0.0 |
> rpc.rpc.RpcAuthorizationSuccesses | ams-test-1.c.pramod-thangali.internal |
> datanode |
> | 17.0 | 14 | 2.0 | 0.0 |
> rpc.rpc.RpcAuthorizationSuccesses | ams-test-1.c.pramod-thangali.internal |
> namenode |
> | 0.0 | 14 | 0.0 | 0.0 |
> rpc.rpc.RpcAuthorizationSuccesses | ams-test-2.c.pramod-thangali.internal |
> datanode |
> +-----------------+-------------------+-----------------+-----------------+-------------+------------+------------+
> {code}
> *Query plan*:
> {code}
> 0: jdbc:phoenix:localhost:61181:/hbase> explain select SUM(METRIC_SUM),
> SUM(METRIC_COUNT), MAX(METRIC_MAX), MIN(METRIC_MIN), METRIC_NAME, HOSTNAME,
> APP_ID from METRIC_RECORD WHERE SERVER_TIME > 1440106830000 AND SERVER_TIME <
> 1440106950000 GROUP BY METRIC_NAME, HOSTNAME, APP_ID;
> +------------+
> | PLAN |
> +------------+
> | CLIENT PARALLEL 1-WAY FULL SCAN OVER METRIC_RECORD |
> | SERVER FILTER BY (SERVER_TIME > 1440106830000 AND SERVER_TIME <
> 1440106950000) |
> | SERVER AGGREGATE INTO DISTINCT ROWS BY [METRIC_NAME, HOSTNAME, APP_ID] |
> | CLIENT MERGE SORT |
> +------------+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
