[
https://issues.apache.org/jira/browse/AMBARI-17382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421781#comment-16421781
]
Hudson commented on AMBARI-17382:
---------------------------------
FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8958 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/8958/])
AMBARI-17382 : Migrate AMS queries to use ROW_TIMESTAMP instead of (avijayan:
[https://gitbox.apache.org/repos/asf?p=ambari.git&a=commit&h=935f3329574d712dd1d976fb3ad5b154ca82ccb1])
* (edit)
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/SingleValuedTimelineMetric.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITClusterAggregator.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
* (edit)
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/MetricsRequestHelper.java
* (edit)
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetrics.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
* (edit)
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/source/RawMetricsSourceTest.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostAggregator.java
* (edit)
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregator.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITMetricAggregator.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
* (edit)
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/source/cache/InternalMetricsCache.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
* (edit)
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/smoketest_metrics.json.j2
* (edit)
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
* (edit)
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java
* (edit)
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsEhCacheSizeOfEngine.java
> Migrate AMS queries to use ROW_TIMESTAMP instead of native timerange hint
> -------------------------------------------------------------------------
>
> Key: AMBARI-17382
> URL: https://issues.apache.org/jira/browse/AMBARI-17382
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Aravindan Vijayan
> Assignee: Aravindan Vijayan
> Priority: Critical
> Labels: ambari-metrics
> Fix For: 3.0.0
>
> Attachments: AMBARI-17382-1.patch
>
>
> With PHOENIX-914, there is a change in implementation , As earlier, timestamp
> range was passed as a hint to the query to get advantage of native timerange
> optimization in hbase but with new implementation we can mark the timestamp
> column in the schema as a ROW_TIMESTAMP and pass timestamp range with “where”
> clause only to achieve equivalent performance and better accuracy.
> For eq:-
> With earlier implementation , AMS forms query like this:-
> SELECT /*+ NATIVE_TIME_RANGE(1448029523000) */ METRIC_NAME, APP_ID,
> INSTANCE_ID, SERVER_TIME, UNITS, METRIC_SUM, HOSTS_COUNT, METRIC_MAX,
> METRIC_MIN FROM METRIC_AGGREGATE WHERE (METRIC_NAME IN
> ('regionserver.Server.totalRequestCount',
> 'regionserver.Server.blockCacheCountHitPercent',
> 'regionserver.Server.regionCount',
> 'regionserver.Server.compactionQueueLength',
> 'regionserver.Server.storeFileCount', 'master.Server.averageLoad')) AND
> APP_ID = 'ams-hbase' AND SERVER_TIME >= 1448029643000 AND SERVER_TIME <
> 1448033243 ORDER BY METRIC_NAME, SERVER_TIME LIMIT 11520
> But with PHOENIX-914 :-
> Declare SERVER_TIME as ROW_TIMESTAMP in schema:-
> CREATE TABLE DESTINATION_METRICS_TABLE (SERVER_TIME DATE not null, METRIC_ID
> CHAR(15) not null, METRIC_VALUE bigint … CONSTRAINT PK PRIMARY
> KEY(CREATED_DATE ROW_TIMESTAMP, METRIC_ID…)) …;
> And remove hint from the query:-
> SELECT METRIC_NAME, APP_ID, INSTANCE_ID, SERVER_TIME, UNITS, METRIC_SUM,
> HOSTS_COUNT, METRIC_MAX, METRIC_MIN FROM METRIC_AGGREGATE WHERE (METRIC_NAME
> IN ('regionserver.Server.totalRequestCount',
> 'regionserver.Server.blockCacheCountHitPercent',
> 'regionserver.Server.regionCount',
> 'regionserver.Server.compactionQueueLength',
> 'regionserver.Server.storeFileCount', 'master.Server.averageLoad')) AND
> APP_ID = 'ams-hbase' AND SERVER_TIME >= 1448029643000 AND SERVER_TIME <
> 1448033243 ORDER BY METRIC_NAME, SERVER_TIME LIMIT 11520
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
