----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58424/#review173087 -----------------------------------------------------------
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py Lines 21 (patched) <https://reviews.apache.org/r/58424/#comment246166> Can we replace version with expression or make it parameterized for future proofing? Easier to change config vs code-edit. ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py Lines 44 (patched) <https://reviews.apache.org/r/58424/#comment246167> Protocol support for https? Maybe not important for local write but maybe it should be parameterized. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java Lines 32 (patched) <https://reviews.apache.org/r/58424/#comment246153> Case on arguments should be camel vs pythonic ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java Lines 126 (patched) <https://reviews.apache.org/r/58424/#comment246155> Potential NPE without default. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorThread.java Lines 47 (patched) <https://reviews.apache.org/r/58424/#comment246156> This looks funky, instead of static method we should be calling a member on the cache reference. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorThread.java Lines 81 (patched) <https://reviews.apache.org/r/58424/#comment246158> Can we move initiazliation outside? ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorThread.java Lines 110 (patched) <https://reviews.apache.org/r/58424/#comment246160> Since AggregatorThread also send data we should rename to something like AggregatorPublisher and RawMetricPublisher. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorThread.java Lines 114 (patched) <https://reviews.apache.org/r/58424/#comment246159> formatting. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/SenderThread.java Lines 48 (patched) <https://reviews.apache.org/r/58424/#comment246161> Again intialize once. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/SenderThread.java Lines 60 (patched) <https://reviews.apache.org/r/58424/#comment246162> Can we move this to a helper class to avoid dup? ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java Lines 12 (patched) <https://reviews.apache.org/r/58424/#comment246164> Why static, not a best prcatice just make it a singleton. ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java Lines 23 (patched) <https://reviews.apache.org/r/58424/#comment246165> Again adding Raw vs aggreate will imprive readbility. - Sid Wagle On April 26, 2017, 4:44 p.m., Dmytro Sen wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58424/ > ----------------------------------------------------------- > > (Updated April 26, 2017, 4:44 p.m.) > > > Review request for Ambari, Aravindan Vijayan and Sid Wagle. > > > Bugs: AMBARI-20758 > https://issues.apache.org/jira/browse/AMBARI-20758 > > > Repository: ambari > > > Description > ------- > > Abstract Sink checks if there is local known monitor port available to write > (e.g.: 61888) > Handshake / Ping to make sure its the monitor and is accepting writes > Start sending metrics only to the monitor > Monitor sends metrics at the global send interval (1 minute) with all the > granularity of the sink > Monitor aggregates first interval (5 min) data and sends it every interval > seconds. (5 min) > The monitor interface look the same as that of collector (http server: maybe > Flask) > Security concerns need to be addressed as well. (https/spnego?) > > > Diffs > ----- > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java > 2d1bf40 > ambari-metrics/ambari-metrics-assembly/pom.xml a4b87de > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java > 46f32f9 > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregationResult.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/MetricAggregate.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/MetricClusterAggregate.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/MetricHostAggregate.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java > 44c9d4a > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetricWithAggregatedValues.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/AbstractTimelineMetricSinkTest.java > 9b0cdbe > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java > a393a96 > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java > 32fe32e > > ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java > 3fdf3f4 > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > 8e0de03 > ambari-metrics/ambari-metrics-host-aggregator/pom.xml PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorThread.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/SenderThread.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/main/resources/log4j.properties > PRE-CREATION > > ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/AppTest.java > PRE-CREATION > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py > PRE-CREATION > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py > 5686c50 > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py > c0feed5 > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py > ba3f18e > > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/stop_handler.py > bfb6957 > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/main.py > d218015 > > ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java > 9d492cb > > ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java > 9a55f10 > > ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java > 60c1427 > > ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java > 535fae0 > > ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java > f58f549 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > fa095a0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 65bbc4c > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 0d5042f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > 121a8ae > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java > 65d54c0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/MetricAggregate.java > 825ac25 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/MetricClusterAggregate.java > 9c837b6 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/MetricHostAggregate.java > 340ec75 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAppAggregator.java > 44aca03 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregator.java > 02677b9 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > a5a3499 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricHostAggregator.java > 0ea9c08 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java > b5f49fb > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > 6278c59 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > 0087fd9 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java > 37ec134 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java > a910cc2 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestMetricHostAggregate.java > 44f48e8 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > b40481d > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java > fa0cfe9 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITClusterAggregator.java > 590f82a > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITMetricAggregator.java > 9873643 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java > 78db11d > ambari-metrics/pom.xml 2d88912 > > ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java > f268149 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py > 53cf002 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/templates/hadoop-metrics2-accumulo.properties.j2 > f7926d0 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > 8e1671e > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml > 740a91a > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 919f26d > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/hadoop-metrics2-hbase.properties.j2 > 8c6f86f > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 > 1505f9b > > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py > b143941 > > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/templates/flume-metrics2.properties.j2 > cf2bd6c > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py > 3177643 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2 > e808f5d > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2 > 7e1abbc > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hadoop-metrics2.properties.xml > 4aadb83 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py > 7615ee7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-hivemetastore.properties.j2 > 82f71c5 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-hiveserver2.properties.j2 > f1bfa03 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-llapdaemon.j2 > d63dfb7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-llaptaskscheduler.j2 > 4c9c981 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/params_linux.py > 7615ee7 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/templates/hadoop-metrics2-hivemetastore.properties.j2 > 82f71c5 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/templates/hadoop-metrics2-hiveserver2.properties.j2 > f1bfa03 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/templates/hadoop-metrics2-llapdaemon.j2 > d63dfb7 > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/templates/hadoop-metrics2-llaptaskscheduler.j2 > 4c9c981 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml > f2b4a76 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py > 44b256e > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/config.yaml.j2 > f2a75cf > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2 > 001ff73 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py > 70ebfeb > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2 > 2f3aab6 > > ambari-server/src/main/resources/stacks/HDP/2.6/services/HDFS/configuration/hadoop-metrics2.properties.xml > 4aadb83 > > ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/params.py > 5a5361c > > ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/hadoop-metrics2.properties.j2 > 2f3aab6 > > ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/configuration/hadoop-metrics2.properties.xml > 4aadb83 > > ambari-server/src/test/java/org/apache/ambari/server/metric/system/impl/TestAmbariMetricsSinkImpl.java > 4a60892 > > contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/before-START/scripts/params.py > 6a70675 > > contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks/before-START/templates/hadoop-metrics2.properties.j2 > fcd9b23 > > > Diff: https://reviews.apache.org/r/58424/diff/2/ > > > Testing > ------- > > Under development > > > Thanks, > > Dmytro Sen > >
