----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5145/#review7978 -----------------------------------------------------------
trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/MetricCollector.java <https://reviews.apache.org/r/5145/#comment17324> Shall log instead of printStackTree? trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/MetricCollector.java <https://reviews.apache.org/r/5145/#comment17325> Looks like the ex is swallowed here. But should be okay. Better to have more detailed infor for error printing, such as metrics context info, updater info, etc. - Mingjie On 2012-05-17 05:24:59, Mubarak Seyed wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5145/ > ----------------------------------------------------------- > > (Updated 2012-05-17 05:24:59) > > > Review request for Flume. > > > Summary > ------- > > Code review for "Create metric collection infrastructure". This patch adds > fixes for the following > > - Added MetricAware interface, MetricContext data-structure, MetricCollector > - Implemented MetricAware in AvroSource, HDFSEventSink, and AvroSink > - Monitoring plugin processor: Added support in application bootstrap code to > parse and configure monitoring plugin(s) > - Monitoring plugin implementation: Added Ganglia 3.0 and 3.1 plugin to emit > metrics to gmond host:port > > Configuration for monitoring plugin: > ------------------------------- > agent.monitoring = ganglia_monitoring jmx_monitoring > > # plugin type: ganglia31 for version 3.1 compatible, ganglia3 for version 3.0 > compatible and wire protocol is different for both versions. > agent.monitoring.ganglia_monitoring.type = ganglia31 > # hostname where gmond daemon runs to collect and persist metrics in RRD > agent.monitoring.ganglia_monitoring.hostName = ganglia_host > # gmond port > agent.monitoring.ganglia_monitoring.port = 8649 > # polls every x seconds to collect metrics from metrics collector and emit > them to gmond > agent.monitoring.ganglia_monitoring.pollingInterval = 5 > > agent.monitoring.jmx_monitoring.type = jmx > agent.monitoring.jmx_monitoring.port = 8081 > > TODO: > ------ > - Convert all the sources, channels, sinks to use MetricAware interface and > let them be polled by MetricCollector > - Aggregation of metrics in collector side? Most of the counters uses > AtomicLong and increments the value upon success/failure, how do we aggregare > for minutes, hours? I think monitoring solution does the aggregation (e.g, > ganglia) > - MetricContext: Different data type (Double, Float) to store decimal values. > - Metrics: Other than counts, what else we needed? JVM metrics (we can get > from java.lang.management MXBeans) > - Channel: We can keep track of capacity, number of puts, number of takes, > remaining capacity (or exception count?) > - Monitoring plugins: JMX, REST > > Thanks, > Mubarak > > > Diffs > ----- > > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/BasicConfigurationConstants.java > 1338454 > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfiguration.java > 1338454 > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfigurationFactory.java > 1338454 > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java > 1338454 > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/monitoring/MonitoringConfiguration.java > PRE-CREATION > > trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/monitoring/MonitoringType.java > PRE-CREATION > trunk/flume-ng-core/pom.xml 1338454 > trunk/flume-ng-core/src/main/java/org/apache/flume/Monitoring.java > PRE-CREATION > trunk/flume-ng-core/src/main/java/org/apache/flume/MonitoringFactory.java > PRE-CREATION > > trunk/flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java > 1338454 > > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/DefaultMonitoringFactory.java > PRE-CREATION > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/MetricAware.java > PRE-CREATION > > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/MetricCollector.java > PRE-CREATION > > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/MetricContext.java > PRE-CREATION > > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/plugin/Ganglia31Monitoring.java > PRE-CREATION > > trunk/flume-ng-core/src/main/java/org/apache/flume/metrics/plugin/Ganglia3Monitoring.java > PRE-CREATION > trunk/flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java > 1338454 > > trunk/flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java > 1338454 > trunk/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java > 1338530 > trunk/flume-ng-core/src/test/java/org/apache/flume/channel/MockChannel.java > 1338454 > > trunk/flume-ng-node/src/main/java/org/apache/flume/conf/file/AbstractFileConfigurationProvider.java > 1338454 > > trunk/flume-ng-node/src/main/java/org/apache/flume/conf/file/SimpleNodeConfiguration.java > 1338454 > > trunk/flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java > 1338454 > trunk/flume-ng-node/src/main/java/org/apache/flume/node/Application.java > 1338454 > > trunk/flume-ng-node/src/main/java/org/apache/flume/node/NodeConfiguration.java > 1338454 > > trunk/flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java > 1338454 > > trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java > 1338454 > > Diff: https://reviews.apache.org/r/5145/diff > > > Testing > ------- > > Yes. Tested in lab environment. > > > Thanks, > > Mubarak > >
