Repository: storm
Updated Branches:
  refs/heads/master 4f5fdd928 -> 2ed793d9b


STORM-2908 Document Metrics V2 for 2.0.0

* commit squashed by Jungtaek Lim <kabh...@gmail.com>

This closes #2628


Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/2c61eba9
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/2c61eba9
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/2c61eba9

Branch: refs/heads/master
Commit: 2c61eba95caa2600e978972fc1749379cc57c5e1
Parents: 4f5fdd9
Author: Aaron Gresch <agre...@yahoo-inc.com>
Authored: Tue Apr 10 08:26:41 2018 -0500
Committer: Jungtaek Lim <kabh...@gmail.com>
Committed: Sat Apr 14 02:15:36 2018 +0900

----------------------------------------------------------------------
 conf/storm.yaml.example |  26 +++++++-
 docs/index.md           |   3 +-
 docs/metrics_v2.md      | 149 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 176 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/storm/blob/2c61eba9/conf/storm.yaml.example
----------------------------------------------------------------------
diff --git a/conf/storm.yaml.example b/conf/storm.yaml.example
index 6d75963..6a8d3c5 100644
--- a/conf/storm.yaml.example
+++ b/conf/storm.yaml.example
@@ -80,4 +80,28 @@
 #   - class: "org.apache.storm.metric.FileBasedEventLogger"
 #   - class: "org.mycompany.MyEventLogger"
 #     arguments:
-#       endpoint: "event-logger.mycompany.org"
\ No newline at end of file
+#       endpoint: "event-logger.mycompany.org"
+
+# Metrics v2 configuration (optional)
+#storm.metrics.reporters:
+#  # Graphite Reporter
+#  - class: "org.apache.storm.metrics2.reporters.GraphiteStormReporter"
+#    daemons:
+#        - "supervisor"
+#        - "nimbus"
+#        - "worker"
+#    report.period: 60
+#    report.period.units: "SECONDS"
+#    graphite.host: "localhost"
+#    graphite.port: 2003
+#
+#  # Console Reporter
+#  - class: "org.apache.storm.metrics2.reporters.ConsoleStormReporter"
+#    daemons:
+#        - "worker"
+#    report.period: 10
+#    report.period.units: "SECONDS"
+#    filter:
+#        class: "org.apache.storm.metrics2.filters.RegexFilter"
+#        expression: ".*my_component.*emitted.*"
+

http://git-wip-us.apache.org/repos/asf/storm/blob/2c61eba9/docs/index.md
----------------------------------------------------------------------
diff --git a/docs/index.md b/docs/index.md
index af7c182..b1d881f 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -77,7 +77,8 @@ But small change will not affect the user experience. We will 
notify the user wh
 * [Distributed RPC](Distributed-RPC.html)
 * [Transactional topologies](Transactional-topologies.html)
 * [Hooks](Hooks.html)
-* [Metrics](Metrics.html)
+* [Metrics (Deprecated)](Metrics.html)
+* [Metrics V2](metrics_v2.html)
 * [State Checkpointing](State-checkpointing.html)
 * [Windowing](Windowing.html)
 * [Joining Streams](Joins.html)

http://git-wip-us.apache.org/repos/asf/storm/blob/2c61eba9/docs/metrics_v2.md
----------------------------------------------------------------------
diff --git a/docs/metrics_v2.md b/docs/metrics_v2.md
new file mode 100644
index 0000000..e90ec76
--- /dev/null
+++ b/docs/metrics_v2.md
@@ -0,0 +1,149 @@
+---
+title: Metrics Reporting API v2
+layout: documentation
+documentation: true
+---
+Apache Storm version 1.2 introduced a new metrics system for reporting
+internal statistics (e.g. acked, failed, emitted, transferred, disruptor queue 
metrics, etc.) as well as a 
+new API for user defined metrics.
+
+The new metrics system is based on [Dropwizard 
Metrics](http://metrics.dropwizard.io).
+
+
+## User Defined Metrics
+To allow users to define custom metrics, the following methods have been added 
to the `TopologyContext`
+class, an instance of which is passed to spout's `open()` method and bolt's 
`prepare()` method:
+
+    public Timer registerTimer(String name)
+
+    public Histogram registerHistogram(String name)
+
+    public Meter registerMeter(String name)
+
+    public Counter registerCounter(String name)
+
+    public Gauge registerGauge(String name, Gauge gauge)
+
+API documentation: 
[Timer](http://metrics.dropwizard.io/4.0.0/apidocs/com/codahale/metrics/Timer.html),
 
+[Histogram](http://metrics.dropwizard.io/4.0.0/apidocs/com/codahale/metrics/Histogram.html),
+[Meter](http://metrics.dropwizard.io/4.0.0/apidocs/com/codahale/metrics/Meter.html),
+[Counter](http://metrics.dropwizard.io/4.0.0/apidocs/com/codahale/metrics/Counter.html),
+[Gauge](http://metrics.dropwizard.io/4.0.0/apidocs/com/codahale/metrics/Gauge.html)
+
+Each of these methods takes a `name` parameter that acts as an identifier. 
When metrics are 
+registered, Storm will add additional information such as hostname, port, 
topology ID, etc. to form a unique metric
+identifier. For example, if we register a metric named `myCounter` as follows:
+
+```java
+    Counter myCounter = topologyContext.registerCounter("myCounter");
+```
+the resulting name sent to metrics reporters will expand to:
+
+```
+   storm.topology.{topology ID}.{hostname}.{component ID}.{task ID}.{worker 
port}-myCounter 
+```
+
+The additional information allows for the unique identification of metrics for 
component instances across the cluster.
+
+*Important Note:* In order to ensure metric names can be reliably parsed, any 
`.` characters in name components will
+be replaced with an underscore (`_`) character. For example, the hostname 
`storm.example.com` will appear as
+`storm_example_com` in the metric name. This character substitution *is not 
applied to the user-supplied `name` parameter.
+
+### Example: Tuple Counter Bolt
+The following example is a simple bolt implementation that will report the 
running total up tuples received by a bolt:
+
+```java
+public class TupleCountingBolt extends BaseRichBolt {
+    private Counter tupleCounter;
+    @Override
+    public void prepare(Map stormConf, TopologyContext context, 
OutputCollector collector) {
+        this.tupleCounter = context.registerCounter("tupleCount");
+    }
+
+    @Override
+    public void execute(Tuple input) {
+        this.tupleCounter.inc();
+    }
+}
+```
+ 
+## Metric Reporter Configuration
+
+ For metrics to be useful they must be *reported*, in other words sent 
somewhere where they can be consumed and analyzed.
+ That can be as simple as writing them to a log file, sending them to a time 
series database, or exposing them via JMX.
+ 
+ As of Storm 1.2.0 the following metric reporters are supported
+ 
+  * Console Reporter 
(`org.apache.storm.metrics2.reporters.ConsoleStormReporter`):
+    Reports metrics to `System.out`.
+  * CSV Reporter (`org.apache.storm.metrics2.reporters.CsvReporter`):
+    Reports metrics to a CSV file.
+  * Ganglia Reporter 
(`org.apache.storm.metrics2.reporters.GagliaStormReporter`):
+    Reports metrics to a [Ganglia](http://ganglia.info) server.
+  * Graphite Reporter 
(`org.apache.storm.metrics2.reporters.GraphiteStormReporter`):
+    Reports metrics to a [Graphite](https://graphiteapp.org) server.
+  * JMX Reporter (`org.apache.storm.metrics2.reporters.JmxStormReporter`):
+    Exposes metrics via JMX.
+  
+  
+ Metrics reporters are configured in the `storm.yaml` file. By default, Storm 
will collect metrics but not "report" or
+ send the collected metrics anywhere. To enable metrics reporting, add a 
`storm.metrics.reporters` section to `storm.yaml`
+ and configure one or more reporters.
+ 
+ The following example configuration sets up two reporters: a Graphite 
Reporter and a Console Reporter:
+ 
+ ```yaml
+storm.metrics.reporters:
+  # Graphite Reporter
+  - class: "org.apache.storm.metrics2.reporters.GraphiteStormReporter"
+    daemons:
+        - "supervisor"
+        - "nimbus"
+        - "worker"
+    report.period: 60
+    report.period.units: "SECONDS"
+    graphite.host: "localhost"
+    graphite.port: 2003
+
+  # Console Reporter
+  - class: "org.apache.storm.metrics2.reporters.ConsoleStormReporter"
+    daemons:
+        - "worker"
+    report.period: 10
+    report.period.units: "SECONDS"
+    filter:
+        class: "org.apache.storm.metrics2.filters.RegexFilter"
+        expression: ".*my_component.*emitted.*"
+
+```
+
+Each reporter section begins with a `class` parameter representing the 
fully-qualified class name of the reporter 
+implementation. The `daemons` section determines which daemons the reporter 
will apply to (in the example Graphite
+Reporter is configured to report metrics from all Storm daemons, while the 
Console reporter will only report worker and
+topology metrics).
+
+Many reporter implementations are *scheduled*, meaning they report metrics at 
regular intervals. The reporting interval
+is determined by the `report.period` and `report.period.units` parameters.
+
+Reporters can also be configured with an optional filter that determines which 
metrics get reported. Storm includes the
+`org.apache.storm.metrics2.filters.RegexFilter` filter which uses a regular 
expression to determine which metrics get
+reported. Custom filters can be created by implementing the 
`org.apache.storm.metrics2.filters.StormMetricFilter`
+interface:
+
+```java
+public interface StormMetricsFilter extends MetricFilter {
+
+    /**
+     * Called after the filter is instantiated.
+     * @param config A map of the properties from the 'filter' section of the 
reporter configuration.
+     */
+    void prepare(Map<String, Object> config);
+    
+   /**
+    *  Returns true if the given metric should be reported.
+    */
+    boolean matches(String name, Metric metric);
+
+}
+```
+

Reply via email to