agresch opened a new pull request #3329:
URL: https://github.com/apache/storm/pull/3329
## What is the purpose of the change
Allow V2 metric reporters to report metrics with long names or with a short
name and set of dimensions. ConsoleStormReporter was modified to allow both
options.
Long metric names get stored in a metrics registry as previously.
Additionally, we create a map of Task/component/stream dimensions to a
TaskMetricRepo that contains task specific metrics.
If a reporter supports and is configured for short metric names, it will
receive the metrics one repo at a time, first setting the dimensions for the
repo, and then reporting the metrics for that Task set.
## How was the change tested
Ran a topology with ConsoleStormReporters set to report both long and short
metrics.
./bin/storm jar ~/storm/storm-starter.jar
org.apache.storm.starter.WordCountTopology wc -c
topology.metrics.consumer.register="[{\"argument\":null,\"class\":\"org.apache.storm.metric.LoggingMetricsConsumer\",\"parallelism.hint\":1}]"
-c topology.metrics.reporters="[{\"report.period\": 11,\"class\":
\"org.apache.storm.metrics2.reporters.ConsoleStormReporter\",
\"report.dimensions.enabled\": true }, { \"report.period\": 9, \"class\":
\"org.apache.storm.metrics2.reporters.ConsoleStormReporter\",
\"report.dimensions.enabled\": false } ]"
long metric name logging:
```
2020-09-02 13:10:01.280 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO]
storm.topology.wc-1-1599070178.worker_hostname.__system.-1.6701-memory.pools.PS-Old-Gen.committed
2020-09-02 13:10:01.280 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO] value = 133693440
2020-09-02 13:10:01.281 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO]
storm.topology.wc-1-1599070178.worker_hostname.__system.-1.6701-memory.pools.PS-Old-Gen.init
2020-09-02 13:10:01.281 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO] value = 179306496
2020-09-02 13:10:01.281 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO]
storm.topology.wc-1-1599070178.worker_hostname.__system.-1.6701-memory.pools.PS-Old-Gen.max
2020-09-02 13:10:01.281 c.c.m.ConsoleReporter
metrics-console-reporter-3-thread-1 [INFO] value = 894959616
```
Short metric names with dimensions:
```
2020-09-02 13:10:03.259 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] Using dimensions:
2020-09-02 13:10:03.260 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] hostname : worker_hostname
2020-09-02 13:10:03.260 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] componentId : count
2020-09-02 13:10:03.260 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] topologyName : wc
2020-09-02 13:10:03.260 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] streamId : default
2020-09-02 13:10:03.261 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] port : 6701
2020-09-02 13:10:03.261 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] taskid : 11
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] 9/2/20 1:10:03 PM
==============================================================
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] -- Counters
--------------------------------------------------------------------
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] __ack-count-split:default
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] count = 200
2020-09-02 13:10:03.261 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] __emit-count-default
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] count = 200
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
__execute-count-split:default
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] count = 180
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] __transfer-count-default
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] count = 0
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
2020-09-02 13:10:03.262 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
2020-09-02 13:10:03.262 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] Using dimensions:
2020-09-02 13:10:03.262 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] hostname : worker_hostname
2020-09-02 13:10:03.262 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] componentId : spout
2020-09-02 13:10:03.263 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] topologyName : wc
2020-09-02 13:10:03.263 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] port : 6701
2020-09-02 13:10:03.263 o.a.s.m.r.ConsoleStormReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] taskid : 26
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] 9/2/20 1:10:03 PM
==============================================================
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] -- Gauges
----------------------------------------------------------------------
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
receive-queue-arrival_rate_secs
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0.0
2020-09-02 13:10:03.263 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] receive-queue-capacity
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 32768
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
receive-queue-dropped_messages
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
receive-queue-insert_failures
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0.0
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] receive-queue-overflow
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] receive-queue-pct_full
2020-09-02 13:10:03.264 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0.0
2020-09-02 13:10:03.265 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] receive-queue-population
2020-09-02 13:10:03.265 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0
2020-09-02 13:10:03.265 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO]
receive-queue-sojourn_time_ms
2020-09-02 13:10:03.265 c.c.m.ConsoleReporter
metrics-ConsoleDimensionalReporter-2-thread-1 [INFO] value = 0.0
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
