[
https://issues.apache.org/jira/browse/STORM-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036100#comment-14036100
]
ASF GitHub Bot commented on STORM-200:
--------------------------------------
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/38#discussion_r13931874
--- Diff: storm-core/test/clj/backtype/storm/metrics_test.clj ---
@@ -143,6 +153,62 @@
(advance-cluster-time cluster 5)
(assert-buckets! "2" "my-custom-metric" [1 0 0 0 0 0 2]))))
+(defn mk-shell-bolt-with-metrics-spec
+ [inputs command & kwargs]
+ (let [command (into-array String command)]
+ (apply thrift/mk-bolt-spec inputs
+ (PythonShellMetricsBolt. command) kwargs)))
+
+(deftest test-custom-metric-with-multilang-py
+ (with-local-cluster
+ [cluster :daemon-conf {TOPOLOGY-METRICS-CONSUMER-REGISTER
+ [{"class"
"clojure.storm.metric.testing.FakeMetricConsumer"}]
+ "storm.zookeeper.connection.timeout" 30000
+ "storm.zookeeper.session.timeout" 60000
+ }]
+ (let [feeder (feeder-spout ["field1"])
+ topology (thrift/mk-topology
+ {"1" (thrift/mk-spout-spec feeder)}
+ {"2" (mk-shell-bolt-with-metrics-spec {"1" :global}
["python" "tester_bolt_metrics.py"])})]
+ (submit-local-topology (:nimbus cluster) "shell-metrics-tester" {}
topology)
+
+ (.feed feeder ["a"] 1)
+ (Thread/sleep 6000)
+ (assert-buckets! "2" "my-custom-shell-metric" [1])
+
+ (Thread/sleep 5000)
+ (assert-buckets! "2" "my-custom-shell-metric" [1 0])
+
+ (Thread/sleep 20000)
+ (assert-buckets! "2" "my-custom-shell-metric" [1 0 0 0 0 0])
+
+ (.feed feeder ["b"] 2)
+ (.feed feeder ["c"] 3)
+ (Thread/sleep 5000)
--- End diff --
Sleep in a test to get the timing right is not Ideal. I can see in the
other multi-lang tests they mostly just verify that the topology ran, nothing
about the metrics being consumed. I would prefer to see you use
with-simulated-time-local-cluster instead. I don't see any reason why they
should not work with shell bolts and spouts.
> Proposal for Multilang's Metrics feature
> ----------------------------------------
>
> Key: STORM-200
> URL: https://issues.apache.org/jira/browse/STORM-200
> Project: Apache Storm (Incubating)
> Issue Type: New Feature
> Reporter: DashengJu
> Priority: Minor
>
> Storm 0.9.0.1 exposes a metrics interface to report summary statistics across
> the full topology. We can build our own metric, and build metrics consumer to
> use those statistics.
> But when we use Multilang(ie. Python), we can not use this feature. So we
> want to summit a proposal for multilang's metrics.
> The specifics of the proposal:
> 1. The main idea is: when user want to add a metric statistics in
> multilang(python) bolt，
> a) he need first create a metric object and register in ShellBolt's
> sub-class,
> b) then update the metric in Python bolt process through RPC call.
> 2. In Metrics API:
> a) extend IMetric interface add a method for RPC call：public void
> updateMetricFromRPC(List<Object> params);
> b) modify IMetric implements, to support updateMetricFromRPC;
> 3. In ShellBolt，
> a) we have a Map<String, IMetric> to hold user's registered metrics
> object；
> b) we have a method registerMetric(String name, T metric) for user to
> register their metris object;
> c) we have a method handleMetrics(Map action) to handle RPC call from
> Python process；
> 4) In Multilang protocol: add a command "metrics" for shell process to make
> RPC call. The protocol is: {"command":"metrics",
> "name":"metric-registerd-name", "params":["param-1", param-2]}
> 5) In storm.py：add rpcMetrics(name, params), user can update remote metric
> through this RPC call.
> any suggestions?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
