Re: Is this the list of metrics reported by storm in JMX?

[Tech Id]

Ok, so even JmxPreparableReporter does not give any more metrics about the
workers.
The only way to get the metrics for spouts/bolts/topologies is non-JMX -
staslev-metrics-reporter or the verisign-metrics-reporter.

-T.I.


On Fri, Oct 14, 2016 at 7:42 AM, Kishorkumar Patil <
kpa...@yahoo-inc.com.invalid> wrote:

> Alessandro,
> One caution is "num-getClusterInfo-calls" is not strictly number of thrift
> calls but # of times method was invoked. We do not have separate thrift
> entry method from rest of the internal calls to getClusterInfo
>
> Regards,
> -Kishor
>
>
> On Friday, October 14, 2016 8:31 AM, Alessandro Bellina <
> abell...@yahoo-inc.com.INVALID> wrote:
>
>
>  T.I.
>
> I ran jconsole and ended up with the same result for a worker.
>
> That said, if you turn on JMX for the daemons you end up with a new MBean
> "metrics" (as given by storm.daemon.metrics.reporter.plugin.domain).
> This is coming from: JmxPreparableReporter.
>
>
> - For nimbus I see stuff like: num-getClusterInfo-calls for thrift calls.
> - For ui, I see metrics around the REST calls.
> - For supervisor, currently I only see "num-slots-used-gauge".
>
> Someone else please correct me if I've missed something.
>
> Thanks
>
> Alessandro
>
>
>
>
>
> On Friday, October 14, 2016 12:58 AM, Tech Id 
> wrote:
> Hi,
>
> I know there are plans to revamp the current metrics system.
> But until then, I just wanted to confirm the metrics I have got through JMX
> and if that is expected.
>
> I added the following to *worker.options*:
>
> -Dcom.sun.management.jmxremote \
> -Dcom.sun.management.jmxremote.port=1%ID% \
> -Dcom.sun.management.jmxremote.authenticate=false \
> -Dcom.sun.management.jmxremote.ssl=false
>
> And I used  cmdline-jmxclient
> 's jar file as:
> java -jar /tmp/cmdline-jmxclient-0.10.3.jar -:- 127.0.0.1:16700 | grep -v
> log4j
> | sort -u
>
>
> The metrics I got were:
> com.sun.management:type=DiagnosticCommand
> com.sun.management:type=HotSpotDiagnostic
> java.lang:name=CodeCacheManager,type=MemoryManager
> java.lang:name=Code Cache,type=MemoryPool
> java.lang:name=Compressed Class Space,type=MemoryPool
> java.lang:name=Metaspace Manager,type=MemoryManager
> java.lang:name=Metaspace,type=MemoryPool
> java.lang:name=PS Eden Space,type=MemoryPool
> java.lang:name=PS MarkSweep,type=GarbageCollector
> java.lang:name=PS Old Gen,type=MemoryPool
> java.lang:name=PS Scavenge,type=GarbageCollector
> java.lang:name=PS Survivor Space,type=MemoryPool
> java.lang:type=ClassLoading
> java.lang:type=Compilation
> java.lang:type=Memory
> java.lang:type=OperatingSystem
> java.lang:type=Runtime
> java.lang:type=Threading
> java.nio:name=direct,type=BufferPool
> java.nio:name=mapped,type=BufferPool
> java.util.logging:type=Logging
> JMImplementation:type=MBeanServerDelegate
>
> None of these seems to be storm-specific and that is kind of expected
> because storm does not report those metrics to JMX.
> Can someone confirm if those really are the metrics I should expect or are
> there more I can get from JMX?
>
> Thanks
> T.I.
>
>
>
>

Re: Is this the list of metrics reported by storm in JMX?

[Kishorkumar Patil]

Alessandro,
One caution is "num-getClusterInfo-calls" is not strictly number of thrift 
calls but # of times method was invoked. We do not have separate thrift entry 
method from rest of the internal calls to getClusterInfo 

Regards,
-Kishor
 

On Friday, October 14, 2016 8:31 AM, Alessandro Bellina 
 wrote:
 

 T.I.

I ran jconsole and ended up with the same result for a worker.

That said, if you turn on JMX for the daemons you end up with a new MBean 
"metrics" (as given by storm.daemon.metrics.reporter.plugin.domain).  This is 
coming from: JmxPreparableReporter.


- For nimbus I see stuff like: num-getClusterInfo-calls for thrift calls. 
- For ui, I see metrics around the REST calls.
- For supervisor, currently I only see "num-slots-used-gauge".

Someone else please correct me if I've missed something.

Thanks

Alessandro





On Friday, October 14, 2016 12:58 AM, Tech Id  wrote:
Hi,

I know there are plans to revamp the current metrics system.
But until then, I just wanted to confirm the metrics I have got through JMX
and if that is expected.

I added the following to *worker.options*:

-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=1%ID% \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false

And I used  cmdline-jmxclient
's jar file as:
java -jar /tmp/cmdline-jmxclient-0.10.3.jar -:- 127.0.0.1:16700 | grep -v log4j
| sort -u


The metrics I got were:
com.sun.management:type=DiagnosticCommand
com.sun.management:type=HotSpotDiagnostic
java.lang:name=CodeCacheManager,type=MemoryManager
java.lang:name=Code Cache,type=MemoryPool
java.lang:name=Compressed Class Space,type=MemoryPool
java.lang:name=Metaspace Manager,type=MemoryManager
java.lang:name=Metaspace,type=MemoryPool
java.lang:name=PS Eden Space,type=MemoryPool
java.lang:name=PS MarkSweep,type=GarbageCollector
java.lang:name=PS Old Gen,type=MemoryPool
java.lang:name=PS Scavenge,type=GarbageCollector
java.lang:name=PS Survivor Space,type=MemoryPool
java.lang:type=ClassLoading
java.lang:type=Compilation
java.lang:type=Memory
java.lang:type=OperatingSystem
java.lang:type=Runtime
java.lang:type=Threading
java.nio:name=direct,type=BufferPool
java.nio:name=mapped,type=BufferPool
java.util.logging:type=Logging
JMImplementation:type=MBeanServerDelegate

None of these seems to be storm-specific and that is kind of expected
because storm does not report those metrics to JMX.
Can someone confirm if those really are the metrics I should expect or are
there more I can get from JMX?

Thanks
T.I.


   

Re: Is this the list of metrics reported by storm in JMX?

[Alessandro Bellina]

T.I.

I ran jconsole and ended up with the same result for a worker.

That said, if you turn on JMX for the daemons you end up with a new MBean 
"metrics" (as given by storm.daemon.metrics.reporter.plugin.domain).  This is 
coming from: JmxPreparableReporter.


- For nimbus I see stuff like: num-getClusterInfo-calls for thrift calls. 
- For ui, I see metrics around the REST calls.
- For supervisor, currently I only see "num-slots-used-gauge".

Someone else please correct me if I've missed something.

Thanks

Alessandro





On Friday, October 14, 2016 12:58 AM, Tech Id  wrote:
Hi,

I know there are plans to revamp the current metrics system.
But until then, I just wanted to confirm the metrics I have got through JMX
and if that is expected.

I added the following to *worker.options*:

-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=1%ID% \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false

And I used  cmdline-jmxclient
's jar file as:
java -jar /tmp/cmdline-jmxclient-0.10.3.jar -:- 127.0.0.1:16700 | grep -v log4j
| sort -u


The metrics I got were:
com.sun.management:type=DiagnosticCommand
com.sun.management:type=HotSpotDiagnostic
java.lang:name=CodeCacheManager,type=MemoryManager
java.lang:name=Code Cache,type=MemoryPool
java.lang:name=Compressed Class Space,type=MemoryPool
java.lang:name=Metaspace Manager,type=MemoryManager
java.lang:name=Metaspace,type=MemoryPool
java.lang:name=PS Eden Space,type=MemoryPool
java.lang:name=PS MarkSweep,type=GarbageCollector
java.lang:name=PS Old Gen,type=MemoryPool
java.lang:name=PS Scavenge,type=GarbageCollector
java.lang:name=PS Survivor Space,type=MemoryPool
java.lang:type=ClassLoading
java.lang:type=Compilation
java.lang:type=Memory
java.lang:type=OperatingSystem
java.lang:type=Runtime
java.lang:type=Threading
java.nio:name=direct,type=BufferPool
java.nio:name=mapped,type=BufferPool
java.util.logging:type=Logging
JMImplementation:type=MBeanServerDelegate

None of these seems to be storm-specific and that is kind of expected
because storm does not report those metrics to JMX.
Can someone confirm if those really are the metrics I should expect or are
there more I can get from JMX?

Thanks
T.I.