Great thanks. I haven’t used LoggingMetrics so will take a look at the output there.
We use scout for monitoring (via JMX plugin for java apps), so I think it should be possible to implement a JMX based impl of IMetricsConsumer, for exposing storm metrics via JMX Thanks Tyson On May 27, 2014, at 10:11 PM, Harsha <[email protected]> wrote: > Hi Tyson, > Yes kafka trident has offset metric and kafkaFetchAvg, > kafkaFetchMax > https://github.com/apache/incubator-storm/blob/master/external/storm-kafka/src/jvm/storm/kafka/trident/TridentKafkaEmitter.java#L64 > -Harsha > > On Tue, May 27, 2014, at 06:55 PM, Tyson Norris wrote: >> Do Trident variants of kafka spouts do something similar? >> Thanks >> Tyson >> >>> On May 27, 2014, at 3:19 PM, "Harsha" <[email protected]> wrote: >>> >>> Raphael, >>> kafka spout sends metrics for kafkaOffset and kafkaPartition you can >>> look at those by using LoggingMetrics or setting up a ganglia. Kafka uses >>> its own zookeeper to store state info per topic & group.id you can look at >>> kafka offsets using >>> kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker >>> -Harsha >>> >>> >>>> On Tue, May 27, 2014, at 03:01 PM, Raphael Hsieh wrote: >>>> Is there a way to tell where in the kafka stream my topology is starting >>>> from? >>>> From my understanding Storm will use zookeeper in order to tell its place >>>> in the Kafka stream. Where can I find metrics on this ? >>>> How can I see how large the stream is? What how much data is sitting in >>>> the stream and what the most recent/oldest position is? >>>> >>>> Thanks >>>> >>>> -- >>>> Raphael Hsieh
