Oops, this fell off the mailing list at some point. I'm not too familiar with those settings, so I'm going off the documentation for those settings in Config.java ( https://github.com/apache/storm/blob/c08cfba0b65513b7b11d5ddbd399f7f15438f9cf/storm-client/src/jvm/org/apache/storm/Config.java). I think the topology.component.resources.* parameters are meant for use with the RAS scheduler ( http://storm.apache.org/releases/2.0.0-SNAPSHOT/Resource_Aware_Scheduler_overview.html). I'd guess that setting worker.heap.memory.mb is enough if you're not using that scheduler, but I'm not certain.
2017-07-27 18:50 GMT+02:00 pradeep s <[email protected]>: > Thanks so much Stig. I have one more question regarding the heap settings > for worker node . > Can you please confirm the difference of below params > worker.heap.memory.mb > topology.component.resources.onheap.memory.mb > If i set worker heap as 4GB, do i need to set the resources heap also? > > > On Wed, Jul 26, 2017 at 11:36 AM, Stig Rohde Døssing <[email protected]> > wrote: > >> They should be in your logs directory on the machine running the UI >> daemon ("storm ui"). You're probably looking for the one called ui.log. >> >> 2017-07-26 20:22 GMT+02:00 pradeep s <[email protected]>: >> >>> Hi Stig, >>> I have tried the lag endpoint using the topology id and its returning a >>> empty response .Any way to debug this? >>> Where can i see the logs for the ui rest api. >>> Regards >>> Pradeep S >>> >>> On Tue, Jul 25, 2017 at 4:30 AM, Stig Rohde Døssing <[email protected]> >>> wrote: >>> >>>> I don't think the endpoint is listed in the docs yet. Here's the source >>>> https://github.com/apache/storm/blob/e40d213de7067f7d3aa4d49 >>>> 92b81890d8ed6ff31/storm-core/src/clj/org/apache/storm/ui/core.clj#L1160 >>>> >>>> 2017-07-24 22:38 GMT+02:00 pradeep s <[email protected]>: >>>> >>>>> Thanks Stig.I cant see a lag related endpoint in docs. >>>>> There is spout stats available from storm ui rest api.But nothing >>>>> related to consumer lag. >>>>> /api/v1/topology/:id/component/:component (GET) >>>>> >>>>> Can you please help in finding out the lag endpoint. >>>>> >>>>> On Fri, Jul 21, 2017 at 1:11 AM, Stig Rohde Døssing <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> If you are on Storm version 1.1.0 or above you should be able to see >>>>>> consumer lag in Storm UI when you look at a topology that contains a >>>>>> Kafka >>>>>> spout. You can use your browser's developer tools to see the URL for the >>>>>> consumer lag endpoint, it will be ending in /lag. If you want you can run >>>>>> the same code manually via https://github.com/apache/stor >>>>>> m/blob/master/bin/storm-kafka-monitor, which will be in your Storm >>>>>> installation's /bin. An alternative is to use the >>>>>> kafka-consumer-groups.sh >>>>>> script in your Kafka installation's /bin, which I believe can also show >>>>>> this information. >>>>>> >>>>>> I'm not sure we have a messages per second built in, but Storm UI can >>>>>> show number of tuples emitted/acked for the previous 10 minutes, and you >>>>>> could probably derive messages per second from that. I think you can get >>>>>> the source metrics via the metrics API if you don't want to go via Storm >>>>>> UI, take a look at https://github.com/revans2/inc >>>>>> ubator-storm/blob/88de24a5afd99df28c4fe304eafa5d53473a46c2/d >>>>>> ocs/Metrics.md. >>>>>> >>>>>> 2017-07-21 <20%2017%2007%2021> 4:22 GMT+02:00 pradeep s < >>>>>> [email protected]>: >>>>>> >>>>>>> Hi >>>>>>> How can I get consumer group related metrics for my kafka spout .i >>>>>>> want to expose consumer lag and messages per second metrics via hmm for >>>>>>> reporting and alerting >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
