Hi Abhishek, no bolt is not sending tupes on the metrics strem. Number of emitted and transfered tuples is 340 and is not inceasing.
Thanks Stepan 2015-10-07 7:16 GMT+02:00 Abhishek Agarwal <[email protected]>: > Can you check on UI that the bolt is sending tuples on the metric stream? > You have to click on "show system stats" on UI. > On Oct 6, 2015 2:34 PM, "Stepan Urban" <[email protected]> wrote: > >> Hi, >> I tried to increase parallelism of bolts and create some performance >> optimizations with no effect. Metrics stop logging after few hours. Is >> there some mechanism that Storm stops logging when bolts are overloaded? >> Any other sugesstions? >> >> Thanks >> Stepan >> >> 2015-10-01 19:39 GMT+02:00 Stepan Urban <[email protected]>: >> >>> 1. All metrics are not being logged >>> 2. Yes topology is still prgressing, everything other works as normal, >>> worker logs are still running >>> 3. Bolt is slower, it writes data to disk, do some simple preprocessing >>> and send them to another bolt, spout only receive it and send it to bolt. >>> >>> Thanks >>> Stepan >>> >>> 2015-10-01 13:19 GMT+02:00 Abhishek Agarwal <[email protected]>: >>> >>>> Please check the following - >>>> 1. Are all the metrics not being logged or is it just the custom >>>> metrics? Are the in-built system metrics being logged? >>>> 2. When the metrics are not getting logged, is the topology still >>>> progressing? >>>> 3. Is your bolt too slow compared to the spout? >>>> >>>> On Thu, Oct 1, 2015 at 3:14 PM, Stepan Urban <[email protected]> >>>> wrote: >>>> >>>>> Hi Ahishek, >>>>> thanks for the response. Registring metrics is normal: >>>>> transient MultiCountMetric countMetric; >>>>> transient MultiReducedMetric meanMetric; >>>>> >>>>> countMetric = new MultiCountMetric(); >>>>> meanMetric = new MultiReducedMetric(new MeanReducer()); >>>>> context.registerMetric("count", countMetric, 1); >>>>> context.registerMetric("mean", meanMetric, 1); >>>>> >>>>> and >>>>> Config conf = new Config(); >>>>> conf.registerMetricsConsumer(LoggingMetricsConsumer.class, 1); >>>>> >>>>> even with default cluster.xml it stops logging on production computer. >>>>> >>>>> Yesterday I found that everything work on dev computer with data >>>>> loaded from file. But on production computer, when data are receided from >>>>> net it stops logging after few hours. Is it possible to find if some >>>>> thread >>>>> is not working properly using jstack? Or any other way how to debug it? >>>>> >>>>> Thanks >>>>> Stepan >>>>> >>>>> >>>>> >>>>> 2015-10-01 6:26 GMT+02:00 Anishek Agarwal <[email protected]>: >>>>> >>>>>> From what i have found dont remove/add appenders in cluster.xml. not >>>>>> sure about why it works for a few hours and then stops. any specific way >>>>>> you are registering log metrics, is it possible to put some code out ? >>>>>> >>>>>> On Tue, Sep 29, 2015 at 9:14 PM, Stepan Urban < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I am using LoggingMetricsConsumer and MultiReducedMetric to count >>>>>>> several metrics in Storm topology. After few hours of running it stops >>>>>>> logging these metrics into metrics.log file without an error in any log >>>>>>> file. Any sugestions how to find where is the problem? First I used >>>>>>> default >>>>>>> cluster.xml with increased size of files, now I use only >>>>>>> ch.qos.logback.core.FileAppender with the same results. After few mb of >>>>>>> logs it stops logging. >>>>>>> >>>>>>> Thanks >>>>>>> Stepan >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Abhishek Agarwal >>>> >>>> >>> >>
