Can you elaborate on the sleep/backoff strategy part? On Fri, Jul 29, 2022, 06:31 Mohit Goyal <[email protected]> wrote:
> Hi Rui, > > I'm using storm 1.2.1. I've written my spouts on top of BaseRichSpout. > Passing default values. > > *Thanks & Regards* > *Mohit Goyal* > > > On Fri, Jul 22, 2022 at 7:45 PM Rui Abreu <[email protected]> wrote: > >> Hi Mohit, >> >> Which version of Storm are you using? >> What are the values you are passing to the Spout on KafkaSpoutConfig ? >> >> On Thu, 21 Jul 2022 at 08:17, Mohit Goyal <[email protected]> >> wrote: >> >>> Hi All, >>> >>> We are using apache storm to read msgs from kafka queues. Traffic >>> usually is high in these kafka queues so processing is almost real time. >>> But I'm facing one problem when load is relatively low and spout doesn't >>> have msgs to read. In this case, the spout will go into sleep for sometime >>> (~5 seconds) depending upon my sleep/backoff strategy. During this spout >>> sleep interval, If there is a msg available in kafka queues, my spout will >>> not read it. Spout will only be able to process the msg once the >>> sleep/backoff time is done. Hence there is a delay or around 3-4 sec in >>> processing of that msg. Is there any way present in Apache Storm to >>> notify the sleeping spout so that it can start processing again? >>> >>> Any leads are highly appreciated. >>> >>> *Thanks & Regards* >>> *Mohit Goyal* >>> >>
