Anshu, Thanks for your reply. For the first part, can I know for how much duration the topology will be paused. If it is like restarting the existing topology with new configuration, can I assume that topology will be down for the time equal to the DAG timeout for topology, i.e. 'topology.message.timeout.secs' property. My understanding is, if a topology is shut down, it will wait for the DAG timeout to expire before killing the topology so that any messages already in the topology could be processed. I am drawing my first conclusion with this understanding, is it correct?
Thanks, Jayant On Wed, Jul 17, 2019 at 11:55 AM anshu shukla <[email protected]> wrote: > Added Comments Inline > > On Wed, Jul 17, 2019 at 10:04 AM Jayant Sharma <[email protected]> > wrote: > >> Hi, >> >> I have 2 questions: >> 1 - Do storm topologies continue to process new tuples when rebalance for >> that topology is going on. Specifically, I have increased the executors of >> a particular component of that topology by using rebalance command. >> > > - * New tuples: If you mean to say for the tuples received after > rebalance or during rebalance then these msgs will be lost... and need to > replyed for guaranteed processing. Basically all rebalance is like > restarting and submitting the same topology with the new configuration.* > > 2 - I have noticed that rebalance sometimes takes few minutes to complete, >> is this normal or can I somehow reduce the time it takes? >> My concern is if it takes minutes to rebalance a topology and the >> topology is paused while it is being rebalanced, this would cause >> significant latency in my application. >> > > - I have done some work on the same lines in past .. if u want more > details u can skim thru the following paper... > > https://arxiv.org/pdf/1712.00605.pdf > > > >> >> Thanks, >> Jayant Sharma >> > > > -- > Thanks & Regards, > Anshu Shukla >
