-Reply On Wed, Jul 17, 2019 at 1:11 PM Jayant Sharma <[email protected]> wrote:
> Anshu, > > Thanks for your reply. For the first part, can I know for how much > duration the topology will be paused. > - It will be paused for 30 sec by default before killing the executors.. > If it is like restarting the existing topology with new configuration, can > I assume that topology will be down for the time equal to the DAG timeout > for topology, i.e. 'topology.message.timeout.secs' property. > - YES My understanding is, if a topology is shut down, it will wait for the DAG > timeout to expire before killing the topology so that any messages already > in the topology could be processed. I am drawing my first conclusion with > this understanding, is it correct? > - Yes .. you are correct. .. but sometimes we need more time than this... so we can not always say that all inflight msgs will be lost before killing the topo. > > Thanks, > Jayant > > On Wed, Jul 17, 2019 at 11:55 AM anshu shukla <[email protected]> > wrote: > >> Added Comments Inline >> >> On Wed, Jul 17, 2019 at 10:04 AM Jayant Sharma <[email protected]> >> wrote: >> >>> Hi, >>> >>> I have 2 questions: >>> 1 - Do storm topologies continue to process new tuples when rebalance >>> for that topology is going on. Specifically, I have increased the executors >>> of a particular component of that topology by using rebalance command. >>> >> >> - * New tuples: If you mean to say for the tuples received after >> rebalance or during rebalance then these msgs will be lost... and need to >> replyed for guaranteed processing. Basically all rebalance is like >> restarting and submitting the same topology with the new configuration.* >> >> 2 - I have noticed that rebalance sometimes takes few minutes to >>> complete, is this normal or can I somehow reduce the time it takes? >>> My concern is if it takes minutes to rebalance a topology and the >>> topology is paused while it is being rebalanced, this would cause >>> significant latency in my application. >>> >> >> - I have done some work on the same lines in past .. if u want more >> details u can skim thru the following paper... >> >> https://arxiv.org/pdf/1712.00605.pdf >> >> >> >>> >>> Thanks, >>> Jayant Sharma >>> >> >> >> -- >> Thanks & Regards, >> Anshu Shukla >> > -- Thanks & Regards, Anshu Shukla
