Thanks a lot, this is really helpful. I will read the code logic and may again need to revert back if required.
- Mainly I have to look that *how spout* came to know that *mkAssignments in [3] is complete. [Doubt- either it waits for some internal __system ack or just start emitting]* On Mon, Apr 17, 2017 at 9:41 PM, Arun Mahadevan <[email protected]> wrote: > During a rebalance [1] nimbus resets the component_executor map and > arranges for a “DO_REBALANCE” to be triggered after a delay (see [2]). > > > > When nimbus receives the “DO_REBALANCE”, it does the actual rebalance > where it computes and makes the new assignments [3]. > > > > I would assume the spouts/bolts stops sometime after [1] and resumes only > after mkAssignments in [3] is complete. > > > > Thanks, > > Arun > > > > [1] https://github.com/apache/storm/blob/master/storm- > server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L2652 > > [2] https://github.com/apache/storm/blob/master/storm- > server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L260 > > [3] https://github.com/apache/storm/blob/master/storm- > server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L1181 > > > > > > *From: *anshu shukla <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Monday, April 17, 2017 at 8:11 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Spout resume after rebalance > > > > *Does spout receives any special msg and only then it resumes to emit > msgs ? * > > > > On Sat, Apr 15, 2017 at 5:03 PM, anshu shukla <[email protected]> > wrote: > > Hello , > > > > I have a doubt that How storm identifies that on rebalancing when spout > should start emitting msgs. > > Say in case I gave a wait time of 0 (-w 0) then rebalance will start > without pausing spout and msgs between will be lost. > > > > *(Inflight msgs are lost but what about new incoming msgs ?)* > > In short: > > *How does spout confirms that all downstream bolts are ready to receive > msgs after rebalance and spout should start emitting msgs?* > > > > -- > > Thanks & Regards, > Anshu Shukla > > > > > > -- > > Thanks & Regards, > Anshu Shukla > -- Thanks & Regards, Anshu Shukla
