Hey , Nice discussion, but as per my knowledge, every *task have its own state* which gets lost on rebalance operation(this is the reason why tasks can not be increased at run time). Mentioned @ https://github.com/apache/storm/pull/1874
So I have 2 questions - *1- How this state correlates to <key,val> pair in these stateful tasks mentioned in the links. *Is it separate for every task ? *2- Whats is the impact fo rebalance operation on these bolts.* On Tue, Feb 7, 2017 at 11:40 AM, Arun Mahadevan <[email protected]> wrote: > > #1 when a worker is died or killed by manually, storm framework will > restart this worker, is there a ID which doesn't change for the new worker > and the died worker? if there is, how to get it? > > Task Id does not change. The worker can be restarted in the same or > different host and the tasks would be reassigned. > > > > > #2 when a worker is died, is there a case storm framework can't start a > new worker? In this case, how about the grouped message handled by this > worker, it will auto take over by other workers? > > As long as the state is per task it can be handled by other worker. > > > > > #3 suppose we have started a topology with 3 workers, rebalance topology > can change the worker numbers? > > Rebalance can change the number of workers, but the overall number of > tasks remains the same. Only that the task assignment to workers could > change. > > > > Not sure if you are building something different, but storm already has > basic support for stateful bolts (see https://github.com/apache/ > storm/blob/master/docs/State-checkpointing.md). > > Thanks, > > Arun > > > > > > *From: *"shawn.du" <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Tuesday, February 7, 2017 at 11:25 AM > *To: *user <[email protected]> > *Subject: *implmement state management with Apache Ignite > > > > Hi community, > > > > Recently we are trying implement state management using apache ignite. > > Before do the real work, I have several questions, please help. > > > > #1 when a worker is died or killed by manually, storm framework will > restart this worker, is there a ID which doesn't change for the new worker > and the died worker? if there is, how to get it? > > #2 when a worker is died, is there a case storm framework can't start a > new worker? In this case, how about the grouped message handled by this > worker, it will auto take over by other workers? > > #3 suppose we have started a topology with 3 workers, rebalance topology > can change the worker numbers? > > > > Thanks > > Shawn > > > -- Thanks & Regards, Anshu Shukla
