Hi Vincent, >From our experience, rebalancing does not keep the inner state of the bolts/spouts but prepare them from fresh. It does give your components time to clean up or commit offsets so you can recover cleanly afterwards.
Kobi On Thu, Dec 4, 2014 at 12:21 AM, Vincent Russell <[email protected]> wrote: > > I have a question regarding serialization of member variables in spouts. > We are using storm to parse files in HDFS with storm spouts. Because > files usually emit many tuples we store which file we are parsing as a > member variable in the spout. > > We have noticed that when rebalancing happens we tend to lose track of the > file that is being parsed if the file isn't completed. > > Can member variables be deserialized during rebalancing? > > Is there some other good tricks to use where state must be saved between > nextTuple calls to protect yourself from rebalancing. > > > I'm using storm version 0.9.0.1. > > Thanks, > Vincent > -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you.
