Hi Thilina, 1. With Storm’s Core API (spouts/bolts) if you are storing state in an in-memory structure such as a HashMap, that state will be lost in the event of a worker crash. It is up to you to provide the logic for persisting state and restoring it in the event of a failure; Storm will not do it for you. Storm’s trident API on the other hand has an abstraction for persistent state [1] and there are implementations for various backing stores like HBase, Redis, and others. In the storm-hbase module there is an example of a persistent word count.
2. Storm uses zookeeper for things like nimbus assignments and worker/supervisor heartbeats. The Trident API also uses zookeeper for tracking transactions. Other components may use zookeeper as well. The kafka spout, for example, uses zookeeper for tracking offsets. -Taylor [1] https://storm.apache.org/documentation/Trident-state.html > On Jul 7, 2015, at 2:16 AM, Thilina Rathnayake <[email protected]> wrote: > > Hi All, > > I am a newbie to storm and have a few questions regarding the state in storm. > I will be > really grateful if someone can help me out with the following. > > * Consider the word count topology in `storm-starter`. We have a map called > `counts` > in it. Does storm store `count` in zookeeper or anywhere else so that if > the worker > dies we are able to recover the data inside `counts`? > > * What state information of supervisors/workers are stored in the zookeeper? > > Thanks in advance ! > > Regards, > Thilina
signature.asc
Description: Message signed with OpenPGP using GPGMail
