Let me ask a follow up question. What happens with configuration information that the bolt reads from the classpath? And what happens to those that the bolt reads from an external configuration file. The above statement would imply that they would be lost in the latter case. Is this correct? Would they also be lost in case of classpath?
Thanks, Max. On Wed, Jul 8, 2015 at 10:49 AM, Thilina Rathnayake <[email protected]> wrote: > Hi Taylor, > > Thank you very much for the information. You cleared all my doubts. > > Regards, > Thilina > > On Tue, Jul 7, 2015 at 9:19 PM, P. Taylor Goetz <[email protected]> wrote: > >> Hi Thilina, >> >> 1. With Storm’s Core API (spouts/bolts) if you are storing state in an >> in-memory structure such as a HashMap, that state will be lost in the event >> of a worker crash. It is up to you to provide the logic for persisting >> state and restoring it in the event of a failure; Storm will not do it for >> you. Storm’s trident API on the other hand has an abstraction for >> persistent state [1] and there are implementations for various backing >> stores like HBase, Redis, and others. In the storm-hbase module there is an >> example of a persistent word count. >> >> 2. Storm uses zookeeper for things like nimbus assignments and >> worker/supervisor heartbeats. The Trident API also uses zookeeper for >> tracking transactions. Other components may use zookeeper as well. The >> kafka spout, for example, uses zookeeper for tracking offsets. >> >> -Taylor >> >> >> [1] https://storm.apache.org/documentation/Trident-state.html >> >> >> > On Jul 7, 2015, at 2:16 AM, Thilina Rathnayake <[email protected]> >> wrote: >> > >> > Hi All, >> > >> > I am a newbie to storm and have a few questions regarding the state in >> storm. I will be >> > really grateful if someone can help me out with the following. >> > >> > * Consider the word count topology in `storm-starter`. We have a map >> called `counts` >> > in it. Does storm store `count` in zookeeper or anywhere else so that >> if the worker >> > dies we are able to recover the data inside `counts`? >> > >> > * What state information of supervisors/workers are stored in the >> zookeeper? >> > >> > Thanks in advance ! >> > >> > Regards, >> > Thilina >> >> >
