I don't think you are necessarily limited to one approach. Personally, I do something more like the former, although rather than relying on putting custom properties in the storm conf, I just pass them as arguments to topology component constructors...although this may not be best practice.
The key is that the injector isn't going to be serializable; and in my case the object graph I build with the injector is also not serializable, so that entire construct has to be built once the topology is thrown "over the wall" to the workers. Think of all the pre-serialization parameters as "configuration" (either via StormConf or a more pojo based approach) and post-serialization objects as true dependencies that can acquire and hold resources from the target environment, etc. There is nothing stopping you from using DI to build the topology components pre-serialization if all the injected values are serializable, although doing both pre and post serialization DI is probably going to get hairy as there is no clear way to tell guice where to "stop" injecting the object graph. It really comes down to what you're trying to achieve. I hope this helps. On Thu, Mar 27, 2014 at 7:23 PM, Software Dev <[email protected]>wrote: > Actually.. some more questions. We are running 0.9.1-incubating so we > do in fact have the prepare method. > > Confused on.. > > Should I add all configuration values (from a properties file) onto > the StormConf and then in the prepare method of my workers just > instantiate an Injector? > > Or... > > Should I instantiate an injector right away that is used to > instantiate some factory classes that will create bolts with primitive > arguments? > > On Thu, Mar 27, 2014 at 10:13 AM, Software Dev > <[email protected]> wrote: > > Thanks. I think that cleared up most of my misunderstanding. > > > > On Thu, Mar 27, 2014 at 6:16 AM, Adam Lewis <[email protected]> wrote: > >> Yes that is exactly right, the submission to Nimbus is in the form of a > big > >> thrift message describing the topology...this message includes java > >> serialized blobs of your topology components (spouts/bolts). They get > >> instantiated within the VM calling StormSubmitter. Typically you would > pass > >> configuration info to the constructor, but "dependencies" (e.g. DB > >> connection pool, etc) are transient fields. Then in the prepare method > >> (called after deserialization on the worker) you use the serialized > >> configuration fields to initialize the transient ones. Of course Guice > fits > >> naturally into that step. > >> > >> > >> On Thu, Mar 27, 2014 at 12:37 AM, Software Dev < > [email protected]> > >> wrote: > >>> > >>> Ok so you would configure the map in the main method before submitting > >>> the topology. Then this conf can be used to create guice injectors. Is > >>> that correct? > >>> > >>> In the book "Getting Started with Storm" it states: > >>> > >>> "To customize a bolt, you should set parameters in its constructor and > >>> save them as instance variables so they will be serialized when > >>> submitting the bolt to the cluster." > >>> > >>> Does this mean bolts are instantiated on the client side before being > >>> submitted to nimbus/cluster? > >>> > >>> On Wed, Mar 26, 2014 at 2:05 PM, Svend Vanderveken > >>> <[email protected]> wrote: > >>> > > >>> > The storm configuration map is part of the arguments received by each > >>> > prepare() method, in most Storm primitives, on each worker. It's > >>> > serialised > >>> > to each worker when a topology instance is started there. The initial > >>> > storm > >>> > configuration map is provided at deploy time to Nimbus, in the class > >>> > containing the main() method, specified in the "storm jar blabla.jar > >>> > some.class.here" command. > >>> > > >>> > > >>> > > >>> > > >>> > On Wed, Mar 26, 2014 at 4:42 PM, Software Dev > >>> > <[email protected]> > >>> > wrote: > >>> >> > >>> >> How does one get the configuration map to each worker? > >>> >> > >>> >> On Wed, Mar 26, 2014 at 6:41 AM, Adam Lewis <[email protected]> > wrote: > >>> >> > Or, since this is only being called from prepare at startup > anyway, > >>> >> > simpler: > >>> >> > > >>> >> > public class InjectorProvider { > >>> >> > > >>> >> > private static Injector injector; > >>> >> > public static synchronized Injector get(Map conf) { > >>> >> > if (injector == null) { > >>> >> > injector = Guice.createInjector( > >>> >> > new DAOModule(conf), > >>> >> > new S3Module(conf)); > >>> >> > } > >>> >> > > >>> >> > return injector; > >>> >> > } > >>> >> > } > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > On Wed, Mar 26, 2014 at 9:26 AM, Svend Vanderveken > >>> >> > <[email protected]> wrote: > >>> >> >> > >>> >> >> > private static Injector injector; > >>> >> >> > >>> >> >> or better: > >>> >> >> > >>> >> >> private static volatile Injector injector; > >>> >> >> > >>> >> >> > >>> >> >> see > >>> >> >> > >>> >> >> > http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html > >>> >> >> > >>> >> >> > >>> >> >> On Tue, Mar 25, 2014 at 9:55 PM, Patricio Echagüe > >>> >> >> <[email protected]> > >>> >> >> wrote: > >>> >> >>> > >>> >> >>> It's fine. You can synchronize with a static monitor the > creation > >>> >> >>> on > >>> >> >>> the > >>> >> >>> injector per worker. That's how I do it. > >>> >> >>> > >>> >> >>> public class InjectorProvider { > >>> >> >>> > >>> >> >>> private static Injector injector; > >>> >> >>> > >>> >> >>> public static Injector get() { > >>> >> >>> if (injector == null) { > >>> >> >>> synchronized (InjectorProvider.class) { > >>> >> >>> if (injector == null) { > >>> >> >>> injector = Guice.createInjector( > >>> >> >>> new DAOModule(), > >>> >> >>> new S3Module(); > >>> >> >>> } > >>> >> >>> } > >>> >> >>> } > >>> >> >>> > >>> >> >>> return injector; > >>> >> >>> } > >>> >> >>> > >>> >> >>> > >>> >> >>> On Tue, Mar 25, 2014 at 6:24 PM, Adam Lewis <[email protected] > > > >>> >> >>> wrote: > >>> >> >>>> > >>> >> >>>> > >>> >> >>>>> Doesn't Storm 0.9 have a prepare for the worker? > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> No, I don't think it does, but please point this out if I'm > >>> >> >>>> mistaken. > >>> >> >>>> I > >>> >> >>>> found the right JIRA issue though: > >>> >> >>>> https://issues.apache.org/jira/browse/STORM-126 > >>> >> >>>> > >>> >> >>>> Seems like the patch was well along but hasn't seen any recent > >>> >> >>>> activity. > >>> >> >>> > >>> >> >>> > >>> >> >> > >>> >> > > >>> > > >>> > > >> > >> >
