Thanks again
On Fri, Mar 28, 2014 at 6:49 AM, Adam Lewis <[email protected]> wrote: > I don't think you are necessarily limited to one approach. Personally, I do > something more like the former, although rather than relying on putting > custom properties in the storm conf, I just pass them as arguments to > topology component constructors...although this may not be best practice. > > The key is that the injector isn't going to be serializable; and in my case > the object graph I build with the injector is also not serializable, so that > entire construct has to be built once the topology is thrown "over the wall" > to the workers. Think of all the pre-serialization parameters as > "configuration" (either via StormConf or a more pojo based approach) and > post-serialization objects as true dependencies that can acquire and hold > resources from the target environment, etc. > > There is nothing stopping you from using DI to build the topology components > pre-serialization if all the injected values are serializable, although > doing both pre and post serialization DI is probably going to get hairy as > there is no clear way to tell guice where to "stop" injecting the object > graph. > > It really comes down to what you're trying to achieve. I hope this helps. > > > On Thu, Mar 27, 2014 at 7:23 PM, Software Dev <[email protected]> > wrote: >> >> Actually.. some more questions. We are running 0.9.1-incubating so we >> do in fact have the prepare method. >> >> Confused on.. >> >> Should I add all configuration values (from a properties file) onto >> the StormConf and then in the prepare method of my workers just >> instantiate an Injector? >> >> Or... >> >> Should I instantiate an injector right away that is used to >> instantiate some factory classes that will create bolts with primitive >> arguments? >> >> On Thu, Mar 27, 2014 at 10:13 AM, Software Dev >> <[email protected]> wrote: >> > Thanks. I think that cleared up most of my misunderstanding. >> > >> > On Thu, Mar 27, 2014 at 6:16 AM, Adam Lewis <[email protected]> wrote: >> >> Yes that is exactly right, the submission to Nimbus is in the form of a >> >> big >> >> thrift message describing the topology...this message includes java >> >> serialized blobs of your topology components (spouts/bolts). They get >> >> instantiated within the VM calling StormSubmitter. Typically you would >> >> pass >> >> configuration info to the constructor, but "dependencies" (e.g. DB >> >> connection pool, etc) are transient fields. Then in the prepare method >> >> (called after deserialization on the worker) you use the serialized >> >> configuration fields to initialize the transient ones. Of course Guice >> >> fits >> >> naturally into that step. >> >> >> >> >> >> On Thu, Mar 27, 2014 at 12:37 AM, Software Dev >> >> <[email protected]> >> >> wrote: >> >>> >> >>> Ok so you would configure the map in the main method before submitting >> >>> the topology. Then this conf can be used to create guice injectors. Is >> >>> that correct? >> >>> >> >>> In the book "Getting Started with Storm" it states: >> >>> >> >>> "To customize a bolt, you should set parameters in its constructor and >> >>> save them as instance variables so they will be serialized when >> >>> submitting the bolt to the cluster." >> >>> >> >>> Does this mean bolts are instantiated on the client side before being >> >>> submitted to nimbus/cluster? >> >>> >> >>> On Wed, Mar 26, 2014 at 2:05 PM, Svend Vanderveken >> >>> <[email protected]> wrote: >> >>> > >> >>> > The storm configuration map is part of the arguments received by >> >>> > each >> >>> > prepare() method, in most Storm primitives, on each worker. It's >> >>> > serialised >> >>> > to each worker when a topology instance is started there. The >> >>> > initial >> >>> > storm >> >>> > configuration map is provided at deploy time to Nimbus, in the class >> >>> > containing the main() method, specified in the "storm jar blabla.jar >> >>> > some.class.here" command. >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > On Wed, Mar 26, 2014 at 4:42 PM, Software Dev >> >>> > <[email protected]> >> >>> > wrote: >> >>> >> >> >>> >> How does one get the configuration map to each worker? >> >>> >> >> >>> >> On Wed, Mar 26, 2014 at 6:41 AM, Adam Lewis <[email protected]> >> >>> >> wrote: >> >>> >> > Or, since this is only being called from prepare at startup >> >>> >> > anyway, >> >>> >> > simpler: >> >>> >> > >> >>> >> > public class InjectorProvider { >> >>> >> > >> >>> >> > private static Injector injector; >> >>> >> > public static synchronized Injector get(Map conf) { >> >>> >> > if (injector == null) { >> >>> >> > injector = Guice.createInjector( >> >>> >> > new DAOModule(conf), >> >>> >> > new S3Module(conf)); >> >>> >> > } >> >>> >> > >> >>> >> > return injector; >> >>> >> > } >> >>> >> > } >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > On Wed, Mar 26, 2014 at 9:26 AM, Svend Vanderveken >> >>> >> > <[email protected]> wrote: >> >>> >> >> >> >>> >> >> > private static Injector injector; >> >>> >> >> >> >>> >> >> or better: >> >>> >> >> >> >>> >> >> private static volatile Injector injector; >> >>> >> >> >> >>> >> >> >> >>> >> >> see >> >>> >> >> >> >>> >> >> >> >>> >> >> http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html >> >>> >> >> >> >>> >> >> >> >>> >> >> On Tue, Mar 25, 2014 at 9:55 PM, Patricio Echagüe >> >>> >> >> <[email protected]> >> >>> >> >> wrote: >> >>> >> >>> >> >>> >> >>> It's fine. You can synchronize with a static monitor the >> >>> >> >>> creation >> >>> >> >>> on >> >>> >> >>> the >> >>> >> >>> injector per worker. That's how I do it. >> >>> >> >>> >> >>> >> >>> public class InjectorProvider { >> >>> >> >>> >> >>> >> >>> private static Injector injector; >> >>> >> >>> >> >>> >> >>> public static Injector get() { >> >>> >> >>> if (injector == null) { >> >>> >> >>> synchronized (InjectorProvider.class) { >> >>> >> >>> if (injector == null) { >> >>> >> >>> injector = Guice.createInjector( >> >>> >> >>> new DAOModule(), >> >>> >> >>> new S3Module(); >> >>> >> >>> } >> >>> >> >>> } >> >>> >> >>> } >> >>> >> >>> >> >>> >> >>> return injector; >> >>> >> >>> } >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On Tue, Mar 25, 2014 at 6:24 PM, Adam Lewis >> >>> >> >>> <[email protected]> >> >>> >> >>> wrote: >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>>> Doesn't Storm 0.9 have a prepare for the worker? >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>> No, I don't think it does, but please point this out if I'm >> >>> >> >>>> mistaken. >> >>> >> >>>> I >> >>> >> >>>> found the right JIRA issue though: >> >>> >> >>>> https://issues.apache.org/jira/browse/STORM-126 >> >>> >> >>>> >> >>> >> >>>> Seems like the patch was well along but hasn't seen any recent >> >>> >> >>>> activity. >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >> >>> >> > >> >>> > >> >>> > >> >> >> >> > >
