You can also use the power of Zookeeper to share data across JVM's, much like what storm and Kafka do.
Thanks and Regards, Devang On 6 May 2015 09:07, "Supun Kamburugamuva" <[email protected]> wrote: > I think you will need something like redis. > > Supun.. > On May 5, 2015 8:17 PM, "Huy Le Van" <[email protected]> wrote: > >> Could someone help me please? >> >> On Thursday, Apr 30, 2015 at 4:27 a.m., Huy Le Van < >> [email protected]>, wrote: >> >>> Hi Michael, >>> >>> If a bolt of type A on a JVM modifies the shared data, will another bolt >>> of type B on another JVM see the changes? To be clear, I have a use-case >>> where I need multiple bolts of multiple types to be able to read/write on >>> the shared collection. You can think of it as a shared database. In this >>> case, can I still use this solution or do I need to use >>> Hazelcast/Redis/Memcached? >>> >>> Best regards, >>> Huy, Le Van >>> >>> On Thursday, Dec 4, 2014 at 8:53 p.m., Michael Rose < >>> [email protected]>, wrote: >>> >>>> Yes, as long as they're in the same JVM. If you have 4 workers, that's >>>> 4 JVMs, and 4 copies of the data. >>>> >>>> Storm will ensure that if you have 4 bolts of type A, and 8 of type B, >>>> that 1 of type A and 2 of type B exist in each JVM. Since the singleton's >>>> field is static, it'll be shared. >>>> >>>> Michael Rose (@Xorlev <https://twitter.com/xorlev>) >>>> Senior Platform Engineer, FullContact <http://www.fullcontact.com/> >>>> [email protected] >>>> >>>> On Thu, Dec 4, 2014 at 1:34 PM, Eyal Golan <[email protected]> wrote: >>>> >>>>> Michael, >>>>> I understand your solution, however, if bolt of type A "wins" the >>>>> initialization, will bolts of type B see the same instance? >>>>> >>>>> Sent from my Android phone >>>>> On Dec 4, 2014 10:26 PM, "Eyal Golan" <[email protected]> wrote: >>>>> >>>>>> Thanks Michael and Ben ! >>>>>> >>>>>> Sent from my Android phone >>>>>> On Dec 4, 2014 9:53 PM, "Flint, Ben" <[email protected]> wrote: >>>>>> >>>>>>> I’ve used Hazelcast for use cases like this with great success. >>>>>>> It makes it relatively straightforward to implement a distributed data >>>>>>> structure. >>>>>>> >>>>>>> From: Eyal Golan <[email protected]> >>>>>>> Reply-To: "[email protected]" <[email protected]> >>>>>>> Date: Thursday, December 4, 2014 at 11:40 AM >>>>>>> To: "[email protected]" <[email protected]> >>>>>>> Subject: Re: Shared static data between bolts >>>>>>> >>>>>>> Thanks! >>>>>>> All the answers give me good idea. >>>>>>> Here's why I need this list be sharable. >>>>>>> One bolt needs the list for computation. >>>>>>> Another bolt will periodically update the list. >>>>>>> In other words, the list is changeable. >>>>>>> >>>>>>> I tried passing it in conf but there are some 3rd party libs in the >>>>>>> wrapper. >>>>>>> >>>>>>> Anyway, I also need to improve the topology. Separate bolt to two >>>>>>> etc. That way, it would be easier to handle that cache. >>>>>>> >>>>>>> One more question, I thought about giving hazelcast a try. >>>>>>> Create local node and then the bolts will have clients. >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Sent from my Android phone >>>>>>> On Dec 4, 2014 9:14 PM, "Kushan Maskey" < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Eyal, >>>>>>>> >>>>>>>> I had the similar issue earlier. Static objects are not a good way >>>>>>>> of passing the variables around in Storm due to clustered nature of the >>>>>>>> Storm. You will see null for all the static variables. >>>>>>>> >>>>>>>> This is how I implemented mine, for instance I wanted to pass >>>>>>>> around the server properties such and host name and username/password. >>>>>>>> I >>>>>>>> read the properties file and then created a map of these properties >>>>>>>> then >>>>>>>> set it into the Config object. Like below. >>>>>>>> >>>>>>>> *In topology:* >>>>>>>> >>>>>>>> Map<Stirng, Stirng> map = new HashMap(); >>>>>>>> map.put("hostname", "localhost"); >>>>>>>> map.put("username", "testuser"); >>>>>>>> map.put("password", "testpassord"); >>>>>>>> map.put("port", "1234"); >>>>>>>> >>>>>>>> Config config = new Config(); >>>>>>>> >>>>>>>> config.put("MY_MAP", map) >>>>>>>> *In Bolts:* >>>>>>>> >>>>>>>> public void prepare(Map stormConf, TopologyContext context) { >>>>>>>> >>>>>>>> HashMap<Stirng, Stirng> map = stormConfig.get("MY_MAP"); >>>>>>>> >>>>>>>> } >>>>>>>> This how i resolved my problem.Try that if that helps. >>>>>>>> -- >>>>>>>> Kushan Maskey >>>>>>>> 817.403.7500 >>>>>>>> M. Miller & Associates <http://mmillerassociates.com/> >>>>>>>> [email protected] >>>>>>>> >>>>>>>> On Thu, Dec 4, 2014 at 1:01 PM, Eyal Golan <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> I am newbie with Storm and having some issues with static objects >>>>>>>>> in the topology. >>>>>>>>> In the main class, where I set the topology, I also set a static >>>>>>>>> object. It's actually a wrapper for a collection of strings. >>>>>>>>> This object needs to he shared between different bolts. >>>>>>>>> It's big so I want it to be only one instance. >>>>>>>>> >>>>>>>>> The problem is that when a bolt tries to get this static instance: >>>>>>>>> example - MyMain.Shared... It's null. Although I instantiated it in >>>>>>>>> the >>>>>>>>> main method. >>>>>>>>> >>>>>>>>> I guess I'm doing something wrong. >>>>>>>>> >>>>>>>>> Thanks for any assistance. >>>>>>>>> >>>>>>>>> Eyal >>>>>>>>> >>>>>>>>> Sent from my Android phone >>>>>>>>> >>>>>>>> >>>>>>>> >>>>
