I think you will need something like redis. Supun.. On May 5, 2015 8:17 PM, "Huy Le Van" <[email protected]> wrote:
> Could someone help me please? > > On Thursday, Apr 30, 2015 at 4:27 a.m., Huy Le Van < > [email protected]>, wrote: > >> Hi Michael, >> >> If a bolt of type A on a JVM modifies the shared data, will another bolt >> of type B on another JVM see the changes? To be clear, I have a use-case >> where I need multiple bolts of multiple types to be able to read/write on >> the shared collection. You can think of it as a shared database. In this >> case, can I still use this solution or do I need to use >> Hazelcast/Redis/Memcached? >> >> Best regards, >> Huy, Le Van >> >> On Thursday, Dec 4, 2014 at 8:53 p.m., Michael Rose < >> [email protected]>, wrote: >> >>> Yes, as long as they're in the same JVM. If you have 4 workers, that's 4 >>> JVMs, and 4 copies of the data. >>> >>> Storm will ensure that if you have 4 bolts of type A, and 8 of type B, >>> that 1 of type A and 2 of type B exist in each JVM. Since the singleton's >>> field is static, it'll be shared. >>> >>> Michael Rose (@Xorlev <https://twitter.com/xorlev>) >>> Senior Platform Engineer, FullContact <http://www.fullcontact.com/> >>> [email protected] >>> >>> On Thu, Dec 4, 2014 at 1:34 PM, Eyal Golan <[email protected]> wrote: >>> >>>> Michael, >>>> I understand your solution, however, if bolt of type A "wins" the >>>> initialization, will bolts of type B see the same instance? >>>> >>>> Sent from my Android phone >>>> On Dec 4, 2014 10:26 PM, "Eyal Golan" <[email protected]> wrote: >>>> >>>>> Thanks Michael and Ben ! >>>>> >>>>> Sent from my Android phone >>>>> On Dec 4, 2014 9:53 PM, "Flint, Ben" <[email protected]> wrote: >>>>> >>>>>> I’ve used Hazelcast for use cases like this with great success. It >>>>>> makes it relatively straightforward to implement a distributed data >>>>>> structure. >>>>>> >>>>>> From: Eyal Golan <[email protected]> >>>>>> Reply-To: "[email protected]" <[email protected]> >>>>>> Date: Thursday, December 4, 2014 at 11:40 AM >>>>>> To: "[email protected]" <[email protected]> >>>>>> Subject: Re: Shared static data between bolts >>>>>> >>>>>> Thanks! >>>>>> All the answers give me good idea. >>>>>> Here's why I need this list be sharable. >>>>>> One bolt needs the list for computation. >>>>>> Another bolt will periodically update the list. >>>>>> In other words, the list is changeable. >>>>>> >>>>>> I tried passing it in conf but there are some 3rd party libs in the >>>>>> wrapper. >>>>>> >>>>>> Anyway, I also need to improve the topology. Separate bolt to two >>>>>> etc. That way, it would be easier to handle that cache. >>>>>> >>>>>> One more question, I thought about giving hazelcast a try. >>>>>> Create local node and then the bolts will have clients. >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Sent from my Android phone >>>>>> On Dec 4, 2014 9:14 PM, "Kushan Maskey" < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Eyal, >>>>>>> >>>>>>> I had the similar issue earlier. Static objects are not a good way >>>>>>> of passing the variables around in Storm due to clustered nature of the >>>>>>> Storm. You will see null for all the static variables. >>>>>>> >>>>>>> This is how I implemented mine, for instance I wanted to pass around >>>>>>> the server properties such and host name and username/password. I read >>>>>>> the >>>>>>> properties file and then created a map of these properties then set it >>>>>>> into >>>>>>> the Config object. Like below. >>>>>>> >>>>>>> *In topology:* >>>>>>> >>>>>>> Map<Stirng, Stirng> map = new HashMap(); >>>>>>> map.put("hostname", "localhost"); >>>>>>> map.put("username", "testuser"); >>>>>>> map.put("password", "testpassord"); >>>>>>> map.put("port", "1234"); >>>>>>> >>>>>>> Config config = new Config(); >>>>>>> >>>>>>> config.put("MY_MAP", map) >>>>>>> *In Bolts:* >>>>>>> >>>>>>> public void prepare(Map stormConf, TopologyContext context) { >>>>>>> >>>>>>> HashMap<Stirng, Stirng> map = stormConfig.get("MY_MAP"); >>>>>>> >>>>>>> } >>>>>>> This how i resolved my problem.Try that if that helps. >>>>>>> -- >>>>>>> Kushan Maskey >>>>>>> 817.403.7500 >>>>>>> M. Miller & Associates <http://mmillerassociates.com/> >>>>>>> [email protected] >>>>>>> >>>>>>> On Thu, Dec 4, 2014 at 1:01 PM, Eyal Golan <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> I am newbie with Storm and having some issues with static objects >>>>>>>> in the topology. >>>>>>>> In the main class, where I set the topology, I also set a static >>>>>>>> object. It's actually a wrapper for a collection of strings. >>>>>>>> This object needs to he shared between different bolts. >>>>>>>> It's big so I want it to be only one instance. >>>>>>>> >>>>>>>> The problem is that when a bolt tries to get this static instance: >>>>>>>> example - MyMain.Shared... It's null. Although I instantiated it in the >>>>>>>> main method. >>>>>>>> >>>>>>>> I guess I'm doing something wrong. >>>>>>>> >>>>>>>> Thanks for any assistance. >>>>>>>> >>>>>>>> Eyal >>>>>>>> >>>>>>>> Sent from my Android phone >>>>>>>> >>>>>>> >>>>>>> >>>
